2023-07-06T19:42:49.533 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-06T19:42:49.534 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} --archive /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148 --verbose --owner scheduled_rishabh@teuthology --name rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi -- /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148/orig.config.yaml 2023-07-06T19:42:49.593 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-06T19:42:49.686 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148 branch: wip-rishabh-2023Jul5 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} email: null first_in_suite: false job_id: '7328148' kernel: client: branch: testing flavor: default kdb: true sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul5 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul5 sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul5 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi098.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi156.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 500 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b53f236c1077f1d295baafb9e0ca9a4b447019d7 timestamp: 2023-07-06_19:33:28 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-06T19:42:49.687 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa; will attempt to use it 2023-07-06T19:42:49.688 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks 2023-07-06T19:42:49.688 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-06T19:42:49.690 INFO:teuthology.task.internal:Checking packages... 2023-07-06T19:42:49.718 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '79365bb3b803780276d69efb2cc4afe5b160de9e' 2023-07-06T19:42:49.718 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-06T19:42:49.718 INFO:teuthology.packaging:ref: None 2023-07-06T19:42:49.719 INFO:teuthology.packaging:tag: None 2023-07-06T19:42:49.719 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul5 2023-07-06T19:42:49.719 INFO:teuthology.packaging:sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T19:42:49.719 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul5 2023-07-06T19:42:49.878 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4742.g79365bb3 2023-07-06T19:42:49.878 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-06T19:42:49.885 INFO:teuthology.task.internal:no buildpackages task found 2023-07-06T19:42:49.885 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-06T19:42:49.909 INFO:teuthology.task.internal:Saving configuration 2023-07-06T19:42:49.930 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-06T19:42:49.936 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T19:42:49.961 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 19:34:37.258375', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-06T19:42:49.988 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 19:34:37.259548', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-06T19:42:50.011 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 19:34:37.257099', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-06T19:42:50.011 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-06T19:42:50.017 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-06T19:42:50.018 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-06T19:42:50.018 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-06T19:42:50.018 INFO:teuthology.run_tasks:Running task console_log... 2023-07-06T19:42:50.149 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0528b023a0>, signals=[15]) 2023-07-06T19:42:50.150 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-06T19:42:50.157 INFO:teuthology.task.internal:Opening connections... 2023-07-06T19:42:50.157 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2023-07-06T19:42:50.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:42:50.225 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2023-07-06T19:42:50.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:42:50.293 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2023-07-06T19:42:50.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:42:50.360 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-06T19:42:50.367 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2023-07-06T19:42:50.388 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2023-07-06T19:42:50.388 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2023-07-06T19:42:50.447 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="Red Hat Enterprise Linux" 2023-07-06T19:42:50.447 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="8.6 (Ootpa)" 2023-07-06T19:42:50.447 INFO:teuthology.orchestra.run.smithi069.stdout:ID="rhel" 2023-07-06T19:42:50.447 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="fedora" 2023-07-06T19:42:50.448 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="8.6" 2023-07-06T19:42:50.448 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el8" 2023-07-06T19:42:50.448 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-06T19:42:50.448 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2023-07-06T19:42:50.448 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-06T19:42:50.448 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://www.redhat.com/" 2023-07-06T19:42:50.448 INFO:teuthology.orchestra.run.smithi069.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-06T19:42:50.449 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T19:42:50.449 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T19:42:50.449 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T19:42:50.449 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-06T19:42:50.449 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-06T19:42:50.449 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-06T19:42:50.450 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2023-07-06T19:42:50.479 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2023-07-06T19:42:50.499 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2023-07-06T19:42:50.500 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2023-07-06T19:42:50.558 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="Red Hat Enterprise Linux" 2023-07-06T19:42:50.559 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8.6 (Ootpa)" 2023-07-06T19:42:50.559 INFO:teuthology.orchestra.run.smithi098.stdout:ID="rhel" 2023-07-06T19:42:50.559 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="fedora" 2023-07-06T19:42:50.559 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8.6" 2023-07-06T19:42:50.559 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2023-07-06T19:42:50.559 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-06T19:42:50.560 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2023-07-06T19:42:50.560 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-06T19:42:50.560 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.redhat.com/" 2023-07-06T19:42:50.560 INFO:teuthology.orchestra.run.smithi098.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-06T19:42:50.560 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T19:42:50.560 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T19:42:50.561 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T19:42:50.561 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-06T19:42:50.561 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-06T19:42:50.561 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-06T19:42:50.561 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2023-07-06T19:42:50.588 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2023-07-06T19:42:50.608 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2023-07-06T19:42:50.608 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2023-07-06T19:42:50.667 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Red Hat Enterprise Linux" 2023-07-06T19:42:50.667 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="8.6 (Ootpa)" 2023-07-06T19:42:50.667 INFO:teuthology.orchestra.run.smithi156.stdout:ID="rhel" 2023-07-06T19:42:50.667 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="fedora" 2023-07-06T19:42:50.667 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="8.6" 2023-07-06T19:42:50.668 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el8" 2023-07-06T19:42:50.668 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-06T19:42:50.668 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2023-07-06T19:42:50.668 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-06T19:42:50.668 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.redhat.com/" 2023-07-06T19:42:50.668 INFO:teuthology.orchestra.run.smithi156.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-06T19:42:50.669 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T19:42:50.669 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T19:42:50.669 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T19:42:50.669 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-06T19:42:50.669 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-06T19:42:50.669 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-06T19:42:50.670 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-07-06T19:42:50.696 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-06T19:42:50.707 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-06T19:42:50.734 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-06T19:42:50.734 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2023-07-06T19:42:50.736 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2023-07-06T19:42:50.738 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2023-07-06T19:42:50.757 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-06T19:42:50.767 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-06T19:42:50.767 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2023-07-06T19:42:50.795 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2023-07-06T19:42:50.797 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2023-07-06T19:42:50.823 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-06T19:42:50.913 INFO:teuthology.run_tasks:Running task kernel... 2023-07-06T19:42:50.928 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'} 2023-07-06T19:42:50.929 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-07-06T19:42:50.929 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-07-06T19:42:50.930 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T19:42:50.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T19:42:50.949 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-07-06T19:42:51.263 DEBUG:teuthology.task.kernel:sha1 for client.0 is 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T19:42:51.263 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "84139c18d56f518615369e8310ae4ecf07a36dcd"... 2023-07-06T19:42:51.263 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2023-07-06T19:42:51.283 INFO:teuthology.orchestra.run.smithi069.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-06T19:42:51.283 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T19:42:51.283 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-06T19:42:51.284 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum upgrade -y linux-firmware 2023-07-06T19:42:52.515 INFO:teuthology.orchestra.run.smithi069.stdout:Updating Subscription Management repositories. 2023-07-06T19:42:52.516 INFO:teuthology.orchestra.run.smithi069.stdout:Unable to read consumer identity 2023-07-06T19:42:56.486 INFO:teuthology.orchestra.run.smithi069.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T19:43:01.192 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 3.3 MB/s | 16 MB 00:04 2023-07-06T19:43:04.686 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 378 kB/s | 24 kB 00:00 2023-07-06T19:43:05.427 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2023-07-06T19:43:05.428 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2023-07-06T19:43:05.428 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2023-07-06T19:43:05.549 INFO:teuthology.task.kernel:Downloading kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on client.0... 2023-07-06T19:43:05.550 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T19:43:05.765 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ 2023-07-06T19:43:05.765 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.4.0_g84139c18d56f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-06T19:43:05.768 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi069.front.sepia.ceph.com to complete... 2023-07-06T19:43:25.387 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06 19:43:25 URL:https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/kernel-6.4.0_g84139c18d56f-1.x86_64.rpm [379858600/379858600] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-06T19:43:25.388 INFO:teuthology.orchestra.run.smithi069.stderr:FINISHED --2023-07-06 19:43:25-- 2023-07-06T19:43:25.388 INFO:teuthology.orchestra.run.smithi069.stderr:Total wall clock time: 20s 2023-07-06T19:43:25.388 INFO:teuthology.orchestra.run.smithi069.stderr:Downloaded: 1 files, 362M in 19s (18.7 MB/s) 2023-07-06T19:43:25.389 INFO:teuthology.task.kernel:Installing kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on client.0... 2023-07-06T19:43:25.389 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-06T19:43:26.964 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying... ######################################## 2023-07-06T19:43:27.018 INFO:teuthology.orchestra.run.smithi069.stdout:Preparing... ######################################## 2023-07-06T19:43:27.122 INFO:teuthology.orchestra.run.smithi069.stdout:Updating / installing... 2023-07-06T19:44:04.914 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-6.4.0_g84139c18d56f-1 ######################################## 2023-07-06T19:44:29.047 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi069.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-06T19:44:29.048 DEBUG:teuthology.orchestra.run.smithi069:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-06T19:44:29.099 INFO:teuthology.orchestra.run.smithi069.stdout:/boot/System.map-6.4.0-g84139c18d56f 2023-07-06T19:44:29.100 INFO:teuthology.orchestra.run.smithi069.stdout:/boot/config-6.4.0-g84139c18d56f 2023-07-06T19:44:29.100 INFO:teuthology.orchestra.run.smithi069.stdout:/boot/vmlinuz-6.4.0-g84139c18d56f 2023-07-06T19:44:29.100 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f 2023-07-06T19:44:29.100 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel 2023-07-06T19:44:29.100 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch 2023-07-06T19:44:29.100 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86 2023-07-06T19:44:29.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto 2023-07-06T19:44:29.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-06T19:44:29.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-06T19:44:29.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-06T19:44:29.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-06T19:44:29.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-06T19:44:29.101 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-06T19:44:29.102 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-06T19:44:29.102 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-06T19:44:29.102 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-06T19:44:29.102 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-06T19:44:29.102 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-06T19:44:29.102 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-06T19:44:29.102 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-06T19:44:29.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-06T19:44:29.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-06T19:44:29.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-06T19:44:29.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-06T19:44:29.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-06T19:44:29.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-06T19:44:29.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-06T19:44:29.103 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel 2023-07-06T19:44:29.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu 2023-07-06T19:44:29.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce 2023-07-06T19:44:29.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-06T19:44:29.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm 2023-07-06T19:44:29.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-06T19:44:29.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-06T19:44:29.104 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm.ko 2023-07-06T19:44:29.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto 2023-07-06T19:44:29.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ansi_cprng.ko 2023-07-06T19:44:29.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/anubis.ko 2023-07-06T19:44:29.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/arc4.ko 2023-07-06T19:44:29.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx 2023-07-06T19:44:29.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_memcpy.ko 2023-07-06T19:44:29.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_pq.ko 2023-07-06T19:44:29.105 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-06T19:44:29.106 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_tx.ko 2023-07-06T19:44:29.106 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_xor.ko 2023-07-06T19:44:29.106 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/raid6test.ko 2023-07-06T19:44:29.106 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authenc.ko 2023-07-06T19:44:29.106 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authencesn.ko 2023-07-06T19:44:29.106 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blake2b_generic.ko 2023-07-06T19:44:29.106 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_common.ko 2023-07-06T19:44:29.107 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_generic.ko 2023-07-06T19:44:29.107 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/camellia_generic.ko 2023-07-06T19:44:29.107 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast5_generic.ko 2023-07-06T19:44:29.107 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast6_generic.ko 2023-07-06T19:44:29.107 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast_common.ko 2023-07-06T19:44:29.107 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ccm.ko 2023-07-06T19:44:29.107 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cmac.ko 2023-07-06T19:44:29.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crc32_generic.ko 2023-07-06T19:44:29.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_engine.ko 2023-07-06T19:44:29.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_user.ko 2023-07-06T19:44:29.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cts.ko 2023-07-06T19:44:29.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/des_generic.ko 2023-07-06T19:44:29.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/dh_generic.ko 2023-07-06T19:44:29.108 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/echainiv.ko 2023-07-06T19:44:29.109 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/essiv.ko 2023-07-06T19:44:29.109 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/fcrypt.ko 2023-07-06T19:44:29.109 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/khazad.ko 2023-07-06T19:44:29.109 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4.ko 2023-07-06T19:44:29.109 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4hc.ko 2023-07-06T19:44:29.109 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/md4.ko 2023-07-06T19:44:29.109 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/michael_mic.ko 2023-07-06T19:44:29.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcbc.ko 2023-07-06T19:44:29.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcrypt.ko 2023-07-06T19:44:29.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/rmd160.ko 2023-07-06T19:44:29.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/seed.ko 2023-07-06T19:44:29.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/serpent_generic.ko 2023-07-06T19:44:29.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tcrypt.ko 2023-07-06T19:44:29.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tea.ko 2023-07-06T19:44:29.110 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_common.ko 2023-07-06T19:44:29.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_generic.ko 2023-07-06T19:44:29.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/vmac.ko 2023-07-06T19:44:29.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/wp512.ko 2023-07-06T19:44:29.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xcbc.ko 2023-07-06T19:44:29.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xor.ko 2023-07-06T19:44:29.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xxhash_generic.ko 2023-07-06T19:44:29.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers 2023-07-06T19:44:29.111 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi 2023-07-06T19:44:29.112 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-06T19:44:29.112 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_pad.ko 2023-07-06T19:44:29.112 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/custom_method.ko 2023-07-06T19:44:29.112 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/ec_sys.ko 2023-07-06T19:44:29.112 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/platform_profile.ko 2023-07-06T19:44:29.112 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbs.ko 2023-07-06T19:44:29.112 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbshc.ko 2023-07-06T19:44:29.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/video.ko 2023-07-06T19:44:29.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata 2023-07-06T19:44:29.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/acard-ahci.ko 2023-07-06T19:44:29.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ahci_platform.ko 2023-07-06T19:44:29.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ata_generic.ko 2023-07-06T19:44:29.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/libahci_platform.ko 2023-07-06T19:44:29.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_acpi.ko 2023-07-06T19:44:29.113 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ali.ko 2023-07-06T19:44:29.114 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_amd.ko 2023-07-06T19:44:29.114 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_artop.ko 2023-07-06T19:44:29.114 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atiixp.ko 2023-07-06T19:44:29.114 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atp867x.ko 2023-07-06T19:44:29.114 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd640.ko 2023-07-06T19:44:29.114 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd64x.ko 2023-07-06T19:44:29.114 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cypress.ko 2023-07-06T19:44:29.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_efar.ko 2023-07-06T19:44:29.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt366.ko 2023-07-06T19:44:29.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt37x.ko 2023-07-06T19:44:29.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-06T19:44:29.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-06T19:44:29.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it8213.ko 2023-07-06T19:44:29.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it821x.ko 2023-07-06T19:44:29.115 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_jmicron.ko 2023-07-06T19:44:29.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_marvell.ko 2023-07-06T19:44:29.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_mpiix.ko 2023-07-06T19:44:29.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_netcell.ko 2023-07-06T19:44:29.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ninja32.ko 2023-07-06T19:44:29.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87410.ko 2023-07-06T19:44:29.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87415.ko 2023-07-06T19:44:29.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_oldpiix.ko 2023-07-06T19:44:29.116 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_opti.ko 2023-07-06T19:44:29.117 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_optidma.ko 2023-07-06T19:44:29.117 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pcmcia.ko 2023-07-06T19:44:29.117 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-06T19:44:29.117 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-06T19:44:29.117 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_piccolo.ko 2023-07-06T19:44:29.117 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_rdc.ko 2023-07-06T19:44:29.117 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sch.ko 2023-07-06T19:44:29.118 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_serverworks.ko 2023-07-06T19:44:29.118 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sil680.ko 2023-07-06T19:44:29.118 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sis.ko 2023-07-06T19:44:29.118 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sl82c105.ko 2023-07-06T19:44:29.118 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_triflex.ko 2023-07-06T19:44:29.118 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_via.ko 2023-07-06T19:44:29.118 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pdc_adma.ko 2023-07-06T19:44:29.118 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_inic162x.ko 2023-07-06T19:44:29.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_mv.ko 2023-07-06T19:44:29.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_nv.ko 2023-07-06T19:44:29.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_promise.ko 2023-07-06T19:44:29.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_qstor.ko 2023-07-06T19:44:29.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil.ko 2023-07-06T19:44:29.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil24.ko 2023-07-06T19:44:29.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sis.ko 2023-07-06T19:44:29.119 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_svw.ko 2023-07-06T19:44:29.120 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sx4.ko 2023-07-06T19:44:29.120 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_uli.ko 2023-07-06T19:44:29.120 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_via.ko 2023-07-06T19:44:29.120 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_vsc.ko 2023-07-06T19:44:29.120 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay 2023-07-06T19:44:29.120 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay/ks0108.ko 2023-07-06T19:44:29.120 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base 2023-07-06T19:44:29.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap 2023-07-06T19:44:29.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-06T19:44:29.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma 2023-07-06T19:44:29.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma/bcma.ko 2023-07-06T19:44:29.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block 2023-07-06T19:44:29.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe 2023-07-06T19:44:29.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe/aoe.ko 2023-07-06T19:44:29.121 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/brd.ko 2023-07-06T19:44:29.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd 2023-07-06T19:44:29.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd/drbd.ko 2023-07-06T19:44:29.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/floppy.ko 2023-07-06T19:44:29.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/loop.ko 2023-07-06T19:44:29.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx 2023-07-06T19:44:29.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-06T19:44:29.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/nbd.ko 2023-07-06T19:44:29.122 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk 2023-07-06T19:44:29.123 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk/null_blk.ko 2023-07-06T19:44:29.123 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/pktcdvd.ko 2023-07-06T19:44:29.123 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/rbd.ko 2023-07-06T19:44:29.123 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/virtio_blk.ko 2023-07-06T19:44:29.123 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback 2023-07-06T19:44:29.123 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-06T19:44:29.123 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkfront.ko 2023-07-06T19:44:29.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram 2023-07-06T19:44:29.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram/zram.ko 2023-07-06T19:44:29.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char 2023-07-06T19:44:29.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hangcheck-timer.ko 2023-07-06T19:44:29.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random 2023-07-06T19:44:29.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-06T19:44:29.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-06T19:44:29.124 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-06T19:44:29.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/via-rng.ko 2023-07-06T19:44:29.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-06T19:44:29.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi 2023-07-06T19:44:29.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-06T19:44:29.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-06T19:44:29.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-06T19:44:29.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-06T19:44:29.125 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-06T19:44:29.126 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-06T19:44:29.126 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/lp.ko 2023-07-06T19:44:29.126 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave 2023-07-06T19:44:29.126 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave/mwave.ko 2023-07-06T19:44:29.126 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ppdev.ko 2023-07-06T19:44:29.126 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tlclk.ko 2023-07-06T19:44:29.126 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm 2023-07-06T19:44:29.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm.ko 2023-07-06T19:44:29.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-06T19:44:29.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-06T19:44:29.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-06T19:44:29.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-06T19:44:29.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-06T19:44:29.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/uv_mmtimer.ko 2023-07-06T19:44:29.127 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/virtio_console.ko 2023-07-06T19:44:29.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq 2023-07-06T19:44:29.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-06T19:44:29.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-06T19:44:29.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-06T19:44:29.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-06T19:44:29.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-06T19:44:29.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-06T19:44:29.128 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto 2023-07-06T19:44:29.129 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp 2023-07-06T19:44:29.129 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-06T19:44:29.129 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp.ko 2023-07-06T19:44:29.129 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel 2023-07-06T19:44:29.129 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat 2023-07-06T19:44:29.129 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common 2023-07-06T19:44:29.129 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-06T19:44:29.129 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-06T19:44:29.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-06T19:44:29.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-aes.ko 2023-07-06T19:44:29.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-sha.ko 2023-07-06T19:44:29.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio 2023-07-06T19:44:29.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-06T19:44:29.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca 2023-07-06T19:44:29.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca/dca.ko 2023-07-06T19:44:29.130 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq 2023-07-06T19:44:29.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-06T19:44:29.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma 2023-07-06T19:44:29.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw 2023-07-06T19:44:29.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-06T19:44:29.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat 2023-07-06T19:44:29.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-06T19:44:29.131 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac 2023-07-06T19:44:29.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/amd64_edac.ko 2023-07-06T19:44:29.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/e752x_edac.ko 2023-07-06T19:44:29.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/edac_mce_amd.ko 2023-07-06T19:44:29.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3000_edac.ko 2023-07-06T19:44:29.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3200_edac.ko 2023-07-06T19:44:29.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5100_edac.ko 2023-07-06T19:44:29.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5400_edac.ko 2023-07-06T19:44:29.132 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7300_edac.ko 2023-07-06T19:44:29.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7core_edac.ko 2023-07-06T19:44:29.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i82975x_edac.ko 2023-07-06T19:44:29.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/ie31200_edac.ko 2023-07-06T19:44:29.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/sb_edac.ko 2023-07-06T19:44:29.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/x38_edac.ko 2023-07-06T19:44:29.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware 2023-07-06T19:44:29.133 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/edd.ko 2023-07-06T19:44:29.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-06T19:44:29.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio 2023-07-06T19:44:29.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-ich.ko 2023-07-06T19:44:29.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-06T19:44:29.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid 2023-07-06T19:44:29.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-appleir.ko 2023-07-06T19:44:29.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-aureal.ko 2023-07-06T19:44:29.134 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-axff.ko 2023-07-06T19:44:29.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-dr.ko 2023-07-06T19:44:29.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elecom.ko 2023-07-06T19:44:29.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elo.ko 2023-07-06T19:44:29.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-emsff.ko 2023-07-06T19:44:29.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gaff.ko 2023-07-06T19:44:29.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gt683r.ko 2023-07-06T19:44:29.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gyration.ko 2023-07-06T19:44:29.135 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-06T19:44:29.136 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-06T19:44:29.136 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtekff.ko 2023-07-06T19:44:29.136 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-hyperv.ko 2023-07-06T19:44:29.136 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-icade.ko 2023-07-06T19:44:29.136 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-keytouch.ko 2023-07-06T19:44:29.136 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-kye.ko 2023-07-06T19:44:29.136 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lcpower.ko 2023-07-06T19:44:29.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-led.ko 2023-07-06T19:44:29.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lenovo.ko 2023-07-06T19:44:29.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-06T19:44:29.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-06T19:44:29.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-multitouch.ko 2023-07-06T19:44:29.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-ortek.ko 2023-07-06T19:44:29.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-penmount.ko 2023-07-06T19:44:29.137 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-petalynx.ko 2023-07-06T19:44:29.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-picolcd.ko 2023-07-06T19:44:29.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-pl.ko 2023-07-06T19:44:29.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-plantronics.ko 2023-07-06T19:44:29.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-primax.ko 2023-07-06T19:44:29.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-rmi.ko 2023-07-06T19:44:29.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-06T19:44:29.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-common.ko 2023-07-06T19:44:29.138 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-06T19:44:29.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-06T19:44:29.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-06T19:44:29.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-06T19:44:29.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-06T19:44:29.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-06T19:44:29.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-06T19:44:29.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-06T19:44:29.139 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-06T19:44:29.140 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat.ko 2023-07-06T19:44:29.140 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-saitek.ko 2023-07-06T19:44:29.140 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-samsung.ko 2023-07-06T19:44:29.140 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-06T19:44:29.140 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sjoy.ko 2023-07-06T19:44:29.140 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sony.ko 2023-07-06T19:44:29.140 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-speedlink.ko 2023-07-06T19:44:29.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-steelseries.ko 2023-07-06T19:44:29.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sunplus.ko 2023-07-06T19:44:29.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-06T19:44:29.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tivo.ko 2023-07-06T19:44:29.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tmff.ko 2023-07-06T19:44:29.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-topseed.ko 2023-07-06T19:44:29.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-twinhan.ko 2023-07-06T19:44:29.141 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-uclogic.ko 2023-07-06T19:44:29.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-waltop.ko 2023-07-06T19:44:29.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-wiimote.ko 2023-07-06T19:44:29.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-xinmo.ko 2023-07-06T19:44:29.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zpff.ko 2023-07-06T19:44:29.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zydacron.ko 2023-07-06T19:44:29.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/uhid.ko 2023-07-06T19:44:29.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/wacom.ko 2023-07-06T19:44:29.142 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv 2023-07-06T19:44:29.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_balloon.ko 2023-07-06T19:44:29.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_utils.ko 2023-07-06T19:44:29.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_vmbus.ko 2023-07-06T19:44:29.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon 2023-07-06T19:44:29.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru.ko 2023-07-06T19:44:29.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru3.ko 2023-07-06T19:44:29.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-06T19:44:29.143 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7414.ko 2023-07-06T19:44:29.144 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7418.ko 2023-07-06T19:44:29.144 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adc128d818.ko 2023-07-06T19:44:29.144 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1025.ko 2023-07-06T19:44:29.144 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1026.ko 2023-07-06T19:44:29.144 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1029.ko 2023-07-06T19:44:29.144 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1031.ko 2023-07-06T19:44:29.144 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm9240.ko 2023-07-06T19:44:29.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ads7828.ko 2023-07-06T19:44:29.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7410.ko 2023-07-06T19:44:29.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7411.ko 2023-07-06T19:44:29.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7462.ko 2023-07-06T19:44:29.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7470.ko 2023-07-06T19:44:29.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7475.ko 2023-07-06T19:44:29.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7x10.ko 2023-07-06T19:44:29.145 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/amc6821.ko 2023-07-06T19:44:29.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/applesmc.ko 2023-07-06T19:44:29.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asb100.ko 2023-07-06T19:44:29.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asc7621.ko 2023-07-06T19:44:29.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-06T19:44:29.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/atxp1.ko 2023-07-06T19:44:29.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/coretemp.ko 2023-07-06T19:44:29.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-06T19:44:29.146 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dme1737.ko 2023-07-06T19:44:29.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds1621.ko 2023-07-06T19:44:29.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds620.ko 2023-07-06T19:44:29.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc1403.ko 2023-07-06T19:44:29.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc6w201.ko 2023-07-06T19:44:29.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71805f.ko 2023-07-06T19:44:29.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71882fg.ko 2023-07-06T19:44:29.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f75375s.ko 2023-07-06T19:44:29.147 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fam15h_power.ko 2023-07-06T19:44:29.148 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fschmd.ko 2023-07-06T19:44:29.148 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g760a.ko 2023-07-06T19:44:29.148 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g762.ko 2023-07-06T19:44:29.148 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl518sm.ko 2023-07-06T19:44:29.148 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl520sm.ko 2023-07-06T19:44:29.148 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-06T19:44:29.148 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5500_temp.ko 2023-07-06T19:44:29.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5k_amb.ko 2023-07-06T19:44:29.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmaem.ko 2023-07-06T19:44:29.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmpex.ko 2023-07-06T19:44:29.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina209.ko 2023-07-06T19:44:29.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina2xx.ko 2023-07-06T19:44:29.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/it87.ko 2023-07-06T19:44:29.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k10temp.ko 2023-07-06T19:44:29.149 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k8temp.ko 2023-07-06T19:44:29.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lineage-pem.ko 2023-07-06T19:44:29.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm63.ko 2023-07-06T19:44:29.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm73.ko 2023-07-06T19:44:29.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm75.ko 2023-07-06T19:44:29.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm77.ko 2023-07-06T19:44:29.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm78.ko 2023-07-06T19:44:29.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm80.ko 2023-07-06T19:44:29.150 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm83.ko 2023-07-06T19:44:29.151 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm85.ko 2023-07-06T19:44:29.151 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm87.ko 2023-07-06T19:44:29.151 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm90.ko 2023-07-06T19:44:29.151 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm92.ko 2023-07-06T19:44:29.151 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm93.ko 2023-07-06T19:44:29.151 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95234.ko 2023-07-06T19:44:29.151 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95241.ko 2023-07-06T19:44:29.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95245.ko 2023-07-06T19:44:29.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc2945.ko 2023-07-06T19:44:29.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4151.ko 2023-07-06T19:44:29.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4215.ko 2023-07-06T19:44:29.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4222.ko 2023-07-06T19:44:29.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4245.ko 2023-07-06T19:44:29.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4260.ko 2023-07-06T19:44:29.152 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4261.ko 2023-07-06T19:44:29.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max16065.ko 2023-07-06T19:44:29.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1619.ko 2023-07-06T19:44:29.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1668.ko 2023-07-06T19:44:29.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max197.ko 2023-07-06T19:44:29.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6639.ko 2023-07-06T19:44:29.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6650.ko 2023-07-06T19:44:29.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6697.ko 2023-07-06T19:44:29.153 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/mcp3021.ko 2023-07-06T19:44:29.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6683.ko 2023-07-06T19:44:29.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775-core.ko 2023-07-06T19:44:29.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775.ko 2023-07-06T19:44:29.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct7802.ko 2023-07-06T19:44:29.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87360.ko 2023-07-06T19:44:29.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87427.ko 2023-07-06T19:44:29.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pcf8591.ko 2023-07-06T19:44:29.154 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus 2023-07-06T19:44:29.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-06T19:44:29.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-06T19:44:29.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-06T19:44:29.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-06T19:44:29.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-06T19:44:29.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-06T19:44:29.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-06T19:44:29.155 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-06T19:44:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-06T19:44:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-06T19:44:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-06T19:44:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-06T19:44:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/powr1220.ko 2023-07-06T19:44:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5627.ko 2023-07-06T19:44:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5636.ko 2023-07-06T19:44:29.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-06T19:44:29.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht15.ko 2023-07-06T19:44:29.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht21.ko 2023-07-06T19:44:29.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/shtc1.ko 2023-07-06T19:44:29.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sis5595.ko 2023-07-06T19:44:29.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47b397.ko 2023-07-06T19:44:29.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m1.ko 2023-07-06T19:44:29.157 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m192.ko 2023-07-06T19:44:29.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/thmc50.ko 2023-07-06T19:44:29.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp102.ko 2023-07-06T19:44:29.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp103.ko 2023-07-06T19:44:29.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp401.ko 2023-07-06T19:44:29.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp421.ko 2023-07-06T19:44:29.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via-cputemp.ko 2023-07-06T19:44:29.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via686a.ko 2023-07-06T19:44:29.158 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt1211.ko 2023-07-06T19:44:29.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt8231.ko 2023-07-06T19:44:29.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627ehf.ko 2023-07-06T19:44:29.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627hf.ko 2023-07-06T19:44:29.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83781d.ko 2023-07-06T19:44:29.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83791d.ko 2023-07-06T19:44:29.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83792d.ko 2023-07-06T19:44:29.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83793.ko 2023-07-06T19:44:29.159 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83795.ko 2023-07-06T19:44:29.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l785ts.ko 2023-07-06T19:44:29.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l786ng.ko 2023-07-06T19:44:29.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c 2023-07-06T19:44:29.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos 2023-07-06T19:44:29.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-06T19:44:29.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-06T19:44:29.160 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses 2023-07-06T19:44:29.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-06T19:44:29.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-06T19:44:29.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-06T19:44:29.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-06T19:44:29.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-06T19:44:29.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-06T19:44:29.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-06T19:44:29.161 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-06T19:44:29.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-06T19:44:29.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-06T19:44:29.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-06T19:44:29.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-06T19:44:29.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-06T19:44:29.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-06T19:44:29.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-06T19:44:29.162 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-06T19:44:29.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-06T19:44:29.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-06T19:44:29.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-06T19:44:29.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-06T19:44:29.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-06T19:44:29.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-06T19:44:29.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-06T19:44:29.163 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-dev.ko 2023-07-06T19:44:29.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-mux.ko 2023-07-06T19:44:29.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-06T19:44:29.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-smbus.ko 2023-07-06T19:44:29.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-stub.ko 2023-07-06T19:44:29.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband 2023-07-06T19:44:29.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core 2023-07-06T19:44:29.164 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-06T19:44:29.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_core.ko 2023-07-06T19:44:29.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-06T19:44:29.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-06T19:44:29.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-06T19:44:29.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-06T19:44:29.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-06T19:44:29.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw 2023-07-06T19:44:29.165 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4 2023-07-06T19:44:29.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-06T19:44:29.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4 2023-07-06T19:44:29.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-06T19:44:29.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5 2023-07-06T19:44:29.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-06T19:44:29.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca 2023-07-06T19:44:29.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-06T19:44:29.166 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp 2023-07-06T19:44:29.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib 2023-07-06T19:44:29.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-06T19:44:29.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser 2023-07-06T19:44:29.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-06T19:44:29.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert 2023-07-06T19:44:29.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-06T19:44:29.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp 2023-07-06T19:44:29.167 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-06T19:44:29.168 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt 2023-07-06T19:44:29.168 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-06T19:44:29.168 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input 2023-07-06T19:44:29.168 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport 2023-07-06T19:44:29.168 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-06T19:44:29.168 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-06T19:44:29.168 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/gameport.ko 2023-07-06T19:44:29.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/lightning.ko 2023-07-06T19:44:29.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/ns558.ko 2023-07-06T19:44:29.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/joydev.ko 2023-07-06T19:44:29.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard 2023-07-06T19:44:29.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-06T19:44:29.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse 2023-07-06T19:44:29.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/appletouch.ko 2023-07-06T19:44:29.169 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/bcm5974.ko 2023-07-06T19:44:29.170 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/cyapatp.ko 2023-07-06T19:44:29.170 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-06T19:44:29.170 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/sermouse.ko 2023-07-06T19:44:29.170 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-06T19:44:29.170 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-06T19:44:29.170 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-06T19:44:29.170 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4 2023-07-06T19:44:29.170 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-06T19:44:29.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio 2023-07-06T19:44:29.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/altera_ps2.ko 2023-07-06T19:44:29.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/arc_ps2.ko 2023-07-06T19:44:29.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-06T19:44:29.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/serio_raw.ko 2023-07-06T19:44:29.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/sparse-keymap.ko 2023-07-06T19:44:29.171 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu 2023-07-06T19:44:29.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd 2023-07-06T19:44:29.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-06T19:44:29.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds 2023-07-06T19:44:29.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-blinkm.ko 2023-07-06T19:44:29.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lm3530.ko 2023-07-06T19:44:29.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lp3944.ko 2023-07-06T19:44:29.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-ss4200.ko 2023-07-06T19:44:29.172 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger 2023-07-06T19:44:29.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-06T19:44:29.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-06T19:44:29.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-06T19:44:29.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-06T19:44:29.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-06T19:44:29.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-06T19:44:29.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-06T19:44:29.173 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-06T19:44:29.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md 2023-07-06T19:44:29.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache 2023-07-06T19:44:29.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache/bcache.ko 2023-07-06T19:44:29.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-bio-prison.ko 2023-07-06T19:44:29.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache-smq.ko 2023-07-06T19:44:29.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache.ko 2023-07-06T19:44:29.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-crypt.ko 2023-07-06T19:44:29.174 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-delay.ko 2023-07-06T19:44:29.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-flakey.ko 2023-07-06T19:44:29.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-log-userspace.ko 2023-07-06T19:44:29.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-multipath.ko 2023-07-06T19:44:29.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-queue-length.ko 2023-07-06T19:44:29.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-raid.ko 2023-07-06T19:44:29.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-round-robin.ko 2023-07-06T19:44:29.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-service-time.ko 2023-07-06T19:44:29.175 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-switch.ko 2023-07-06T19:44:29.176 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-thin-pool.ko 2023-07-06T19:44:29.176 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-verity.ko 2023-07-06T19:44:29.176 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/faulty.ko 2023-07-06T19:44:29.176 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/linear.ko 2023-07-06T19:44:29.176 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/multipath.ko 2023-07-06T19:44:29.176 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data 2023-07-06T19:44:29.176 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-06T19:44:29.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid0.ko 2023-07-06T19:44:29.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid1.ko 2023-07-06T19:44:29.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid10.ko 2023-07-06T19:44:29.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid456.ko 2023-07-06T19:44:29.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message 2023-07-06T19:44:29.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion 2023-07-06T19:44:29.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptbase.ko 2023-07-06T19:44:29.177 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptctl.ko 2023-07-06T19:44:29.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptfc.ko 2023-07-06T19:44:29.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptlan.ko 2023-07-06T19:44:29.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptsas.ko 2023-07-06T19:44:29.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptscsih.ko 2023-07-06T19:44:29.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptspi.ko 2023-07-06T19:44:29.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd 2023-07-06T19:44:29.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_ich.ko 2023-07-06T19:44:29.178 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_sch.ko 2023-07-06T19:44:29.179 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/mfd-core.ko 2023-07-06T19:44:29.179 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/sm501.ko 2023-07-06T19:44:29.179 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/viperboard.ko 2023-07-06T19:44:29.179 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/vx855.ko 2023-07-06T19:44:29.179 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/wl1273-core.ko 2023-07-06T19:44:29.179 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc 2023-07-06T19:44:29.179 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl 2023-07-06T19:44:29.179 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-06T19:44:29.180 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds9802als.ko 2023-07-06T19:44:29.180 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds990x.ko 2023-07-06T19:44:29.180 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/bh1770glc.ko 2023-07-06T19:44:29.180 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710 2023-07-06T19:44:29.180 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710/cb710.ko 2023-07-06T19:44:29.180 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom 2023-07-06T19:44:29.180 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/at24.ko 2023-07-06T19:44:29.181 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-06T19:44:29.181 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-06T19:44:29.181 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/max6875.ko 2023-07-06T19:44:29.181 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/enclosure.ko 2023-07-06T19:44:29.181 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/hpilo.ko 2023-07-06T19:44:29.181 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29003.ko 2023-07-06T19:44:29.181 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29020.ko 2023-07-06T19:44:29.181 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d 2023-07-06T19:44:29.182 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-06T19:44:29.182 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-06T19:44:29.182 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei 2023-07-06T19:44:29.182 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-me.ko 2023-07-06T19:44:29.182 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-txe.ko 2023-07-06T19:44:29.182 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei.ko 2023-07-06T19:44:29.182 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru 2023-07-06T19:44:29.182 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-06T19:44:29.183 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp 2023-07-06T19:44:29.183 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-06T19:44:29.183 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-06T19:44:29.183 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-06T19:44:29.183 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_7xx1.ko 2023-07-06T19:44:29.183 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_core.ko 2023-07-06T19:44:29.183 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tsl2550.ko 2023-07-06T19:44:29.183 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_balloon.ko 2023-07-06T19:44:29.184 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci 2023-07-06T19:44:29.184 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-06T19:44:29.184 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net 2023-07-06T19:44:29.184 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding 2023-07-06T19:44:29.184 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding/bonding.ko 2023-07-06T19:44:29.184 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa 2023-07-06T19:44:29.184 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53 2023-07-06T19:44:29.185 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-06T19:44:29.185 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-06T19:44:29.185 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-06T19:44:29.185 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx 2023-07-06T19:44:29.185 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-06T19:44:29.185 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dummy.ko 2023-07-06T19:44:29.185 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/eql.ko 2023-07-06T19:44:29.185 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet 2023-07-06T19:44:29.186 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com 2023-07-06T19:44:29.186 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-06T19:44:29.186 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-06T19:44:29.186 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-06T19:44:29.186 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-06T19:44:29.186 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390 2023-07-06T19:44:29.186 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-06T19:44:29.186 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-06T19:44:29.187 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-06T19:44:29.187 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-06T19:44:29.187 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec 2023-07-06T19:44:29.187 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-06T19:44:29.187 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere 2023-07-06T19:44:29.187 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-06T19:44:29.187 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon 2023-07-06T19:44:29.187 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-06T19:44:29.188 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera 2023-07-06T19:44:29.188 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-06T19:44:29.188 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd 2023-07-06T19:44:29.188 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-06T19:44:29.188 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-06T19:44:29.188 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-06T19:44:29.188 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros 2023-07-06T19:44:29.188 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx 2023-07-06T19:44:29.189 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-06T19:44:29.189 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-06T19:44:29.189 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-06T19:44:29.189 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-06T19:44:29.189 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-06T19:44:29.189 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx 2023-07-06T19:44:29.189 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-06T19:44:29.189 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-06T19:44:29.190 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom 2023-07-06T19:44:29.190 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-06T19:44:29.190 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-06T19:44:29.190 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-06T19:44:29.190 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-06T19:44:29.190 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-06T19:44:29.190 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet 2023-07-06T19:44:29.190 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-06T19:44:29.191 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-06T19:44:29.191 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade 2023-07-06T19:44:29.191 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna 2023-07-06T19:44:29.191 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-06T19:44:29.191 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio 2023-07-06T19:44:29.191 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-06T19:44:29.191 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-06T19:44:29.191 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-06T19:44:29.192 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-06T19:44:29.192 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-06T19:44:29.192 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-06T19:44:29.192 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-06T19:44:29.192 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-06T19:44:29.192 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-06T19:44:29.192 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-06T19:44:29.193 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco 2023-07-06T19:44:29.193 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic 2023-07-06T19:44:29.193 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-06T19:44:29.193 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec 2023-07-06T19:44:29.193 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip 2023-07-06T19:44:29.193 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-06T19:44:29.193 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-06T19:44:29.193 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-06T19:44:29.194 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-06T19:44:29.194 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-06T19:44:29.194 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-06T19:44:29.194 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink 2023-07-06T19:44:29.194 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-06T19:44:29.194 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-06T19:44:29.194 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dnet.ko 2023-07-06T19:44:29.194 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex 2023-07-06T19:44:29.195 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet 2023-07-06T19:44:29.195 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-06T19:44:29.195 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ethoc.ko 2023-07-06T19:44:29.195 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/fealnx.ko 2023-07-06T19:44:29.195 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel 2023-07-06T19:44:29.195 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-06T19:44:29.195 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000 2023-07-06T19:44:29.195 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-06T19:44:29.196 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e 2023-07-06T19:44:29.196 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-06T19:44:29.196 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k 2023-07-06T19:44:29.196 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-06T19:44:29.196 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e 2023-07-06T19:44:29.196 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-06T19:44:29.196 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf 2023-07-06T19:44:29.196 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-06T19:44:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb 2023-07-06T19:44:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-06T19:44:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf 2023-07-06T19:44:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-06T19:44:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-06T19:44:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-06T19:44:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-06T19:44:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-06T19:44:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/jme.ko 2023-07-06T19:44:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell 2023-07-06T19:44:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-06T19:44:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-06T19:44:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-06T19:44:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox 2023-07-06T19:44:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-06T19:44:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-06T19:44:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-06T19:44:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-06T19:44:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-06T19:44:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-06T19:44:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel 2023-07-06T19:44:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-06T19:44:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom 2023-07-06T19:44:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-06T19:44:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-06T19:44:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi 2023-07-06T19:44:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-06T19:44:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-06T19:44:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion 2023-07-06T19:44:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-06T19:44:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia 2023-07-06T19:44:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-06T19:44:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines 2023-07-06T19:44:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-06T19:44:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-06T19:44:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic 2023-07-06T19:44:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-06T19:44:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-06T19:44:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-06T19:44:29.202 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-06T19:44:29.202 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-06T19:44:29.202 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc 2023-07-06T19:44:29.202 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-06T19:44:29.202 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek 2023-07-06T19:44:29.202 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-06T19:44:29.202 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-06T19:44:29.203 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-06T19:44:29.203 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-06T19:44:29.203 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker 2023-07-06T19:44:29.203 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-06T19:44:29.203 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc 2023-07-06T19:44:29.203 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-06T19:44:29.203 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan 2023-07-06T19:44:29.203 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-06T19:44:29.204 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis 2023-07-06T19:44:29.204 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-06T19:44:29.204 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-06T19:44:29.204 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc 2023-07-06T19:44:29.204 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-06T19:44:29.204 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-06T19:44:29.204 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-06T19:44:29.204 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-06T19:44:29.205 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro 2023-07-06T19:44:29.205 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-06T19:44:29.205 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-06T19:44:29.205 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-06T19:44:29.205 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun 2023-07-06T19:44:29.205 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-06T19:44:29.205 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-06T19:44:29.205 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-06T19:44:29.206 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-06T19:44:29.206 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti 2023-07-06T19:44:29.206 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-06T19:44:29.206 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti 2023-07-06T19:44:29.206 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-06T19:44:29.206 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via 2023-07-06T19:44:29.206 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-06T19:44:29.206 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-06T19:44:29.207 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet 2023-07-06T19:44:29.207 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-06T19:44:29.207 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-06T19:44:29.207 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom 2023-07-06T19:44:29.207 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-06T19:44:29.207 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/geneve.ko 2023-07-06T19:44:29.207 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv 2023-07-06T19:44:29.207 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-06T19:44:29.208 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154 2023-07-06T19:44:29.208 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-06T19:44:29.208 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ifb.ko 2023-07-06T19:44:29.208 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan 2023-07-06T19:44:29.208 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-06T19:44:29.208 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvlan.ko 2023-07-06T19:44:29.208 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvtap.ko 2023-07-06T19:44:29.209 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio 2023-07-06T19:44:29.209 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio.ko 2023-07-06T19:44:29.209 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-06T19:44:29.209 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-06T19:44:29.209 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mii.ko 2023-07-06T19:44:29.209 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/net_failover.ko 2023-07-06T19:44:29.209 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/netconsole.ko 2023-07-06T19:44:29.209 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/nlmon.ko 2023-07-06T19:44:29.210 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs 2023-07-06T19:44:29.210 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-07-06T19:44:29.210 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-06T19:44:29.210 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy 2023-07-06T19:44:29.210 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/amd.ko 2023-07-06T19:44:29.210 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-06T19:44:29.210 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-06T19:44:29.210 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-06T19:44:29.211 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm87xx.ko 2023-07-06T19:44:29.211 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/broadcom.ko 2023-07-06T19:44:29.211 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/cicada.ko 2023-07-06T19:44:29.211 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/davicom.ko 2023-07-06T19:44:29.211 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/dp83640.ko 2023-07-06T19:44:29.211 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/et1011c.ko 2023-07-06T19:44:29.211 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/icplus.ko 2023-07-06T19:44:29.211 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/lxt.ko 2023-07-06T19:44:29.212 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/marvell.ko 2023-07-06T19:44:29.212 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/micrel.ko 2023-07-06T19:44:29.212 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/national.ko 2023-07-06T19:44:29.212 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/phylink.ko 2023-07-06T19:44:29.212 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/qsemi.ko 2023-07-06T19:44:29.212 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/realtek.ko 2023-07-06T19:44:29.212 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/smsc.ko 2023-07-06T19:44:29.213 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/ste10Xp.ko 2023-07-06T19:44:29.213 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/vitesse.ko 2023-07-06T19:44:29.213 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp 2023-07-06T19:44:29.213 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-06T19:44:29.213 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_async.ko 2023-07-06T19:44:29.213 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-06T19:44:29.213 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-06T19:44:29.213 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-06T19:44:29.214 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-06T19:44:29.214 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppoe.ko 2023-07-06T19:44:29.214 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppox.ko 2023-07-06T19:44:29.214 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pptp.ko 2023-07-06T19:44:29.214 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip 2023-07-06T19:44:29.214 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slhc.ko 2023-07-06T19:44:29.214 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slip.ko 2023-07-06T19:44:29.214 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/sungem_phy.ko 2023-07-06T19:44:29.215 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tap.ko 2023-07-06T19:44:29.215 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team 2023-07-06T19:44:29.215 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team.ko 2023-07-06T19:44:29.215 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-06T19:44:29.215 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-06T19:44:29.215 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-06T19:44:29.215 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_random.ko 2023-07-06T19:44:29.215 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-06T19:44:29.216 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tun.ko 2023-07-06T19:44:29.216 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/veth.ko 2023-07-06T19:44:29.216 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/virtio_net.ko 2023-07-06T19:44:29.216 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3 2023-07-06T19:44:29.216 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-06T19:44:29.216 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan 2023-07-06T19:44:29.216 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan/vxlan.ko 2023-07-06T19:44:29.216 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback 2023-07-06T19:44:29.217 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-06T19:44:29.217 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netfront.ko 2023-07-06T19:44:29.217 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb 2023-07-06T19:44:29.217 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb/ntb.ko 2023-07-06T19:44:29.217 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme 2023-07-06T19:44:29.217 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host 2023-07-06T19:44:29.217 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme-core.ko 2023-07-06T19:44:29.217 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme.ko 2023-07-06T19:44:29.218 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport 2023-07-06T19:44:29.218 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport.ko 2023-07-06T19:44:29.218 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_cs.ko 2023-07-06T19:44:29.218 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_pc.ko 2023-07-06T19:44:29.218 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_serial.ko 2023-07-06T19:44:29.218 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci 2023-07-06T19:44:29.218 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug 2023-07-06T19:44:29.218 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-06T19:44:29.219 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie 2023-07-06T19:44:29.219 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-06T19:44:29.219 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/xen-pcifront.ko 2023-07-06T19:44:29.219 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia 2023-07-06T19:44:29.219 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/i82092.ko 2023-07-06T19:44:29.219 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/pd6729.ko 2023-07-06T19:44:29.219 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-06T19:44:29.219 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl 2023-07-06T19:44:29.220 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel 2023-07-06T19:44:29.220 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-06T19:44:29.220 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform 2023-07-06T19:44:29.220 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86 2023-07-06T19:44:29.220 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-06T19:44:29.220 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acerhdf.ko 2023-07-06T19:44:29.220 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-06T19:44:29.220 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-06T19:44:29.221 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-06T19:44:29.221 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-06T19:44:29.221 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-06T19:44:29.221 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-06T19:44:29.221 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-06T19:44:29.221 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-06T19:44:29.221 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-06T19:44:29.221 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/hdaps.ko 2023-07-06T19:44:29.222 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel 2023-07-06T19:44:29.222 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-06T19:44:29.222 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel_ips.ko 2023-07-06T19:44:29.222 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-06T19:44:29.222 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-06T19:44:29.222 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-06T19:44:29.222 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-06T19:44:29.222 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-06T19:44:29.223 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-06T19:44:29.223 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-06T19:44:29.223 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-06T19:44:29.223 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-06T19:44:29.223 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi.ko 2023-07-06T19:44:29.223 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap 2023-07-06T19:44:29.223 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-06T19:44:29.223 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-06T19:44:29.224 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps 2023-07-06T19:44:29.224 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients 2023-07-06T19:44:29.224 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-06T19:44:29.224 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-06T19:44:29.224 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps_parport.ko 2023-07-06T19:44:29.224 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/pps_core.ko 2023-07-06T19:44:29.224 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp 2023-07-06T19:44:29.225 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp.ko 2023-07-06T19:44:29.225 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp_kvm.ko 2023-07-06T19:44:29.225 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc 2023-07-06T19:44:29.225 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-06T19:44:29.225 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-06T19:44:29.225 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-06T19:44:29.225 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-06T19:44:29.225 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-06T19:44:29.226 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-06T19:44:29.226 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-06T19:44:29.226 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-06T19:44:29.226 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-06T19:44:29.226 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-06T19:44:29.226 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-06T19:44:29.226 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-em3027.ko 2023-07-06T19:44:29.226 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-06T19:44:29.227 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-06T19:44:29.227 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-06T19:44:29.227 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-06T19:44:29.227 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-06T19:44:29.227 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-06T19:44:29.227 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-max6900.ko 2023-07-06T19:44:29.227 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-06T19:44:29.227 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-06T19:44:29.228 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-06T19:44:29.228 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-06T19:44:29.228 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-06T19:44:29.228 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-06T19:44:29.228 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-06T19:44:29.228 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-06T19:44:29.228 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-06T19:44:29.228 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-06T19:44:29.229 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-06T19:44:29.229 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-06T19:44:29.229 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-x1205.ko 2023-07-06T19:44:29.229 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi 2023-07-06T19:44:29.229 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-9xxx.ko 2023-07-06T19:44:29.229 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-sas.ko 2023-07-06T19:44:29.229 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-xxxx.ko 2023-07-06T19:44:29.229 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/BusLogic.ko 2023-07-06T19:44:29.230 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/a100u2w.ko 2023-07-06T19:44:29.230 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid 2023-07-06T19:44:29.230 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-06T19:44:29.230 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/advansys.ko 2023-07-06T19:44:29.230 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx 2023-07-06T19:44:29.230 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-06T19:44:29.230 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-06T19:44:29.230 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/am53c974.ko 2023-07-06T19:44:29.231 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr 2023-07-06T19:44:29.231 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-06T19:44:29.231 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/atp870u.ko 2023-07-06T19:44:29.231 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi 2023-07-06T19:44:29.231 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-06T19:44:29.231 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa 2023-07-06T19:44:29.231 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa/bfa.ko 2023-07-06T19:44:29.231 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc 2023-07-06T19:44:29.232 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-06T19:44:29.232 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i 2023-07-06T19:44:29.232 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-06T19:44:29.232 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ch.ko 2023-07-06T19:44:29.232 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor 2023-07-06T19:44:29.232 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-06T19:44:29.232 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi 2023-07-06T19:44:29.232 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-06T19:44:29.233 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-06T19:44:29.233 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-06T19:44:29.233 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-06T19:44:29.233 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-06T19:44:29.233 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dc395x.ko 2023-07-06T19:44:29.233 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler 2023-07-06T19:44:29.233 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-06T19:44:29.233 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-06T19:44:29.234 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-06T19:44:29.234 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-06T19:44:29.234 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dmx3191d.ko 2023-07-06T19:44:29.234 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r 2023-07-06T19:44:29.234 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-06T19:44:29.234 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esp_scsi.ko 2023-07-06T19:44:29.234 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe 2023-07-06T19:44:29.234 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-06T19:44:29.235 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-06T19:44:29.235 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic 2023-07-06T19:44:29.235 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic/fnic.ko 2023-07-06T19:44:29.235 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hpsa.ko 2023-07-06T19:44:29.235 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hptiop.ko 2023-07-06T19:44:29.235 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hv_storvsc.ko 2023-07-06T19:44:29.235 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/initio.ko 2023-07-06T19:44:29.235 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ipr.ko 2023-07-06T19:44:29.236 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ips.ko 2023-07-06T19:44:29.236 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci 2023-07-06T19:44:29.236 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci/isci.ko 2023-07-06T19:44:29.236 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-06T19:44:29.236 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-06T19:44:29.236 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc 2023-07-06T19:44:29.236 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc/libfc.ko 2023-07-06T19:44:29.236 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi.ko 2023-07-06T19:44:29.237 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-06T19:44:29.237 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas 2023-07-06T19:44:29.237 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas/libsas.ko 2023-07-06T19:44:29.237 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc 2023-07-06T19:44:29.237 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-06T19:44:29.237 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid 2023-07-06T19:44:29.237 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid.ko 2023-07-06T19:44:29.237 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-06T19:44:29.238 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-06T19:44:29.238 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-06T19:44:29.238 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas 2023-07-06T19:44:29.238 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-06T19:44:29.238 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas 2023-07-06T19:44:29.238 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-06T19:44:29.238 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvumi.ko 2023-07-06T19:44:29.238 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001 2023-07-06T19:44:29.239 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-06T19:44:29.239 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pmcraid.ko 2023-07-06T19:44:29.239 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla1280.ko 2023-07-06T19:44:29.239 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx 2023-07-06T19:44:29.239 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-06T19:44:29.239 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-06T19:44:29.239 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx 2023-07-06T19:44:29.239 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-06T19:44:29.240 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/raid_class.ko 2023-07-06T19:44:29.240 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_debug.ko 2023-07-06T19:44:29.240 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-06T19:44:29.240 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-06T19:44:29.240 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-06T19:44:29.240 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-06T19:44:29.240 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-06T19:44:29.240 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ses.ko 2023-07-06T19:44:29.241 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/st.ko 2023-07-06T19:44:29.241 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/stex.ko 2023-07-06T19:44:29.241 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2 2023-07-06T19:44:29.241 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-06T19:44:29.241 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/virtio_scsi.ko 2023-07-06T19:44:29.241 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-06T19:44:29.241 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/wd719x.ko 2023-07-06T19:44:29.241 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb 2023-07-06T19:44:29.242 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb/ssb.ko 2023-07-06T19:44:29.242 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target 2023-07-06T19:44:29.242 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi 2023-07-06T19:44:29.242 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-06T19:44:29.242 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback 2023-07-06T19:44:29.242 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-06T19:44:29.242 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_file.ko 2023-07-06T19:44:29.242 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_iblock.ko 2023-07-06T19:44:29.243 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_mod.ko 2023-07-06T19:44:29.243 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_pscsi.ko 2023-07-06T19:44:29.243 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_user.ko 2023-07-06T19:44:29.243 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc 2023-07-06T19:44:29.243 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-06T19:44:29.243 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal 2023-07-06T19:44:29.243 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel 2023-07-06T19:44:29.243 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal 2023-07-06T19:44:29.244 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-06T19:44:29.244 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-06T19:44:29.244 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-06T19:44:29.244 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-06T19:44:29.244 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-06T19:44:29.244 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-06T19:44:29.244 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-06T19:44:29.245 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-06T19:44:29.245 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-06T19:44:29.245 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-06T19:44:29.245 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-06T19:44:29.245 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-06T19:44:29.245 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-06T19:44:29.245 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-06T19:44:29.245 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-06T19:44:29.246 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty 2023-07-06T19:44:29.246 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless 2023-07-06T19:44:29.246 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-06T19:44:29.246 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_gsm.ko 2023-07-06T19:44:29.246 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_hdlc.ko 2023-07-06T19:44:29.246 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/nozomi.ko 2023-07-06T19:44:29.246 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial 2023-07-06T19:44:29.246 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250 2023-07-06T19:44:29.247 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-06T19:44:29.247 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/arc_uart.ko 2023-07-06T19:44:29.247 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm 2023-07-06T19:44:29.247 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-06T19:44:29.247 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/synclink_gt.ko 2023-07-06T19:44:29.247 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs 2023-07-06T19:44:29.247 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core 2023-07-06T19:44:29.247 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-06T19:44:29.248 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host 2023-07-06T19:44:29.248 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-06T19:44:29.248 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio 2023-07-06T19:44:29.248 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio.ko 2023-07-06T19:44:29.248 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_aec.ko 2023-07-06T19:44:29.248 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_cif.ko 2023-07-06T19:44:29.248 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_pci_generic.ko 2023-07-06T19:44:29.248 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_sercos3.ko 2023-07-06T19:44:29.249 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb 2023-07-06T19:44:29.249 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class 2023-07-06T19:44:29.249 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-acm.ko 2023-07-06T19:44:29.249 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-06T19:44:29.249 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usblp.ko 2023-07-06T19:44:29.249 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usbtmc.ko 2023-07-06T19:44:29.249 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host 2023-07-06T19:44:29.249 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-06T19:44:29.250 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image 2023-07-06T19:44:29.250 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/mdc800.ko 2023-07-06T19:44:29.250 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/microtek.ko 2023-07-06T19:44:29.250 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc 2023-07-06T19:44:29.250 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/adutux.ko 2023-07-06T19:44:29.250 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/appledisplay.ko 2023-07-06T19:44:29.250 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi26.ko 2023-07-06T19:44:29.250 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi62.ko 2023-07-06T19:44:29.251 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ezusb.ko 2023-07-06T19:44:29.251 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/idmouse.ko 2023-07-06T19:44:29.251 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/iowarrior.ko 2023-07-06T19:44:29.251 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-06T19:44:29.251 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ldusb.ko 2023-07-06T19:44:29.251 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/legousbtower.ko 2023-07-06T19:44:29.251 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga 2023-07-06T19:44:29.251 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-06T19:44:29.252 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-06T19:44:29.252 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usb3503.ko 2023-07-06T19:44:29.252 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usblcd.ko 2023-07-06T19:44:29.252 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-06T19:44:29.252 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/uss720.ko 2023-07-06T19:44:29.252 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/yurex.ko 2023-07-06T19:44:29.252 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy 2023-07-06T19:44:29.252 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy/phy-generic.ko 2023-07-06T19:44:29.253 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial 2023-07-06T19:44:29.253 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/aircable.ko 2023-07-06T19:44:29.253 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ark3116.ko 2023-07-06T19:44:29.253 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-06T19:44:29.253 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ch341.ko 2023-07-06T19:44:29.253 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cp210x.ko 2023-07-06T19:44:29.253 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cyberjack.ko 2023-07-06T19:44:29.253 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-06T19:44:29.254 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-06T19:44:29.254 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/empeg.ko 2023-07-06T19:44:29.254 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-06T19:44:29.254 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-06T19:44:29.254 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-06T19:44:29.254 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_ti.ko 2023-07-06T19:44:29.254 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipaq.ko 2023-07-06T19:44:29.254 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipw.ko 2023-07-06T19:44:29.255 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ir-usb.ko 2023-07-06T19:44:29.255 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-06T19:44:29.255 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan.ko 2023-07-06T19:44:29.255 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-06T19:44:29.255 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-06T19:44:29.255 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-06T19:44:29.255 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mct_u232.ko 2023-07-06T19:44:29.255 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7720.ko 2023-07-06T19:44:29.256 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7840.ko 2023-07-06T19:44:29.256 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/navman.ko 2023-07-06T19:44:29.256 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/omninet.ko 2023-07-06T19:44:29.256 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/opticon.ko 2023-07-06T19:44:29.256 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/option.ko 2023-07-06T19:44:29.256 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/oti6858.ko 2023-07-06T19:44:29.256 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/pl2303.ko 2023-07-06T19:44:29.256 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcaux.ko 2023-07-06T19:44:29.257 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcserial.ko 2023-07-06T19:44:29.257 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/quatech2.ko 2023-07-06T19:44:29.257 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/safe_serial.ko 2023-07-06T19:44:29.257 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/sierra.ko 2023-07-06T19:44:29.257 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-06T19:44:29.257 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ssu100.ko 2023-07-06T19:44:29.257 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/symbolserial.ko 2023-07-06T19:44:29.257 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-06T19:44:29.258 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-06T19:44:29.258 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_debug.ko 2023-07-06T19:44:29.258 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-06T19:44:29.258 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/visor.ko 2023-07-06T19:44:29.258 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/whiteheat.ko 2023-07-06T19:44:29.258 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-06T19:44:29.258 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage 2023-07-06T19:44:29.258 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/uas.ko 2023-07-06T19:44:29.259 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-06T19:44:29.259 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-06T19:44:29.259 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-06T19:44:29.259 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-06T19:44:29.259 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-06T19:44:29.259 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-06T19:44:29.259 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-06T19:44:29.259 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-karma.ko 2023-07-06T19:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-06T19:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-06T19:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-06T19:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-06T19:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-06T19:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/usb-storage.ko 2023-07-06T19:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip 2023-07-06T19:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-06T19:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-06T19:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-06T19:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio 2023-07-06T19:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci 2023-07-06T19:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-06T19:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-06T19:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio.ko 2023-07-06T19:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-06T19:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost 2023-07-06T19:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost.ko 2023-07-06T19:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-06T19:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_net.ko 2023-07-06T19:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_scsi.ko 2023-07-06T19:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video 2023-07-06T19:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight 2023-07-06T19:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/apple_bl.ko 2023-07-06T19:44:29.263 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/lcd.ko 2023-07-06T19:44:29.263 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-06T19:44:29.263 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio 2023-07-06T19:44:29.263 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio.ko 2023-07-06T19:44:29.263 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_balloon.ko 2023-07-06T19:44:29.263 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_mmio.ko 2023-07-06T19:44:29.263 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci.ko 2023-07-06T19:44:29.263 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-06T19:44:29.264 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-06T19:44:29.264 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_ring.ko 2023-07-06T19:44:29.264 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1 2023-07-06T19:44:29.264 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters 2023-07-06T19:44:29.264 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2482.ko 2023-07-06T19:44:29.264 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2490.ko 2023-07-06T19:44:29.264 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves 2023-07-06T19:44:29.265 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-06T19:44:29.265 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-06T19:44:29.265 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-06T19:44:29.265 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-06T19:44:29.265 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-06T19:44:29.265 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-06T19:44:29.265 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-06T19:44:29.265 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-06T19:44:29.265 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-06T19:44:29.266 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-06T19:44:29.266 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-06T19:44:29.266 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/wire.ko 2023-07-06T19:44:29.266 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog 2023-07-06T19:44:29.266 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-06T19:44:29.266 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-06T19:44:29.266 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-06T19:44:29.266 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/hpwdt.ko 2023-07-06T19:44:29.267 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/i6300esb.ko 2023-07-06T19:44:29.267 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-06T19:44:29.267 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-06T19:44:29.267 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ib700wdt.ko 2023-07-06T19:44:29.267 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ibmasr.ko 2023-07-06T19:44:29.267 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-06T19:44:29.267 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-06T19:44:29.267 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it87_wdt.ko 2023-07-06T19:44:29.268 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/machzwd.ko 2023-07-06T19:44:29.268 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/nv_tco.ko 2023-07-06T19:44:29.268 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-06T19:44:29.268 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-06T19:44:29.268 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-06T19:44:29.268 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-06T19:44:29.268 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/softdog.ko 2023-07-06T19:44:29.269 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-06T19:44:29.269 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/via_wdt.ko 2023-07-06T19:44:29.269 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-06T19:44:29.269 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-06T19:44:29.269 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-06T19:44:29.269 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/wdt_pci.ko 2023-07-06T19:44:29.269 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/xen_wdt.ko 2023-07-06T19:44:29.269 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen 2023-07-06T19:44:29.270 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-06T19:44:29.270 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-evtchn.ko 2023-07-06T19:44:29.270 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntalloc.ko 2023-07-06T19:44:29.270 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntdev.ko 2023-07-06T19:44:29.270 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback 2023-07-06T19:44:29.270 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-06T19:44:29.270 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-privcmd.ko 2023-07-06T19:44:29.270 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs 2023-07-06T19:44:29.271 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-06T19:44:29.271 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs 2023-07-06T19:44:29.271 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p 2023-07-06T19:44:29.271 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p/9p.ko 2023-07-06T19:44:29.271 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs 2023-07-06T19:44:29.271 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs/affs.ko 2023-07-06T19:44:29.271 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs 2023-07-06T19:44:29.271 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs/befs.ko 2023-07-06T19:44:29.272 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/binfmt_misc.ko 2023-07-06T19:44:29.272 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs 2023-07-06T19:44:29.272 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs/btrfs.ko 2023-07-06T19:44:29.272 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles 2023-07-06T19:44:29.272 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles/cachefiles.ko 2023-07-06T19:44:29.272 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph 2023-07-06T19:44:29.272 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph/ceph.ko 2023-07-06T19:44:29.272 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda 2023-07-06T19:44:29.273 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda/coda.ko 2023-07-06T19:44:29.273 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs 2023-07-06T19:44:29.273 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs/cramfs.ko 2023-07-06T19:44:29.273 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm 2023-07-06T19:44:29.273 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm/dlm.ko 2023-07-06T19:44:29.273 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs 2023-07-06T19:44:29.273 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-06T19:44:29.273 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs 2023-07-06T19:44:29.274 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs/f2fs.ko 2023-07-06T19:44:29.274 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat 2023-07-06T19:44:29.274 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/fat.ko 2023-07-06T19:44:29.274 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/msdos.ko 2023-07-06T19:44:29.274 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/vfat.ko 2023-07-06T19:44:29.274 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache 2023-07-06T19:44:29.274 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache/fscache.ko 2023-07-06T19:44:29.274 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse 2023-07-06T19:44:29.275 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/cuse.ko 2023-07-06T19:44:29.275 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/fuse.ko 2023-07-06T19:44:29.275 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2 2023-07-06T19:44:29.275 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2/gfs2.ko 2023-07-06T19:44:29.275 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs 2023-07-06T19:44:29.275 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs/hfs.ko 2023-07-06T19:44:29.275 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus 2023-07-06T19:44:29.275 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus/hfsplus.ko 2023-07-06T19:44:29.276 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs 2023-07-06T19:44:29.276 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs/isofs.ko 2023-07-06T19:44:29.276 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs 2023-07-06T19:44:29.276 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs/jfs.ko 2023-07-06T19:44:29.276 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd 2023-07-06T19:44:29.276 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd/lockd.ko 2023-07-06T19:44:29.276 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix 2023-07-06T19:44:29.276 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix/minix.ko 2023-07-06T19:44:29.277 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs 2023-07-06T19:44:29.277 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs/netfs.ko 2023-07-06T19:44:29.277 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs 2023-07-06T19:44:29.277 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout 2023-07-06T19:44:29.277 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-06T19:44:29.277 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout 2023-07-06T19:44:29.277 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-06T19:44:29.277 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout 2023-07-06T19:44:29.278 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-06T19:44:29.278 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfs.ko 2023-07-06T19:44:29.278 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv3.ko 2023-07-06T19:44:29.278 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv4.ko 2023-07-06T19:44:29.278 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common 2023-07-06T19:44:29.278 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/grace.ko 2023-07-06T19:44:29.278 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/nfs_acl.ko 2023-07-06T19:44:29.278 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd 2023-07-06T19:44:29.279 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd/nfsd.ko 2023-07-06T19:44:29.279 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2 2023-07-06T19:44:29.279 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2/nilfs2.ko 2023-07-06T19:44:29.279 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls 2023-07-06T19:44:29.279 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-celtic.ko 2023-07-06T19:44:29.279 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-centeuro.ko 2023-07-06T19:44:29.279 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-croatian.ko 2023-07-06T19:44:29.279 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-cyrillic.ko 2023-07-06T19:44:29.280 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-gaelic.ko 2023-07-06T19:44:29.280 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-greek.ko 2023-07-06T19:44:29.280 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-iceland.ko 2023-07-06T19:44:29.280 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-inuit.ko 2023-07-06T19:44:29.280 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-roman.ko 2023-07-06T19:44:29.280 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-romanian.ko 2023-07-06T19:44:29.280 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-turkish.ko 2023-07-06T19:44:29.280 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1250.ko 2023-07-06T19:44:29.281 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1251.ko 2023-07-06T19:44:29.281 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1255.ko 2023-07-06T19:44:29.281 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp737.ko 2023-07-06T19:44:29.281 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp775.ko 2023-07-06T19:44:29.281 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp850.ko 2023-07-06T19:44:29.281 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp852.ko 2023-07-06T19:44:29.281 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp855.ko 2023-07-06T19:44:29.281 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp857.ko 2023-07-06T19:44:29.282 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp860.ko 2023-07-06T19:44:29.282 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp861.ko 2023-07-06T19:44:29.282 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp862.ko 2023-07-06T19:44:29.282 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp863.ko 2023-07-06T19:44:29.282 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp864.ko 2023-07-06T19:44:29.282 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp865.ko 2023-07-06T19:44:29.282 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp866.ko 2023-07-06T19:44:29.282 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp869.ko 2023-07-06T19:44:29.283 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp874.ko 2023-07-06T19:44:29.283 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp932.ko 2023-07-06T19:44:29.283 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp936.ko 2023-07-06T19:44:29.283 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp949.ko 2023-07-06T19:44:29.283 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp950.ko 2023-07-06T19:44:29.283 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_euc-jp.ko 2023-07-06T19:44:29.283 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-1.ko 2023-07-06T19:44:29.283 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-13.ko 2023-07-06T19:44:29.284 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-14.ko 2023-07-06T19:44:29.284 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-15.ko 2023-07-06T19:44:29.284 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-2.ko 2023-07-06T19:44:29.284 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-3.ko 2023-07-06T19:44:29.284 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-4.ko 2023-07-06T19:44:29.284 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-5.ko 2023-07-06T19:44:29.284 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-6.ko 2023-07-06T19:44:29.284 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-7.ko 2023-07-06T19:44:29.285 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-9.ko 2023-07-06T19:44:29.285 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-r.ko 2023-07-06T19:44:29.285 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-ru.ko 2023-07-06T19:44:29.285 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-u.ko 2023-07-06T19:44:29.285 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_utf8.ko 2023-07-06T19:44:29.285 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2 2023-07-06T19:44:29.285 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster 2023-07-06T19:44:29.285 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-06T19:44:29.286 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm 2023-07-06T19:44:29.286 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-06T19:44:29.286 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs 2023-07-06T19:44:29.286 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-06T19:44:29.286 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2.ko 2023-07-06T19:44:29.286 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-06T19:44:29.286 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-06T19:44:29.287 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-06T19:44:29.287 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs 2023-07-06T19:44:29.287 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs/overlay.ko 2023-07-06T19:44:29.287 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore 2023-07-06T19:44:29.287 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore/ramoops.ko 2023-07-06T19:44:29.287 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs 2023-07-06T19:44:29.287 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs/reiserfs.ko 2023-07-06T19:44:29.287 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs 2023-07-06T19:44:29.288 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs/romfs.ko 2023-07-06T19:44:29.288 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb 2023-07-06T19:44:29.288 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client 2023-07-06T19:44:29.288 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client/cifs.ko 2023-07-06T19:44:29.288 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common 2023-07-06T19:44:29.288 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_arc4.ko 2023-07-06T19:44:29.288 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_md4.ko 2023-07-06T19:44:29.288 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs 2023-07-06T19:44:29.289 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs/squashfs.ko 2023-07-06T19:44:29.289 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv 2023-07-06T19:44:29.289 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv/sysv.ko 2023-07-06T19:44:29.289 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf 2023-07-06T19:44:29.289 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf/udf.ko 2023-07-06T19:44:29.289 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs 2023-07-06T19:44:29.289 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs/ufs.ko 2023-07-06T19:44:29.289 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs 2023-07-06T19:44:29.290 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs/xfs.ko 2023-07-06T19:44:29.290 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel 2023-07-06T19:44:29.290 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu 2023-07-06T19:44:29.290 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu/rcutorture.ko 2023-07-06T19:44:29.290 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/torture.ko 2023-07-06T19:44:29.290 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace 2023-07-06T19:44:29.290 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-06T19:44:29.290 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib 2023-07-06T19:44:29.291 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/asn1_encoder.ko 2023-07-06T19:44:29.291 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc-itu-t.ko 2023-07-06T19:44:29.291 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc8.ko 2023-07-06T19:44:29.291 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto 2023-07-06T19:44:29.291 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libarc4.ko 2023-07-06T19:44:29.291 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libdes.ko 2023-07-06T19:44:29.291 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/libcrc32c.ko 2023-07-06T19:44:29.291 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lru_cache.ko 2023-07-06T19:44:29.292 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4 2023-07-06T19:44:29.292 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4_compress.ko 2023-07-06T19:44:29.292 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4hc_compress.ko 2023-07-06T19:44:29.292 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math 2023-07-06T19:44:29.292 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math/cordic.ko 2023-07-06T19:44:29.292 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6 2023-07-06T19:44:29.292 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6/raid6_pq.ko 2023-07-06T19:44:29.292 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon 2023-07-06T19:44:29.293 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-06T19:44:29.293 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_bm.ko 2023-07-06T19:44:29.293 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_fsm.ko 2023-07-06T19:44:29.293 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_kmp.ko 2023-07-06T19:44:29.293 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd 2023-07-06T19:44:29.293 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd/zstd_compress.ko 2023-07-06T19:44:29.293 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm 2023-07-06T19:44:29.293 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm/hwpoison-inject.ko 2023-07-06T19:44:29.294 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net 2023-07-06T19:44:29.294 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan 2023-07-06T19:44:29.294 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/6lowpan.ko 2023-07-06T19:44:29.294 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_dest.ko 2023-07-06T19:44:29.294 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_fragment.ko 2023-07-06T19:44:29.294 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_hop.ko 2023-07-06T19:44:29.294 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-06T19:44:29.294 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_mobility.ko 2023-07-06T19:44:29.295 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_routing.ko 2023-07-06T19:44:29.295 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_udp.ko 2023-07-06T19:44:29.295 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802 2023-07-06T19:44:29.295 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/garp.ko 2023-07-06T19:44:29.295 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/mrp.ko 2023-07-06T19:44:29.295 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/p8022.ko 2023-07-06T19:44:29.295 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/psnap.ko 2023-07-06T19:44:29.295 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/stp.ko 2023-07-06T19:44:29.296 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q 2023-07-06T19:44:29.296 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q/8021q.ko 2023-07-06T19:44:29.296 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p 2023-07-06T19:44:29.296 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet.ko 2023-07-06T19:44:29.296 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_fd.ko 2023-07-06T19:44:29.296 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_rdma.ko 2023-07-06T19:44:29.296 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_virtio.ko 2023-07-06T19:44:29.296 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv 2023-07-06T19:44:29.297 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv/batman-adv.ko 2023-07-06T19:44:29.297 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge 2023-07-06T19:44:29.297 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/br_netfilter.ko 2023-07-06T19:44:29.297 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/bridge.ko 2023-07-06T19:44:29.297 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter 2023-07-06T19:44:29.297 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-06T19:44:29.297 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-06T19:44:29.297 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-06T19:44:29.298 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-06T19:44:29.298 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-06T19:44:29.298 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-06T19:44:29.298 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-06T19:44:29.298 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-06T19:44:29.298 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-06T19:44:29.298 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-06T19:44:29.298 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-06T19:44:29.299 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-06T19:44:29.299 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-06T19:44:29.299 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-06T19:44:29.299 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-06T19:44:29.299 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-06T19:44:29.299 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-06T19:44:29.299 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-06T19:44:29.299 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-06T19:44:29.300 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-06T19:44:29.300 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtables.ko 2023-07-06T19:44:29.300 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-06T19:44:29.300 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-06T19:44:29.300 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-06T19:44:29.300 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph 2023-07-06T19:44:29.300 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph/libceph.ko 2023-07-06T19:44:29.300 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core 2023-07-06T19:44:29.301 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/failover.ko 2023-07-06T19:44:29.301 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/pktgen.ko 2023-07-06T19:44:29.301 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp 2023-07-06T19:44:29.301 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp.ko 2023-07-06T19:44:29.301 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_diag.ko 2023-07-06T19:44:29.301 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv4.ko 2023-07-06T19:44:29.301 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv6.ko 2023-07-06T19:44:29.301 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver 2023-07-06T19:44:29.302 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver/dns_resolver.ko 2023-07-06T19:44:29.302 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa 2023-07-06T19:44:29.302 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/dsa_core.ko 2023-07-06T19:44:29.302 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_brcm.ko 2023-07-06T19:44:29.302 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_dsa.ko 2023-07-06T19:44:29.302 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_none.ko 2023-07-06T19:44:29.302 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_trailer.ko 2023-07-06T19:44:29.302 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154 2023-07-06T19:44:29.303 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan 2023-07-06T19:44:29.303 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-06T19:44:29.303 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154.ko 2023-07-06T19:44:29.303 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-06T19:44:29.303 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4 2023-07-06T19:44:29.303 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ah4.ko 2023-07-06T19:44:29.303 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/esp4.ko 2023-07-06T19:44:29.303 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/fou.ko 2023-07-06T19:44:29.304 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/gre.ko 2023-07-06T19:44:29.304 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/inet_diag.ko 2023-07-06T19:44:29.304 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_gre.ko 2023-07-06T19:44:29.304 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_tunnel.ko 2023-07-06T19:44:29.304 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_vti.ko 2023-07-06T19:44:29.304 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipcomp.ko 2023-07-06T19:44:29.304 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipip.ko 2023-07-06T19:44:29.305 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter 2023-07-06T19:44:29.305 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-06T19:44:29.305 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-06T19:44:29.305 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-06T19:44:29.305 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-06T19:44:29.305 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-06T19:44:29.305 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-06T19:44:29.305 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-06T19:44:29.306 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-06T19:44:29.306 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-06T19:44:29.306 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-06T19:44:29.306 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-06T19:44:29.306 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-06T19:44:29.306 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-06T19:44:29.306 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-06T19:44:29.306 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-06T19:44:29.307 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-06T19:44:29.307 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-06T19:44:29.307 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-06T19:44:29.307 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-06T19:44:29.307 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-06T19:44:29.307 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-06T19:44:29.307 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_bic.ko 2023-07-06T19:44:29.307 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_dctcp.ko 2023-07-06T19:44:29.308 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_diag.ko 2023-07-06T19:44:29.308 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_highspeed.ko 2023-07-06T19:44:29.308 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_htcp.ko 2023-07-06T19:44:29.308 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_hybla.ko 2023-07-06T19:44:29.308 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_illinois.ko 2023-07-06T19:44:29.308 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_lp.ko 2023-07-06T19:44:29.308 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_scalable.ko 2023-07-06T19:44:29.308 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_vegas.ko 2023-07-06T19:44:29.309 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_veno.ko 2023-07-06T19:44:29.309 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_westwood.ko 2023-07-06T19:44:29.309 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_yeah.ko 2023-07-06T19:44:29.309 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tunnel4.ko 2023-07-06T19:44:29.309 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_diag.ko 2023-07-06T19:44:29.309 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_tunnel.ko 2023-07-06T19:44:29.309 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-06T19:44:29.309 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6 2023-07-06T19:44:29.310 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ah6.ko 2023-07-06T19:44:29.310 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/esp6.ko 2023-07-06T19:44:29.310 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/fou6.ko 2023-07-06T19:44:29.310 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_tunnel.ko 2023-07-06T19:44:29.310 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-06T19:44:29.310 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_vti.ko 2023-07-06T19:44:29.310 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ipcomp6.ko 2023-07-06T19:44:29.310 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter 2023-07-06T19:44:29.311 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-06T19:44:29.311 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-06T19:44:29.311 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-06T19:44:29.311 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-06T19:44:29.311 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-06T19:44:29.311 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-06T19:44:29.311 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-06T19:44:29.311 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-06T19:44:29.312 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-06T19:44:29.312 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-06T19:44:29.312 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-06T19:44:29.312 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-06T19:44:29.312 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-06T19:44:29.312 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-06T19:44:29.312 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-06T19:44:29.312 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-06T19:44:29.313 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-06T19:44:29.313 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-06T19:44:29.313 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-06T19:44:29.313 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-06T19:44:29.313 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-06T19:44:29.313 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-06T19:44:29.313 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-06T19:44:29.313 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-06T19:44:29.314 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/sit.ko 2023-07-06T19:44:29.314 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/tunnel6.ko 2023-07-06T19:44:29.314 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-06T19:44:29.314 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key 2023-07-06T19:44:29.314 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key/af_key.ko 2023-07-06T19:44:29.314 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp 2023-07-06T19:44:29.314 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_core.ko 2023-07-06T19:44:29.314 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-06T19:44:29.315 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_eth.ko 2023-07-06T19:44:29.315 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip.ko 2023-07-06T19:44:29.315 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip6.ko 2023-07-06T19:44:29.315 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_netlink.ko 2023-07-06T19:44:29.315 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ppp.ko 2023-07-06T19:44:29.315 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc 2023-07-06T19:44:29.315 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc/llc.ko 2023-07-06T19:44:29.315 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154 2023-07-06T19:44:29.316 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154/mac802154.ko 2023-07-06T19:44:29.316 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls 2023-07-06T19:44:29.316 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls/mpls_gso.ko 2023-07-06T19:44:29.316 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter 2023-07-06T19:44:29.316 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset 2023-07-06T19:44:29.316 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set.ko 2023-07-06T19:44:29.316 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-06T19:44:29.316 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-06T19:44:29.317 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-06T19:44:29.317 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-06T19:44:29.317 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-06T19:44:29.317 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-06T19:44:29.317 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-06T19:44:29.317 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-06T19:44:29.317 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-06T19:44:29.317 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-06T19:44:29.318 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-06T19:44:29.318 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-06T19:44:29.318 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-06T19:44:29.318 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-06T19:44:29.318 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-06T19:44:29.318 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs 2023-07-06T19:44:29.318 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-06T19:44:29.318 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-06T19:44:29.319 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-06T19:44:29.319 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-06T19:44:29.319 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-06T19:44:29.319 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-06T19:44:29.319 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-06T19:44:29.319 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-06T19:44:29.319 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-06T19:44:29.319 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-06T19:44:29.320 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-06T19:44:29.320 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-06T19:44:29.320 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-06T19:44:29.320 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-06T19:44:29.320 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conncount.ko 2023-07-06T19:44:29.320 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack.ko 2023-07-06T19:44:29.320 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-06T19:44:29.321 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-06T19:44:29.321 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-06T19:44:29.321 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-06T19:44:29.321 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-06T19:44:29.321 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-06T19:44:29.321 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-06T19:44:29.321 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-06T19:44:29.321 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-06T19:44:29.322 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-06T19:44:29.322 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-06T19:44:29.322 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-06T19:44:29.322 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-06T19:44:29.322 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table.ko 2023-07-06T19:44:29.322 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-06T19:44:29.322 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_log_syslog.ko 2023-07-06T19:44:29.322 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat.ko 2023-07-06T19:44:29.323 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-06T19:44:29.323 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-06T19:44:29.323 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_irc.ko 2023-07-06T19:44:29.323 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_sip.ko 2023-07-06T19:44:29.323 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-06T19:44:29.323 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-06T19:44:29.323 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_tables.ko 2023-07-06T19:44:29.323 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink.ko 2023-07-06T19:44:29.324 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-06T19:44:29.324 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_log.ko 2023-07-06T19:44:29.324 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-06T19:44:29.324 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-06T19:44:29.324 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_chain_nat.ko 2023-07-06T19:44:29.324 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_compat.ko 2023-07-06T19:44:29.324 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_ct.ko 2023-07-06T19:44:29.324 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-06T19:44:29.325 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fib.ko 2023-07-06T19:44:29.325 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-06T19:44:29.325 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_hash.ko 2023-07-06T19:44:29.325 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_limit.ko 2023-07-06T19:44:29.325 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_log.ko 2023-07-06T19:44:29.325 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_masq.ko 2023-07-06T19:44:29.325 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_nat.ko 2023-07-06T19:44:29.325 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_numgen.ko 2023-07-06T19:44:29.326 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_queue.ko 2023-07-06T19:44:29.326 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_quota.ko 2023-07-06T19:44:29.326 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_redir.ko 2023-07-06T19:44:29.326 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject.ko 2023-07-06T19:44:29.326 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject_inet.ko 2023-07-06T19:44:29.326 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_socket.ko 2023-07-06T19:44:29.326 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_tunnel.ko 2023-07-06T19:44:29.326 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_AUDIT.ko 2023-07-06T19:44:29.327 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-06T19:44:29.327 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-06T19:44:29.327 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-06T19:44:29.327 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CT.ko 2023-07-06T19:44:29.327 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_DSCP.ko 2023-07-06T19:44:29.327 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HL.ko 2023-07-06T19:44:29.327 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HMARK.ko 2023-07-06T19:44:29.327 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-06T19:44:29.328 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LED.ko 2023-07-06T19:44:29.328 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LOG.ko 2023-07-06T19:44:29.328 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-06T19:44:29.328 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NETMAP.ko 2023-07-06T19:44:29.328 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFLOG.ko 2023-07-06T19:44:29.328 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-06T19:44:29.328 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_RATEEST.ko 2023-07-06T19:44:29.328 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-06T19:44:29.329 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_SECMARK.ko 2023-07-06T19:44:29.329 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-06T19:44:29.329 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-06T19:44:29.329 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TEE.ko 2023-07-06T19:44:29.329 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TPROXY.ko 2023-07-06T19:44:29.329 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TRACE.ko 2023-07-06T19:44:29.329 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_addrtype.ko 2023-07-06T19:44:29.329 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_bpf.ko 2023-07-06T19:44:29.330 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cgroup.ko 2023-07-06T19:44:29.330 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cluster.ko 2023-07-06T19:44:29.330 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_comment.ko 2023-07-06T19:44:29.330 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connbytes.ko 2023-07-06T19:44:29.330 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlabel.ko 2023-07-06T19:44:29.330 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlimit.ko 2023-07-06T19:44:29.330 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connmark.ko 2023-07-06T19:44:29.330 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_conntrack.ko 2023-07-06T19:44:29.331 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cpu.ko 2023-07-06T19:44:29.331 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dccp.ko 2023-07-06T19:44:29.331 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_devgroup.ko 2023-07-06T19:44:29.331 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dscp.ko 2023-07-06T19:44:29.331 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ecn.ko 2023-07-06T19:44:29.331 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_esp.ko 2023-07-06T19:44:29.331 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hashlimit.ko 2023-07-06T19:44:29.331 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_helper.ko 2023-07-06T19:44:29.332 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hl.ko 2023-07-06T19:44:29.332 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipcomp.ko 2023-07-06T19:44:29.332 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_iprange.ko 2023-07-06T19:44:29.332 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipvs.ko 2023-07-06T19:44:29.332 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_l2tp.ko 2023-07-06T19:44:29.332 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_length.ko 2023-07-06T19:44:29.332 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_limit.ko 2023-07-06T19:44:29.332 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mac.ko 2023-07-06T19:44:29.333 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mark.ko 2023-07-06T19:44:29.333 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_multiport.ko 2023-07-06T19:44:29.333 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nat.ko 2023-07-06T19:44:29.333 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nfacct.ko 2023-07-06T19:44:29.333 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_osf.ko 2023-07-06T19:44:29.333 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_owner.ko 2023-07-06T19:44:29.333 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_physdev.ko 2023-07-06T19:44:29.333 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_pkttype.ko 2023-07-06T19:44:29.334 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_policy.ko 2023-07-06T19:44:29.334 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_quota.ko 2023-07-06T19:44:29.334 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_rateest.ko 2023-07-06T19:44:29.334 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_realm.ko 2023-07-06T19:44:29.334 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_recent.ko 2023-07-06T19:44:29.334 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_sctp.ko 2023-07-06T19:44:29.334 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_set.ko 2023-07-06T19:44:29.334 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_socket.ko 2023-07-06T19:44:29.335 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_state.ko 2023-07-06T19:44:29.335 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_statistic.ko 2023-07-06T19:44:29.335 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_string.ko 2023-07-06T19:44:29.335 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_tcpmss.ko 2023-07-06T19:44:29.335 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_time.ko 2023-07-06T19:44:29.335 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_u32.ko 2023-07-06T19:44:29.335 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink 2023-07-06T19:44:29.335 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink/netlink_diag.ko 2023-07-06T19:44:29.336 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh 2023-07-06T19:44:29.336 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh/nsh.ko 2023-07-06T19:44:29.336 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch 2023-07-06T19:44:29.336 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/openvswitch.ko 2023-07-06T19:44:29.336 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-geneve.ko 2023-07-06T19:44:29.336 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-gre.ko 2023-07-06T19:44:29.336 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-vxlan.ko 2023-07-06T19:44:29.337 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet 2023-07-06T19:44:29.337 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet/af_packet_diag.ko 2023-07-06T19:44:29.337 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds 2023-07-06T19:44:29.337 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds.ko 2023-07-06T19:44:29.337 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_rdma.ko 2023-07-06T19:44:29.337 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_tcp.ko 2023-07-06T19:44:29.337 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched 2023-07-06T19:44:29.337 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_csum.ko 2023-07-06T19:44:29.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_gact.ko 2023-07-06T19:44:29.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_ipt.ko 2023-07-06T19:44:29.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_mirred.ko 2023-07-06T19:44:29.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_nat.ko 2023-07-06T19:44:29.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_pedit.ko 2023-07-06T19:44:29.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_police.ko 2023-07-06T19:44:29.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_simple.ko 2023-07-06T19:44:29.338 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_skbedit.ko 2023-07-06T19:44:29.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_vlan.ko 2023-07-06T19:44:29.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_basic.ko 2023-07-06T19:44:29.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_bpf.ko 2023-07-06T19:44:29.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_flow.ko 2023-07-06T19:44:29.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_fw.ko 2023-07-06T19:44:29.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_route.ko 2023-07-06T19:44:29.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_u32.ko 2023-07-06T19:44:29.339 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_cmp.ko 2023-07-06T19:44:29.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_ipset.ko 2023-07-06T19:44:29.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_meta.ko 2023-07-06T19:44:29.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_nbyte.ko 2023-07-06T19:44:29.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_text.ko 2023-07-06T19:44:29.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_u32.ko 2023-07-06T19:44:29.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_choke.ko 2023-07-06T19:44:29.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_codel.ko 2023-07-06T19:44:29.340 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_drr.ko 2023-07-06T19:44:29.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_fq.ko 2023-07-06T19:44:29.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_gred.ko 2023-07-06T19:44:29.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hfsc.ko 2023-07-06T19:44:29.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hhf.ko 2023-07-06T19:44:29.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_htb.ko 2023-07-06T19:44:29.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_ingress.ko 2023-07-06T19:44:29.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio.ko 2023-07-06T19:44:29.341 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio_lib.ko 2023-07-06T19:44:29.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_multiq.ko 2023-07-06T19:44:29.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_netem.ko 2023-07-06T19:44:29.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_pie.ko 2023-07-06T19:44:29.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_plug.ko 2023-07-06T19:44:29.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_prio.ko 2023-07-06T19:44:29.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_qfq.ko 2023-07-06T19:44:29.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_red.ko 2023-07-06T19:44:29.342 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfb.ko 2023-07-06T19:44:29.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfq.ko 2023-07-06T19:44:29.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_tbf.ko 2023-07-06T19:44:29.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_teql.ko 2023-07-06T19:44:29.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp 2023-07-06T19:44:29.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp.ko 2023-07-06T19:44:29.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp_diag.ko 2023-07-06T19:44:29.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc 2023-07-06T19:44:29.343 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss 2023-07-06T19:44:29.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-06T19:44:29.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-06T19:44:29.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/sunrpc.ko 2023-07-06T19:44:29.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma 2023-07-06T19:44:29.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-06T19:44:29.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc 2023-07-06T19:44:29.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/diag.ko 2023-07-06T19:44:29.344 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/tipc.ko 2023-07-06T19:44:29.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix 2023-07-06T19:44:29.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix/unix_diag.ko 2023-07-06T19:44:29.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock 2023-07-06T19:44:29.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-06T19:44:29.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-06T19:44:29.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock.ko 2023-07-06T19:44:29.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-06T19:44:29.345 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-06T19:44:29.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm 2023-07-06T19:44:29.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-06T19:44:29.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security 2023-07-06T19:44:29.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys 2023-07-06T19:44:29.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys 2023-07-06T19:44:29.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-06T19:44:29.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys 2023-07-06T19:44:29.346 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys/trusted.ko 2023-07-06T19:44:29.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt 2023-07-06T19:44:29.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib 2023-07-06T19:44:29.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib/irqbypass.ko 2023-07-06T19:44:29.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias 2023-07-06T19:44:29.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias.bin 2023-07-06T19:44:29.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin 2023-07-06T19:44:29.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.bin 2023-07-06T19:44:29.347 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.modinfo 2023-07-06T19:44:29.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep 2023-07-06T19:44:29.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep.bin 2023-07-06T19:44:29.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.devname 2023-07-06T19:44:29.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.order 2023-07-06T19:44:29.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.softdep 2023-07-06T19:44:29.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols 2023-07-06T19:44:29.348 INFO:teuthology.orchestra.run.smithi069.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols.bin 2023-07-06T19:44:29.349 DEBUG:teuthology.task.kernel:get_image_version: 6.4.0-g84139c18d56f 2023-07-06T19:44:29.349 DEBUG:teuthology.orchestra.run.smithi069:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-06T19:44:29.375 INFO:teuthology.orchestra.run.smithi069.stdout:postinstall scriptlet (using /bin/sh): 2023-07-06T19:44:29.375 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.4.0-g84139c18d56f -a -r /boot/System.map-6.4.0-g84139c18d56f ]; then 2023-07-06T19:44:29.375 INFO:teuthology.orchestra.run.smithi069.stdout:cp /boot/vmlinuz-6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm 2023-07-06T19:44:29.376 INFO:teuthology.orchestra.run.smithi069.stdout:cp /boot/System.map-6.4.0-g84139c18d56f /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T19:44:29.376 INFO:teuthology.orchestra.run.smithi069.stdout:rm -f /boot/vmlinuz-6.4.0-g84139c18d56f /boot/System.map-6.4.0-g84139c18d56f 2023-07-06T19:44:29.376 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/installkernel 6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T19:44:29.376 INFO:teuthology.orchestra.run.smithi069.stdout:rm -f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T19:44:29.376 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2023-07-06T19:44:29.376 INFO:teuthology.orchestra.run.smithi069.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-06T19:44:29.376 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-06T19:44:29.377 INFO:teuthology.orchestra.run.smithi069.stdout:new-kernel-pkg --remove 6.4.0-g84139c18d56f --rminitrd --initrdfile=/boot/initramfs-6.4.0-g84139c18d56f.img 2023-07-06T19:44:29.377 INFO:teuthology.orchestra.run.smithi069.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-06T19:44:29.377 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-install remove 6.4.0-g84139c18d56f 2023-07-06T19:44:29.377 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2023-07-06T19:44:29.377 INFO:teuthology.orchestra.run.smithi069.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-06T19:44:29.377 INFO:teuthology.orchestra.run.smithi069.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-06T19:44:29.377 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/update-bootloader --remove 6.4.0-g84139c18d56f 2023-07-06T19:44:29.377 INFO:teuthology.orchestra.run.smithi069.stdout:fi 2023-07-06T19:44:29.378 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -qi grub2-tools 2023-07-06T19:44:29.455 INFO:teuthology.orchestra.run.smithi069.stdout:Name : grub2-tools 2023-07-06T19:44:29.455 INFO:teuthology.orchestra.run.smithi069.stdout:Epoch : 1 2023-07-06T19:44:29.455 INFO:teuthology.orchestra.run.smithi069.stdout:Version : 2.02 2023-07-06T19:44:29.456 INFO:teuthology.orchestra.run.smithi069.stdout:Release : 123.el8 2023-07-06T19:44:29.456 INFO:teuthology.orchestra.run.smithi069.stdout:Architecture: x86_64 2023-07-06T19:44:29.456 INFO:teuthology.orchestra.run.smithi069.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-06T19:44:29.456 INFO:teuthology.orchestra.run.smithi069.stdout:Group : System Environment/Base 2023-07-06T19:44:29.456 INFO:teuthology.orchestra.run.smithi069.stdout:Size : 9265450 2023-07-06T19:44:29.456 INFO:teuthology.orchestra.run.smithi069.stdout:License : GPLv3+ 2023-07-06T19:44:29.456 INFO:teuthology.orchestra.run.smithi069.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-06T19:44:29.456 INFO:teuthology.orchestra.run.smithi069.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-06T19:44:29.457 INFO:teuthology.orchestra.run.smithi069.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-06T19:44:29.457 INFO:teuthology.orchestra.run.smithi069.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-06T19:44:29.457 INFO:teuthology.orchestra.run.smithi069.stdout:Relocations : (not relocatable) 2023-07-06T19:44:29.457 INFO:teuthology.orchestra.run.smithi069.stdout:Packager : Red Hat, Inc. 2023-07-06T19:44:29.457 INFO:teuthology.orchestra.run.smithi069.stdout:Vendor : Red Hat, Inc. 2023-07-06T19:44:29.457 INFO:teuthology.orchestra.run.smithi069.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-06T19:44:29.457 INFO:teuthology.orchestra.run.smithi069.stdout:Summary : Support tools for GRUB. 2023-07-06T19:44:29.457 INFO:teuthology.orchestra.run.smithi069.stdout:Description : 2023-07-06T19:44:29.458 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T19:44:29.458 INFO:teuthology.orchestra.run.smithi069.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-06T19:44:29.458 INFO:teuthology.orchestra.run.smithi069.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-06T19:44:29.458 INFO:teuthology.orchestra.run.smithi069.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-06T19:44:29.458 INFO:teuthology.orchestra.run.smithi069.stdout:hardware devices. 2023-07-06T19:44:29.458 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T19:44:29.458 INFO:teuthology.orchestra.run.smithi069.stdout:This subpackage provides tools for support of all platforms. 2023-07-06T19:44:29.459 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-06T19:44:29.459 INFO:teuthology.task.kernel:Updating grub on smithi069 to boot 6.4.0-g84139c18d56f 2023-07-06T19:44:29.459 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-06T19:44:33.604 INFO:teuthology.orchestra.run.smithi069.stderr:Generating grub configuration file ... 2023-07-06T19:44:35.850 INFO:teuthology.orchestra.run.smithi069.stderr:done 2023-07-06T19:44:35.852 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2023-07-06T19:44:35.873 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.s0UsU2oAMA 2023-07-06T19:44:35.873 DEBUG:teuthology.orchestra.run.smithi069:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.s0UsU2oAMA 2023-07-06T19:44:35.947 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.s0UsU2oAMA 2023-07-06T19:44:36.103 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.s0UsU2oAMA is 6KB 2023-07-06T19:44:36.150 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.s0UsU2oAMA 2023-07-06T19:44:36.166 DEBUG:teuthology.orchestra.run.smithi069:> sudo /bin/ls /boot/loader/entries || true 2023-07-06T19:44:36.237 INFO:teuthology.orchestra.run.smithi069.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-06T19:44:36.237 INFO:teuthology.orchestra.run.smithi069.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-06T19:44:36.237 INFO:teuthology.orchestra.run.smithi069.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.4.0-g84139c18d56f.conf 2023-07-06T19:44:36.238 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.4.0-g84139c18d56f 2023-07-06T19:44:36.391 DEBUG:teuthology.orchestra.run.smithi069:> sudo shutdown -r now 2023-07-06T19:45:06.394 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-07-06T19:45:06.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-07-06T19:45:06.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:45:24.848 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-07-06T19:45:33.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-07-06T19:45:33.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:45:36.912 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-07-06T19:45:48.913 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-07-06T19:45:48.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:46:07.344 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-07-06T19:46:22.346 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-07-06T19:46:22.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T19:46:22.903 DEBUG:teuthology.orchestra.run.smithi069:> true 2023-07-06T19:46:23.328 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-07-06T19:46:23.328 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.4.0-g84139c18d56f"... 2023-07-06T19:46:23.329 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2023-07-06T19:46:23.370 INFO:teuthology.orchestra.run.smithi069.stdout:6.4.0-g84139c18d56f 2023-07-06T19:46:23.371 DEBUG:teuthology.task.kernel:current kernel version is 6.4.0-g84139c18d56f vs 6.4.0-g84139c18d56f 2023-07-06T19:46:23.371 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-06T19:46:23.371 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-06T19:46:24.372 DEBUG:teuthology.parallel:result is None 2023-07-06T19:46:24.372 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-06T19:46:24.378 INFO:teuthology.task.internal:Creating test directory... 2023-07-06T19:46:24.379 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T19:46:24.383 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T19:46:24.386 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T19:46:24.421 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-06T19:46:24.428 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-06T19:46:24.448 INFO:teuthology.task.internal:Creating archive directory... 2023-07-06T19:46:24.448 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T19:46:24.466 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T19:46:24.468 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T19:46:24.520 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-06T19:46:24.526 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-06T19:46:24.527 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T19:46:24.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T19:46:24.557 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T19:46:24.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T19:46:24.577 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T19:46:24.581 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T19:46:24.600 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T19:46:24.655 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T19:46:24.658 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T19:46:24.672 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T19:46:24.673 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T19:46:24.747 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T19:46:24.786 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T19:46:24.790 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-06T19:46:24.797 INFO:teuthology.task.internal:Configuring sudo... 2023-07-06T19:46:24.797 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T19:46:24.801 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T19:46:24.803 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T19:46:24.870 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-06T19:46:24.879 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-06T19:46:24.879 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T19:46:24.883 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T19:46:24.885 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T19:46:24.923 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T19:46:25.041 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T19:46:25.108 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T19:46:25.182 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T19:46:25.260 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T19:46:25.260 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T19:46:25.327 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T19:46:25.379 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T19:46:25.431 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T19:46:25.513 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T19:46:25.561 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-07-06T19:46:25.562 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T19:46:25.635 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T19:46:25.685 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T19:46:25.741 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T19:46:25.831 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T19:46:25.871 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-06T19:46:25.871 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T19:46:25.940 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2023-07-06T19:46:25.944 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2023-07-06T19:46:25.946 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2023-07-06T19:46:26.036 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T19:46:26.048 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T19:46:26.062 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T19:46:26.389 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-06T19:46:26.396 INFO:teuthology.task.internal:Starting timer... 2023-07-06T19:46:26.397 INFO:teuthology.run_tasks:Running task pcp... 2023-07-06T19:46:26.430 INFO:teuthology.run_tasks:Running task selinux... 2023-07-06T19:46:26.460 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-06T19:46:26.460 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2023-07-06T19:46:26.561 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2023-07-06T19:46:26.564 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2023-07-06T19:46:26.639 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: 2023-07-06T19:46:26.640 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2023-07-06T19:46:26.693 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2023-07-06T19:46:26.695 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-06T19:46:26.696 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2023-07-06T19:46:26.741 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2023-07-06T19:46:26.742 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2023-07-06T19:46:26.778 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2023-07-06T19:46:26.779 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2023-07-06T19:46:26.801 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2023-07-06T19:46:26.802 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi069.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi098.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi156.front.sepia.ceph.com': 'permissive'} 2023-07-06T19:46:26.802 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T19:46:26.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T19:46:26.870 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T19:46:26.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T19:46:26.898 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T19:46:26.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T19:46:26.927 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-06T19:46:26.927 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2023-07-06T19:46:26.994 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2023-07-06T19:46:27.020 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2023-07-06T19:46:27.047 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-06T19:46:27.058 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-07-06T19:46:27.058 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-06T19:46:27.078 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-06T19:46:27.080 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi069.front.sepia.ceph.com,smithi098.front.sepia.ceph.com,smithi156.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-06T20:03:06.352 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi069.front.sepia.ceph.com'), Remote(name='ubuntu@smithi098.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com')] 2023-07-06T20:03:06.354 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-07-06T20:03:06.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T20:03:06.447 DEBUG:teuthology.orchestra.run.smithi069:> true 2023-07-06T20:03:06.570 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-07-06T20:03:06.571 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-07-06T20:03:06.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T20:03:06.646 DEBUG:teuthology.orchestra.run.smithi098:> true 2023-07-06T20:03:06.712 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-07-06T20:03:06.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-07-06T20:03:06.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T20:03:06.783 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-07-06T20:03:06.857 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-07-06T20:03:06.858 INFO:teuthology.run_tasks:Running task clock... 2023-07-06T20:03:06.871 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-06T20:03:06.872 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T20:03:06.873 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T20:03:06.877 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T20:03:06.878 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T20:03:06.880 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T20:03:06.880 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T20:03:06.915 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T20:03:06.931 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T20:03:06.933 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T20:03:06.948 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T20:03:06.948 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T20:03:06.982 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T20:03:07.006 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2023-07-06T20:03:07.014 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2023-07-06T20:03:07.017 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2023-07-06T20:03:07.028 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2023-07-06T20:03:07.033 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T20:03:07.045 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T20:03:07.047 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T20:03:07.055 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T20:03:07.080 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2023-07-06T20:03:07.109 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2023-07-06T20:03:07.116 INFO:teuthology.orchestra.run.smithi098.stderr:bash: ntpq: command not found 2023-07-06T20:03:07.123 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T20:03:07.124 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2023-07-06T20:03:07.124 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:03:07.125 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:03:07.125 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:03:07.125 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:03:07.132 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-07-06T20:03:07.136 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T20:03:07.136 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-07-06T20:03:07.136 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:03:07.137 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:03:07.137 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:03:07.137 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:03:07.141 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T20:03:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T20:03:07.270 INFO:teuthology.orchestra.run.smithi069.stderr:bash: ntpq: command not found 2023-07-06T20:03:07.277 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T20:03:07.277 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2023-07-06T20:03:07.277 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:03:07.277 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:03:07.277 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:03:07.277 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:03:07.278 INFO:teuthology.run_tasks:Running task pexec... 2023-07-06T20:03:07.289 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-06T20:03:07.290 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi069.front.sepia.ceph.com 2023-07-06T20:03:07.290 DEBUG:teuthology.orchestra.run.smithi069:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-06T20:03:07.292 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi098.front.sepia.ceph.com 2023-07-06T20:03:07.292 DEBUG:teuthology.orchestra.run.smithi098:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-06T20:03:07.292 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi156.front.sepia.ceph.com 2023-07-06T20:03:07.293 DEBUG:teuthology.orchestra.run.smithi156:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-06T20:03:07.664 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-07-06T20:03:07.667 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-07-06T20:03:07.774 INFO:teuthology.orchestra.run.smithi069.stdout:Updating Subscription Management repositories. 2023-07-06T20:03:09.237 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:02:03 ago on Thu 06 Jul 2023 08:01:05 PM UTC. 2023-07-06T20:03:09.285 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:09:11 ago on Thu 06 Jul 2023 07:53:57 PM UTC. 2023-07-06T20:03:09.571 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:06:03 ago on Thu 06 Jul 2023 07:57:06 PM UTC. 2023-07-06T20:03:11.011 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-07-06T20:03:11.012 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-07-06T20:03:11.012 INFO:teuthology.orchestra.run.smithi156.stdout: Package Architecture Version Repository Size 2023-07-06T20:03:11.012 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-07-06T20:03:11.013 INFO:teuthology.orchestra.run.smithi156.stdout:Resetting modules: 2023-07-06T20:03:11.013 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools 2023-07-06T20:03:11.013 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:03:11.013 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-07-06T20:03:11.014 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-07-06T20:03:11.014 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:03:11.114 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2023-07-06T20:03:11.115 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2023-07-06T20:03:11.115 INFO:teuthology.orchestra.run.smithi098.stdout: Package Architecture Version Repository Size 2023-07-06T20:03:11.115 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2023-07-06T20:03:11.115 INFO:teuthology.orchestra.run.smithi098.stdout:Resetting modules: 2023-07-06T20:03:11.116 INFO:teuthology.orchestra.run.smithi098.stdout: container-tools 2023-07-06T20:03:11.116 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T20:03:11.116 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2023-07-06T20:03:11.116 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2023-07-06T20:03:11.116 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T20:03:11.346 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2023-07-06T20:03:11.347 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2023-07-06T20:03:11.347 INFO:teuthology.orchestra.run.smithi069.stdout: Package Architecture Version Repository Size 2023-07-06T20:03:11.347 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2023-07-06T20:03:11.347 INFO:teuthology.orchestra.run.smithi069.stdout:Resetting modules: 2023-07-06T20:03:11.347 INFO:teuthology.orchestra.run.smithi069.stdout: container-tools 2023-07-06T20:03:11.347 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:03:11.347 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2023-07-06T20:03:11.348 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2023-07-06T20:03:11.348 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:03:11.993 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:09:14 ago on Thu 06 Jul 2023 07:53:57 PM UTC. 2023-07-06T20:03:12.047 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:02:06 ago on Thu 06 Jul 2023 08:01:05 PM UTC. 2023-07-06T20:03:12.586 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:06:06 ago on Thu 06 Jul 2023 07:57:06 PM UTC. 2023-07-06T20:03:22.399 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-07-06T20:03:22.893 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-07-06T20:03:23.623 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2023-07-06T20:03:23.861 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2023-07-06T20:03:24.141 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-07-06T20:03:24.444 INFO:teuthology.orchestra.run.smithi069.stdout:Updating Subscription Management repositories. 2023-07-06T20:03:24.546 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:09:27 ago on Thu 06 Jul 2023 07:53:57 PM UTC. 2023-07-06T20:03:25.741 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:02:20 ago on Thu 06 Jul 2023 08:01:05 PM UTC. 2023-07-06T20:03:26.216 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:06:19 ago on Thu 06 Jul 2023 07:57:06 PM UTC. 2023-07-06T20:03:26.274 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-07-06T20:03:26.366 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-07-06T20:03:26.366 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-07-06T20:03:26.366 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-07-06T20:03:26.366 INFO:teuthology.orchestra.run.smithi156.stdout:Installing group/module packages: 2023-07-06T20:03:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-06T20:03:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-06T20:03:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-06T20:03:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-06T20:03:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-06T20:03:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-06T20:03:26.367 INFO:teuthology.orchestra.run.smithi156.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-06T20:03:26.367 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-07-06T20:03:26.368 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-06T20:03:26.368 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-06T20:03:26.368 INFO:teuthology.orchestra.run.smithi156.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-06T20:03:26.368 INFO:teuthology.orchestra.run.smithi156.stdout:Installing module profiles: 2023-07-06T20:03:26.369 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools/common 2023-07-06T20:03:26.369 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:03:26.370 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-07-06T20:03:26.370 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools rhel8 2023-07-06T20:03:26.370 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:03:26.370 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-07-06T20:03:26.370 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-07-06T20:03:26.370 INFO:teuthology.orchestra.run.smithi156.stdout:Install 10 Packages 2023-07-06T20:03:26.370 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:03:26.371 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 18 M 2023-07-06T20:03:26.371 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 65 M 2023-07-06T20:03:26.371 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-07-06T20:03:26.466 INFO:teuthology.orchestra.run.smithi156.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 959 kB/s | 94 kB 00:00 2023-07-06T20:03:26.508 INFO:teuthology.orchestra.run.smithi156.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 292 kB/s | 41 kB 00:00 2023-07-06T20:03:26.534 INFO:teuthology.orchestra.run.smithi156.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 150 kB/s | 25 kB 00:00 2023-07-06T20:03:26.550 INFO:teuthology.orchestra.run.smithi156.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 589 kB/s | 49 kB 00:00 2023-07-06T20:03:26.610 INFO:teuthology.orchestra.run.smithi156.stdout:(5/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-06T20:03:26.684 INFO:teuthology.orchestra.run.smithi156.stdout:(6/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.9 MB/s | 524 kB 00:00 2023-07-06T20:03:26.735 INFO:teuthology.orchestra.run.smithi156.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.7 MB/s | 213 kB 00:00 2023-07-06T20:03:26.918 INFO:teuthology.orchestra.run.smithi156.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 21 MB/s | 8.0 MB 00:00 2023-07-06T20:03:26.993 INFO:teuthology.orchestra.run.smithi156.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 8.5 MB/s | 2.2 MB 00:00 2023-07-06T20:03:27.094 INFO:teuthology.orchestra.run.smithi156.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 17 MB/s | 7.0 MB 00:00 2023-07-06T20:03:27.094 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-07-06T20:03:27.095 INFO:teuthology.orchestra.run.smithi156.stdout:Total 25 MB/s | 18 MB 00:00 2023-07-06T20:03:27.293 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-07-06T20:03:27.312 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-07-06T20:03:27.312 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-07-06T20:03:27.448 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2023-07-06T20:03:27.496 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-07-06T20:03:27.500 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-07-06T20:03:27.542 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================================ 2023-07-06T20:03:27.542 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2023-07-06T20:03:27.543 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================================ 2023-07-06T20:03:27.543 INFO:teuthology.orchestra.run.smithi098.stdout:Installing group/module packages: 2023-07-06T20:03:27.543 INFO:teuthology.orchestra.run.smithi098.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-06T20:03:27.543 INFO:teuthology.orchestra.run.smithi098.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-06T20:03:27.543 INFO:teuthology.orchestra.run.smithi098.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-06T20:03:27.543 INFO:teuthology.orchestra.run.smithi098.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-06T20:03:27.544 INFO:teuthology.orchestra.run.smithi098.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-06T20:03:27.544 INFO:teuthology.orchestra.run.smithi098.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-06T20:03:27.544 INFO:teuthology.orchestra.run.smithi098.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-06T20:03:27.544 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2023-07-06T20:03:27.544 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-06T20:03:27.544 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-06T20:03:27.544 INFO:teuthology.orchestra.run.smithi098.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-06T20:03:27.545 INFO:teuthology.orchestra.run.smithi098.stdout:Installing module profiles: 2023-07-06T20:03:27.545 INFO:teuthology.orchestra.run.smithi098.stdout: container-tools/common 2023-07-06T20:03:27.545 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T20:03:27.545 INFO:teuthology.orchestra.run.smithi098.stdout:Enabling module streams: 2023-07-06T20:03:27.545 INFO:teuthology.orchestra.run.smithi098.stdout: container-tools rhel8 2023-07-06T20:03:27.545 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T20:03:27.545 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2023-07-06T20:03:27.546 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================================ 2023-07-06T20:03:27.546 INFO:teuthology.orchestra.run.smithi098.stdout:Install 10 Packages 2023-07-06T20:03:27.546 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T20:03:27.547 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 18 M 2023-07-06T20:03:27.547 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 65 M 2023-07-06T20:03:27.547 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2023-07-06T20:03:27.724 INFO:teuthology.orchestra.run.smithi098.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 139 kB/s | 25 kB 00:00 2023-07-06T20:03:27.787 INFO:teuthology.orchestra.run.smithi098.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 168 kB/s | 41 kB 00:00 2023-07-06T20:03:27.816 INFO:teuthology.orchestra.run.smithi098.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 346 kB/s | 94 kB 00:00 2023-07-06T20:03:27.844 INFO:teuthology.orchestra.run.smithi098.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 421 kB/s | 49 kB 00:00 2023-07-06T20:03:27.917 INFO:teuthology.orchestra.run.smithi098.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.0 MB/s | 524 kB 00:00 2023-07-06T20:03:27.978 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2023-07-06T20:03:28.031 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-07-06T20:03:28.078 INFO:teuthology.orchestra.run.smithi069.stdout:============================================================================================================ 2023-07-06T20:03:28.078 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2023-07-06T20:03:28.079 INFO:teuthology.orchestra.run.smithi069.stdout:============================================================================================================ 2023-07-06T20:03:28.079 INFO:teuthology.orchestra.run.smithi069.stdout:Installing group/module packages: 2023-07-06T20:03:28.079 INFO:teuthology.orchestra.run.smithi069.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-06T20:03:28.079 INFO:teuthology.orchestra.run.smithi069.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-06T20:03:28.079 INFO:teuthology.orchestra.run.smithi069.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-06T20:03:28.079 INFO:teuthology.orchestra.run.smithi069.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-06T20:03:28.079 INFO:teuthology.orchestra.run.smithi069.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-06T20:03:28.080 INFO:teuthology.orchestra.run.smithi069.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-06T20:03:28.080 INFO:teuthology.orchestra.run.smithi069.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-06T20:03:28.080 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2023-07-06T20:03:28.080 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-06T20:03:28.080 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-06T20:03:28.080 INFO:teuthology.orchestra.run.smithi069.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-06T20:03:28.080 INFO:teuthology.orchestra.run.smithi069.stdout:Installing module profiles: 2023-07-06T20:03:28.081 INFO:teuthology.orchestra.run.smithi069.stdout: container-tools/common 2023-07-06T20:03:28.081 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:03:28.081 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling module streams: 2023-07-06T20:03:28.081 INFO:teuthology.orchestra.run.smithi069.stdout: container-tools rhel8 2023-07-06T20:03:28.081 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:03:28.081 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2023-07-06T20:03:28.081 INFO:teuthology.orchestra.run.smithi069.stdout:============================================================================================================ 2023-07-06T20:03:28.081 INFO:teuthology.orchestra.run.smithi069.stdout:Install 10 Packages 2023-07-06T20:03:28.082 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:03:28.082 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 18 M 2023-07-06T20:03:28.082 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 65 M 2023-07-06T20:03:28.083 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2023-07-06T20:03:28.150 INFO:teuthology.orchestra.run.smithi098.stdout:(6/10): buildah-1.26.2-1.module+el8.6.0+16771+2 24 MB/s | 8.0 MB 00:00 2023-07-06T20:03:28.176 INFO:teuthology.orchestra.run.smithi098.stdout:(7/10): python3-podman-4.0.1-1.module+el8.6.0+1 448 kB/s | 149 kB 00:00 2023-07-06T20:03:28.201 INFO:teuthology.orchestra.run.smithi098.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 750 kB/s | 213 kB 00:00 2023-07-06T20:03:28.218 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-06T20:03:28.268 INFO:teuthology.orchestra.run.smithi069.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 135 kB/s | 25 kB 00:00 2023-07-06T20:03:28.291 INFO:teuthology.orchestra.run.smithi069.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 448 kB/s | 94 kB 00:00 2023-07-06T20:03:28.325 INFO:teuthology.orchestra.run.smithi069.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 168 kB/s | 41 kB 00:00 2023-07-06T20:03:28.414 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-06T20:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 26 MB/s | 7.0 MB 00:00 2023-07-06T20:03:28.491 INFO:teuthology.orchestra.run.smithi069.stdout:(4/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.6 MB/s | 524 kB 00:00 2023-07-06T20:03:28.518 INFO:teuthology.orchestra.run.smithi098.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 6.4 MB/s | 2.2 MB 00:00 2023-07-06T20:03:28.519 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2023-07-06T20:03:28.519 INFO:teuthology.orchestra.run.smithi098.stdout:Total 19 MB/s | 18 MB 00:00 2023-07-06T20:03:28.542 INFO:teuthology.orchestra.run.smithi069.stdout:(5/10): udica-0.2.6-4.module+el8.6.0+16771+28df 179 kB/s | 49 kB 00:00 2023-07-06T20:03:28.575 INFO:teuthology.orchestra.run.smithi069.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.7 MB/s | 149 kB 00:00 2023-07-06T20:03:28.608 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-06T20:03:28.684 INFO:teuthology.orchestra.run.smithi069.stdout:(7/10): buildah-1.26.2-1.module+el8.6.0+16771+2 22 MB/s | 8.0 MB 00:00 2023-07-06T20:03:28.718 INFO:teuthology.orchestra.run.smithi069.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.2 MB/s | 213 kB 00:00 2023-07-06T20:03:28.719 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2023-07-06T20:03:28.737 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2023-07-06T20:03:28.737 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2023-07-06T20:03:28.802 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-06T20:03:28.868 INFO:teuthology.orchestra.run.smithi069.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 24 MB/s | 7.0 MB 00:00 2023-07-06T20:03:28.920 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2023-07-06T20:03:28.924 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2023-07-06T20:03:28.935 INFO:teuthology.orchestra.run.smithi069.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 8.8 MB/s | 2.2 MB 00:00 2023-07-06T20:03:28.935 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2023-07-06T20:03:28.935 INFO:teuthology.orchestra.run.smithi069.stdout:Total 21 MB/s | 18 MB 00:00 2023-07-06T20:03:29.159 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-06T20:03:29.264 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2023-07-06T20:03:29.287 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2023-07-06T20:03:29.287 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2023-07-06T20:03:29.479 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2023-07-06T20:03:29.681 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-06T20:03:29.810 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2023-07-06T20:03:29.814 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2023-07-06T20:03:29.910 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-06T20:03:29.947 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-06T20:03:30.121 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-06T20:03:30.325 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2023-07-06T20:03:30.378 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-06T20:03:30.502 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-06T20:03:30.672 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-06T20:03:30.780 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-06T20:03:30.813 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-06T20:03:30.897 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-06T20:03:31.043 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-06T20:03:31.132 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-06T20:03:31.204 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-06T20:03:31.414 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-06T20:03:31.509 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-06T20:03:31.584 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-06T20:03:31.688 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-06T20:03:31.688 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-06T20:03:31.688 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-06T20:03:31.689 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-06T20:03:31.689 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-06T20:03:31.689 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-06T20:03:31.689 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-06T20:03:31.689 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-06T20:03:31.689 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-06T20:03:31.689 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-06T20:03:32.336 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-06T20:03:32.488 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-06T20:03:32.748 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-06T20:03:32.894 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-06T20:03:32.894 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:09:35 ago on Thu 06 Jul 2023 07:53:57 PM UTC. 2023-07-06T20:03:32.959 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-06T20:03:33.152 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-06T20:03:33.259 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-06T20:03:33.468 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-06T20:03:33.468 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-06T20:03:33.468 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-06T20:03:33.468 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-06T20:03:33.469 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-06T20:03:33.469 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-06T20:03:33.469 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-06T20:03:33.469 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-06T20:03:33.470 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-06T20:03:33.470 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-06T20:03:33.543 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-06T20:03:33.736 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-06T20:03:33.936 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-06T20:03:34.324 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-06T20:03:34.324 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-06T20:03:34.325 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-06T20:03:34.325 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-06T20:03:34.325 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-06T20:03:34.325 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-06T20:03:34.326 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-06T20:03:34.326 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-06T20:03:34.326 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-06T20:03:34.326 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-06T20:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-06T20:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:02:29 ago on Thu 06 Jul 2023 08:01:05 PM UTC. 2023-07-06T20:03:35.718 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-06T20:03:35.718 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:06:29 ago on Thu 06 Jul 2023 07:57:06 PM UTC. 2023-07-06T20:03:42.140 INFO:teuthology.orchestra.run.smithi069.stdout:Installed products updated. 2023-07-06T20:03:42.368 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:03:42.368 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2023-07-06T20:03:42.368 INFO:teuthology.orchestra.run.smithi069.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-06T20:03:42.368 INFO:teuthology.orchestra.run.smithi069.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-06T20:03:42.368 INFO:teuthology.orchestra.run.smithi069.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-06T20:03:42.369 INFO:teuthology.orchestra.run.smithi069.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-06T20:03:42.369 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-06T20:03:42.369 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-06T20:03:42.369 INFO:teuthology.orchestra.run.smithi069.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-06T20:03:42.369 INFO:teuthology.orchestra.run.smithi069.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-06T20:03:42.369 INFO:teuthology.orchestra.run.smithi069.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-06T20:03:42.369 INFO:teuthology.orchestra.run.smithi069.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-06T20:03:42.370 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:03:42.370 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2023-07-06T20:03:42.626 DEBUG:teuthology.parallel:result is None 2023-07-06T20:03:43.607 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-07-06T20:03:43.806 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:03:43.807 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-07-06T20:03:43.807 INFO:teuthology.orchestra.run.smithi156.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-06T20:03:43.807 INFO:teuthology.orchestra.run.smithi156.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-06T20:03:43.807 INFO:teuthology.orchestra.run.smithi156.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-06T20:03:43.808 INFO:teuthology.orchestra.run.smithi156.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-06T20:03:43.808 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-06T20:03:43.808 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-06T20:03:43.808 INFO:teuthology.orchestra.run.smithi156.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-06T20:03:43.809 INFO:teuthology.orchestra.run.smithi156.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-06T20:03:43.809 INFO:teuthology.orchestra.run.smithi156.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-06T20:03:43.809 INFO:teuthology.orchestra.run.smithi156.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-06T20:03:43.810 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:03:43.810 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-07-06T20:03:43.960 DEBUG:teuthology.parallel:result is None 2023-07-06T20:03:45.321 INFO:teuthology.orchestra.run.smithi098.stdout:Installed products updated. 2023-07-06T20:03:45.534 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T20:03:45.534 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2023-07-06T20:03:45.535 INFO:teuthology.orchestra.run.smithi098.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-06T20:03:45.535 INFO:teuthology.orchestra.run.smithi098.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-06T20:03:45.535 INFO:teuthology.orchestra.run.smithi098.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-06T20:03:45.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-06T20:03:45.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-06T20:03:45.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-06T20:03:45.535 INFO:teuthology.orchestra.run.smithi098.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-06T20:03:45.536 INFO:teuthology.orchestra.run.smithi098.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-06T20:03:45.536 INFO:teuthology.orchestra.run.smithi098.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-06T20:03:45.536 INFO:teuthology.orchestra.run.smithi098.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-06T20:03:45.536 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T20:03:45.536 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2023-07-06T20:03:45.687 DEBUG:teuthology.parallel:result is None 2023-07-06T20:03:45.687 INFO:teuthology.run_tasks:Running task install... 2023-07-06T20:03:45.701 DEBUG:teuthology.task.install:project ceph 2023-07-06T20:03:45.702 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}} 2023-07-06T20:03:45.702 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'} 2023-07-06T20:03:45.703 INFO:teuthology.task.install:Using flavor: default 2023-07-06T20:03:45.710 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-06T20:03:45.711 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-06T20:03:45.711 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'tag': None, 'wait_for_package': False} 2023-07-06T20:03:45.712 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:03:45.715 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'tag': None, 'wait_for_package': False} 2023-07-06T20:03:45.716 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:03:45.717 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'tag': None, 'wait_for_package': False} 2023-07-06T20:03:45.718 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:03:45.860 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/ 2023-07-06T20:03:45.860 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4742.g79365bb3 2023-07-06T20:03:45.867 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/ 2023-07-06T20:03:45.868 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4742.g79365bb3 2023-07-06T20:03:45.931 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/ 2023-07-06T20:03:45.961 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4742.g79365bb3 2023-07-06T20:03:46.000 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T20:03:46.001 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:03:46.001 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T20:03:46.003 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T20:03:46.004 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-07-06T20:03:46.004 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T20:03:46.031 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T20:03:46.032 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/79365bb3b803780276d69efb2cc4afe5b160de9e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T20:03:46.068 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T20:03:46.068 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-06T20:03:46.069 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T20:03:46.077 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T20:03:46.077 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/79365bb3b803780276d69efb2cc4afe5b160de9e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T20:03:46.096 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T20:03:46.097 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/79365bb3b803780276d69efb2cc4afe5b160de9e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T20:03:46.104 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T20:03:46.169 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T20:03:46.180 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T20:03:46.189 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T20:03:46.246 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T20:03:46.247 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2023-07-06T20:03:46.247 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2023-07-06T20:03:46.288 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T20:03:46.314 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2023-07-06T20:03:46.316 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2023-07-06T20:03:46.357 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2023-07-06T20:03:46.360 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2023-07-06T20:03:46.637 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-07-06T20:03:46.703 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-07-06T20:03:46.829 INFO:teuthology.orchestra.run.smithi069.stdout:Updating Subscription Management repositories. 2023-07-06T20:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout:44 files removed 2023-07-06T20:03:47.645 INFO:teuthology.orchestra.run.smithi156.stdout:44 files removed 2023-07-06T20:03:47.649 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T20:03:47.691 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T20:03:47.994 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-07-06T20:03:48.026 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-07-06T20:03:48.041 INFO:teuthology.orchestra.run.smithi069.stdout:44 files removed 2023-07-06T20:03:48.103 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T20:03:48.619 INFO:teuthology.orchestra.run.smithi069.stdout:Updating Subscription Management repositories. 2023-07-06T20:03:49.453 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 312 kB/s | 81 kB 00:00 2023-07-06T20:03:49.558 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 224 kB/s | 81 kB 00:00 2023-07-06T20:03:49.808 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 43 kB/s | 9.4 kB 00:00 2023-07-06T20:03:49.812 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 29 kB/s | 9.4 kB 00:00 2023-07-06T20:03:50.059 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 6.5 kB/s | 1.5 kB 00:00 2023-07-06T20:03:50.162 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2023-07-06T20:03:50.196 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 230 kB/s | 81 kB 00:00 2023-07-06T20:03:50.521 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 32 kB/s | 9.4 kB 00:00 2023-07-06T20:03:50.839 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 5.1 kB/s | 1.5 kB 00:00 2023-07-06T20:03:52.718 INFO:teuthology.orchestra.run.smithi098.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T20:03:52.838 INFO:teuthology.orchestra.run.smithi156.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T20:03:53.500 INFO:teuthology.orchestra.run.smithi069.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T20:03:54.455 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 16 MB 00:01 2023-07-06T20:03:55.839 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 5.2 MB/s | 16 MB 00:02 2023-07-06T20:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 432 kB/s | 24 kB 00:00 2023-07-06T20:03:59.240 INFO:teuthology.orchestra.run.smithi098.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 MB/s | 53 MB 00:01 2023-07-06T20:03:59.289 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 416 kB/s | 24 kB 00:00 2023-07-06T20:04:00.433 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-07-06T20:04:07.799 INFO:teuthology.orchestra.run.smithi098.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 MB/s | 47 MB 00:01 2023-07-06T20:04:08.513 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 16 MB 00:14 2023-07-06T20:04:08.544 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-07-06T20:04:12.715 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 356 kB/s | 24 kB 00:00 2023-07-06T20:04:14.050 INFO:teuthology.orchestra.run.smithi069.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 MB/s | 47 MB 00:01 2023-07-06T20:04:18.697 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T20:04:18.698 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T20:04:18.712 INFO:teuthology.orchestra.run.smithi098.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-06T20:04:18.714 INFO:teuthology.orchestra.run.smithi098.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-06T20:04:18.715 INFO:teuthology.orchestra.run.smithi098.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-06T20:04:18.716 INFO:teuthology.orchestra.run.smithi098.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T20:04:18.718 INFO:teuthology.orchestra.run.smithi098.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T20:04:18.721 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T20:04:18.723 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T20:04:18.854 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2023-07-06T20:04:18.865 INFO:teuthology.orchestra.run.smithi098.stdout:=========================================================================================================================== 2023-07-06T20:04:18.866 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2023-07-06T20:04:18.866 INFO:teuthology.orchestra.run.smithi098.stdout:=========================================================================================================================== 2023-07-06T20:04:18.866 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2023-07-06T20:04:18.866 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-06T20:04:18.866 INFO:teuthology.orchestra.run.smithi098.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-06T20:04:18.867 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.4 k 2023-07-06T20:04:18.867 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 5.4 M 2023-07-06T20:04:18.867 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 847 k 2023-07-06T20:04:18.867 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 165 k 2023-07-06T20:04:18.867 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 1.5 M 2023-07-06T20:04:18.868 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 129 k 2023-07-06T20:04:18.868 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 1.7 M 2023-07-06T20:04:18.868 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 7.4 M 2023-07-06T20:04:18.868 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 47 k 2023-07-06T20:04:18.869 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 13 M 2023-07-06T20:04:18.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 48 M 2023-07-06T20:04:18.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 258 k 2023-07-06T20:04:18.871 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 166 k 2023-07-06T20:04:18.871 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 258 k 2023-07-06T20:04:18.871 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 18 k 2023-07-06T20:04:18.871 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-06T20:04:18.871 INFO:teuthology.orchestra.run.smithi098.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-06T20:04:18.871 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-06T20:04:18.871 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 31 k 2023-07-06T20:04:18.872 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 770 k 2023-07-06T20:04:18.872 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 139 k 2023-07-06T20:04:18.872 INFO:teuthology.orchestra.run.smithi098.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-06T20:04:18.872 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-06T20:04:18.873 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-06T20:04:18.873 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-06T20:04:18.873 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-06T20:04:18.873 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-06T20:04:18.873 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 197 k 2023-07-06T20:04:18.873 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 370 k 2023-07-06T20:04:18.873 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 364 k 2023-07-06T20:04:18.874 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 107 k 2023-07-06T20:04:18.874 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 94 k 2023-07-06T20:04:18.874 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.5 M 2023-07-06T20:04:18.874 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 179 k 2023-07-06T20:04:18.874 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-06T20:04:18.874 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2023-07-06T20:04:18.874 INFO:teuthology.orchestra.run.smithi098.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-06T20:04:18.874 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.7 M 2023-07-06T20:04:18.875 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.6 M 2023-07-06T20:04:18.875 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2023-07-06T20:04:18.875 INFO:teuthology.orchestra.run.smithi098.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-06T20:04:18.875 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 22 M 2023-07-06T20:04:18.875 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 23 k 2023-07-06T20:04:18.875 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 2.3 M 2023-07-06T20:04:18.875 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 240 k 2023-07-06T20:04:18.876 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 4.6 M 2023-07-06T20:04:18.876 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 18 M 2023-07-06T20:04:18.876 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 15 k 2023-07-06T20:04:18.876 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 24 k 2023-07-06T20:04:18.876 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-06T20:04:18.876 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T20:04:18.876 INFO:teuthology.orchestra.run.smithi098.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-06T20:04:18.877 INFO:teuthology.orchestra.run.smithi098.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-06T20:04:18.877 INFO:teuthology.orchestra.run.smithi098.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-06T20:04:18.877 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 137 k 2023-07-06T20:04:18.877 INFO:teuthology.orchestra.run.smithi098.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-06T20:04:18.877 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-06T20:04:18.877 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T20:04:18.877 INFO:teuthology.orchestra.run.smithi098.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-06T20:04:18.877 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-06T20:04:18.878 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-06T20:04:18.878 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 475 k 2023-07-06T20:04:18.878 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-06T20:04:18.878 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.7 M 2023-07-06T20:04:18.878 INFO:teuthology.orchestra.run.smithi098.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-06T20:04:18.878 INFO:teuthology.orchestra.run.smithi098.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-06T20:04:18.878 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T20:04:18.879 INFO:teuthology.orchestra.run.smithi098.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-06T20:04:18.879 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-06T20:04:18.879 INFO:teuthology.orchestra.run.smithi098.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-06T20:04:18.879 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-06T20:04:18.879 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-06T20:04:18.879 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-06T20:04:18.879 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-06T20:04:18.879 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-06T20:04:18.880 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T20:04:18.880 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-06T20:04:18.880 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T20:04:18.880 INFO:teuthology.orchestra.run.smithi098.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T20:04:18.880 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T20:04:18.880 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 45 k 2023-07-06T20:04:18.880 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 119 k 2023-07-06T20:04:18.881 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T20:04:18.881 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-06T20:04:18.881 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T20:04:18.881 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T20:04:18.881 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-06T20:04:18.881 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T20:04:18.881 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T20:04:18.881 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T20:04:18.882 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-06T20:04:18.882 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-06T20:04:18.883 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-06T20:04:18.883 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T20:04:18.883 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T20:04:18.883 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T20:04:18.883 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-06T20:04:18.883 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-06T20:04:18.884 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T20:04:18.884 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T20:04:18.884 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-06T20:04:18.884 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-06T20:04:18.884 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-06T20:04:18.884 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T20:04:18.884 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T20:04:18.885 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-06T20:04:18.885 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-06T20:04:18.885 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-06T20:04:18.885 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-06T20:04:18.885 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-06T20:04:18.885 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-06T20:04:18.885 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T20:04:18.885 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T20:04:18.886 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T20:04:18.886 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T20:04:18.886 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-06T20:04:18.886 INFO:teuthology.orchestra.run.smithi098.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T20:04:18.886 INFO:teuthology.orchestra.run.smithi098.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T20:04:18.886 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T20:04:18.886 INFO:teuthology.orchestra.run.smithi098.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T20:04:18.886 INFO:teuthology.orchestra.run.smithi098.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T20:04:18.887 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T20:04:18.887 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T20:04:18.887 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T20:04:18.887 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-06T20:04:18.887 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T20:04:18.887 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-06T20:04:18.887 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T20:04:18.887 INFO:teuthology.orchestra.run.smithi098.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-06T20:04:18.888 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-06T20:04:18.888 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-06T20:04:18.888 INFO:teuthology.orchestra.run.smithi098.stdout:Enabling module streams: 2023-07-06T20:04:18.888 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql 10 2023-07-06T20:04:18.888 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T20:04:18.888 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2023-07-06T20:04:18.888 INFO:teuthology.orchestra.run.smithi098.stdout:=========================================================================================================================== 2023-07-06T20:04:18.889 INFO:teuthology.orchestra.run.smithi098.stdout:Install 128 Packages 2023-07-06T20:04:18.889 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 3 Packages 2023-07-06T20:04:18.889 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T20:04:18.890 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 205 M 2023-07-06T20:04:18.890 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2023-07-06T20:04:19.324 INFO:teuthology.orchestra.run.smithi098.stdout:(1/131): ceph-18.0.0-4742.g79365bb3.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2023-07-06T20:04:19.548 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T20:04:19.549 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T20:04:19.563 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-06T20:04:19.564 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-06T20:04:19.565 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-06T20:04:19.566 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T20:04:19.568 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T20:04:19.571 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T20:04:19.572 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T20:04:19.708 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-07-06T20:04:19.720 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-07-06T20:04:19.721 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-07-06T20:04:19.721 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-07-06T20:04:19.721 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2023-07-06T20:04:19.722 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-06T20:04:19.722 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-06T20:04:19.722 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.4 k 2023-07-06T20:04:19.722 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 5.4 M 2023-07-06T20:04:19.723 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 847 k 2023-07-06T20:04:19.723 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 165 k 2023-07-06T20:04:19.723 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 1.5 M 2023-07-06T20:04:19.724 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 129 k 2023-07-06T20:04:19.724 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 1.7 M 2023-07-06T20:04:19.724 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 7.4 M 2023-07-06T20:04:19.724 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 47 k 2023-07-06T20:04:19.724 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 13 M 2023-07-06T20:04:19.725 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 48 M 2023-07-06T20:04:19.725 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 258 k 2023-07-06T20:04:19.725 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 166 k 2023-07-06T20:04:19.725 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 258 k 2023-07-06T20:04:19.725 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 18 k 2023-07-06T20:04:19.725 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-06T20:04:19.725 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-06T20:04:19.726 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-06T20:04:19.726 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 31 k 2023-07-06T20:04:19.726 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 770 k 2023-07-06T20:04:19.726 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 139 k 2023-07-06T20:04:19.726 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-06T20:04:19.727 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-06T20:04:19.727 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-06T20:04:19.728 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-06T20:04:19.728 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-06T20:04:19.728 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-06T20:04:19.728 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 197 k 2023-07-06T20:04:19.728 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 370 k 2023-07-06T20:04:19.729 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 364 k 2023-07-06T20:04:19.729 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 107 k 2023-07-06T20:04:19.729 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 94 k 2023-07-06T20:04:19.729 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.5 M 2023-07-06T20:04:19.729 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 179 k 2023-07-06T20:04:19.729 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-06T20:04:19.729 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2023-07-06T20:04:19.729 INFO:teuthology.orchestra.run.smithi156.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-06T20:04:19.730 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.7 M 2023-07-06T20:04:19.730 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.6 M 2023-07-06T20:04:19.730 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-07-06T20:04:19.730 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-06T20:04:19.730 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 22 M 2023-07-06T20:04:19.730 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 23 k 2023-07-06T20:04:19.730 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 2.3 M 2023-07-06T20:04:19.730 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 240 k 2023-07-06T20:04:19.731 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 4.6 M 2023-07-06T20:04:19.731 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 18 M 2023-07-06T20:04:19.731 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 15 k 2023-07-06T20:04:19.731 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 24 k 2023-07-06T20:04:19.731 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-06T20:04:19.731 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T20:04:19.731 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-06T20:04:19.732 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-06T20:04:19.732 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-06T20:04:19.732 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 137 k 2023-07-06T20:04:19.732 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-06T20:04:19.732 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-06T20:04:19.732 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T20:04:19.732 INFO:teuthology.orchestra.run.smithi156.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-06T20:04:19.732 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-06T20:04:19.733 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-06T20:04:19.733 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 475 k 2023-07-06T20:04:19.733 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-06T20:04:19.733 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.7 M 2023-07-06T20:04:19.733 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-06T20:04:19.733 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-06T20:04:19.733 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T20:04:19.734 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-06T20:04:19.734 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-06T20:04:19.734 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-06T20:04:19.734 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-06T20:04:19.734 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-06T20:04:19.734 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-06T20:04:19.735 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-06T20:04:19.735 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-06T20:04:19.735 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T20:04:19.735 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-06T20:04:19.735 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T20:04:19.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T20:04:19.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T20:04:19.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 45 k 2023-07-06T20:04:19.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 119 k 2023-07-06T20:04:19.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T20:04:19.736 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-06T20:04:19.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T20:04:19.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T20:04:19.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-06T20:04:19.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T20:04:19.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T20:04:19.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T20:04:19.737 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-06T20:04:19.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-06T20:04:19.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-06T20:04:19.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T20:04:19.738 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T20:04:19.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T20:04:19.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-06T20:04:19.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-06T20:04:19.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T20:04:19.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T20:04:19.739 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-06T20:04:19.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-06T20:04:19.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-06T20:04:19.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T20:04:19.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T20:04:19.740 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-06T20:04:19.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-06T20:04:19.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-06T20:04:19.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-06T20:04:19.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-06T20:04:19.741 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-06T20:04:19.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T20:04:19.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T20:04:19.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T20:04:19.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T20:04:19.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-06T20:04:19.742 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T20:04:19.743 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T20:04:19.743 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T20:04:19.743 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T20:04:19.743 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T20:04:19.743 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T20:04:19.743 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T20:04:19.743 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T20:04:19.744 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-06T20:04:19.744 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T20:04:19.744 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-06T20:04:19.744 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T20:04:19.744 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-06T20:04:19.744 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-06T20:04:19.744 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-06T20:04:19.745 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-07-06T20:04:19.745 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql 10 2023-07-06T20:04:19.745 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:04:19.745 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-07-06T20:04:19.745 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-07-06T20:04:19.746 INFO:teuthology.orchestra.run.smithi156.stdout:Install 128 Packages 2023-07-06T20:04:19.746 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 3 Packages 2023-07-06T20:04:19.746 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:04:19.746 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 205 M 2023-07-06T20:04:19.747 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-07-06T20:04:19.982 INFO:teuthology.orchestra.run.smithi098.stdout:(2/131): ceph-fuse-18.0.0-4742.g79365bb3.el8.x8 1.3 MB/s | 847 kB 00:00 2023-07-06T20:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 1.3 MB/s | 165 kB 00:00 2023-07-06T20:04:20.201 INFO:teuthology.orchestra.run.smithi156.stdout:(1/131): ceph-18.0.0-4742.g79365bb3.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-07-06T20:04:20.868 INFO:teuthology.orchestra.run.smithi156.stdout:(2/131): ceph-fuse-18.0.0-4742.g79365bb3.el8.x8 1.2 MB/s | 847 kB 00:00 2023-07-06T20:04:21.018 INFO:teuthology.orchestra.run.smithi156.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 1.1 MB/s | 165 kB 00:00 2023-07-06T20:04:21.342 INFO:teuthology.orchestra.run.smithi098.stdout:(4/131): ceph-mds-18.0.0-4742.g79365bb3.el8.x86 1.8 MB/s | 2.3 MB 00:01 2023-07-06T20:04:22.000 INFO:teuthology.orchestra.run.smithi098.stdout:(5/131): ceph-mgr-18.0.0-4742.g79365bb3.el8.x86 2.4 MB/s | 1.5 MB 00:00 2023-07-06T20:04:22.052 INFO:teuthology.orchestra.run.smithi156.stdout:(4/131): ceph-mds-18.0.0-4742.g79365bb3.el8.x86 2.2 MB/s | 2.3 MB 00:01 2023-07-06T20:04:22.527 INFO:teuthology.orchestra.run.smithi156.stdout:(5/131): ceph-base-18.0.0-4742.g79365bb3.el8.x8 2.0 MB/s | 5.4 MB 00:02 2023-07-06T20:04:22.811 INFO:teuthology.orchestra.run.smithi156.stdout:(6/131): ceph-mgr-18.0.0-4742.g79365bb3.el8.x86 2.0 MB/s | 1.5 MB 00:00 2023-07-06T20:04:22.825 INFO:teuthology.orchestra.run.smithi069.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 44 MB/s | 53 MB 00:01 2023-07-06T20:04:23.825 INFO:teuthology.orchestra.run.smithi098.stdout:(6/131): ceph-mon-18.0.0-4742.g79365bb3.el8.x86 2.5 MB/s | 4.6 MB 00:01 2023-07-06T20:04:23.934 INFO:teuthology.orchestra.run.smithi098.stdout:(7/131): ceph-base-18.0.0-4742.g79365bb3.el8.x8 1.1 MB/s | 5.4 MB 00:04 2023-07-06T20:04:24.144 INFO:teuthology.orchestra.run.smithi156.stdout:(7/131): ceph-mon-18.0.0-4742.g79365bb3.el8.x86 2.8 MB/s | 4.6 MB 00:01 2023-07-06T20:04:26.126 INFO:teuthology.orchestra.run.smithi098.stdout:(8/131): ceph-common-18.0.0-4742.g79365bb3.el8. 3.1 MB/s | 22 MB 00:07 2023-07-06T20:04:26.201 INFO:teuthology.orchestra.run.smithi098.stdout:(9/131): ceph-selinux-18.0.0-4742.g79365bb3.el8 326 kB/s | 24 kB 00:00 2023-07-06T20:04:27.703 INFO:teuthology.orchestra.run.smithi156.stdout:(8/131): ceph-radosgw-18.0.0-4742.g79365bb3.el8 3.8 MB/s | 13 MB 00:03 2023-07-06T20:04:27.761 INFO:teuthology.orchestra.run.smithi156.stdout:(9/131): ceph-selinux-18.0.0-4742.g79365bb3.el8 420 kB/s | 24 kB 00:00 2023-07-06T20:04:28.053 INFO:teuthology.orchestra.run.smithi156.stdout:(10/131): ceph-osd-18.0.0-4742.g79365bb3.el8.x8 3.4 MB/s | 18 MB 00:05 2023-07-06T20:04:28.178 INFO:teuthology.orchestra.run.smithi156.stdout:(11/131): cephfs-mirror-18.0.0-4742.g79365bb3.e 2.0 MB/s | 258 kB 00:00 2023-07-06T20:04:28.237 INFO:teuthology.orchestra.run.smithi156.stdout:(12/131): libcephfs-devel-18.0.0-4742.g79365bb3 534 kB/s | 31 kB 00:00 2023-07-06T20:04:28.437 INFO:teuthology.orchestra.run.smithi156.stdout:(13/131): libcephfs2-18.0.0-4742.g79365bb3.el8. 3.8 MB/s | 770 kB 00:00 2023-07-06T20:04:28.504 INFO:teuthology.orchestra.run.smithi156.stdout:(14/131): libcephsqlite-18.0.0-4742.g79365bb3.e 2.0 MB/s | 137 kB 00:00 2023-07-06T20:04:28.605 INFO:teuthology.orchestra.run.smithi156.stdout:(15/131): librados-devel-18.0.0-4742.g79365bb3. 1.4 MB/s | 139 kB 00:00 2023-07-06T20:04:28.723 INFO:teuthology.orchestra.run.smithi156.stdout:(16/131): libradosstriper1-18.0.0-4742.g79365bb 4.0 MB/s | 475 kB 00:00 2023-07-06T20:04:28.893 INFO:teuthology.orchestra.run.smithi098.stdout:(10/131): ceph-osd-18.0.0-4742.g79365bb3.el8.x8 3.5 MB/s | 18 MB 00:05 2023-07-06T20:04:29.035 INFO:teuthology.orchestra.run.smithi098.stdout:(11/131): cephfs-mirror-18.0.0-4742.g79365bb3.e 1.8 MB/s | 258 kB 00:00 2023-07-06T20:04:29.135 INFO:teuthology.orchestra.run.smithi098.stdout:(12/131): libcephfs-devel-18.0.0-4742.g79365bb3 311 kB/s | 31 kB 00:00 2023-07-06T20:04:29.377 INFO:teuthology.orchestra.run.smithi098.stdout:(13/131): libcephfs2-18.0.0-4742.g79365bb3.el8. 3.1 MB/s | 770 kB 00:00 2023-07-06T20:04:29.477 INFO:teuthology.orchestra.run.smithi098.stdout:(14/131): libcephsqlite-18.0.0-4742.g79365bb3.e 1.3 MB/s | 137 kB 00:00 2023-07-06T20:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout:(15/131): librados-devel-18.0.0-4742.g79365bb3. 1.5 MB/s | 139 kB 00:00 2023-07-06T20:04:29.728 INFO:teuthology.orchestra.run.smithi098.stdout:(16/131): libradosstriper1-18.0.0-4742.g79365bb 2.9 MB/s | 475 kB 00:00 2023-07-06T20:04:30.105 INFO:teuthology.orchestra.run.smithi156.stdout:(17/131): librgw2-18.0.0-4742.g79365bb3.el8.x86 4.9 MB/s | 6.7 MB 00:01 2023-07-06T20:04:30.180 INFO:teuthology.orchestra.run.smithi156.stdout:(18/131): python3-ceph-argparse-18.0.0-4742.g79 597 kB/s | 45 kB 00:00 2023-07-06T20:04:30.264 INFO:teuthology.orchestra.run.smithi156.stdout:(19/131): python3-ceph-common-18.0.0-4742.g7936 1.4 MB/s | 119 kB 00:00 2023-07-06T20:04:30.557 INFO:teuthology.orchestra.run.smithi098.stdout:(17/131): ceph-radosgw-18.0.0-4742.g79365bb3.el 2.0 MB/s | 13 MB 00:06 2023-07-06T20:04:30.561 INFO:teuthology.orchestra.run.smithi156.stdout:(20/131): python3-cephfs-18.0.0-4742.g79365bb3. 665 kB/s | 197 kB 00:00 2023-07-06T20:04:30.654 INFO:teuthology.orchestra.run.smithi098.stdout:(18/131): python3-ceph-argparse-18.0.0-4742.g79 466 kB/s | 45 kB 00:00 2023-07-06T20:04:30.746 INFO:teuthology.orchestra.run.smithi098.stdout:(19/131): python3-ceph-common-18.0.0-4742.g7936 1.3 MB/s | 119 kB 00:00 2023-07-06T20:04:30.907 INFO:teuthology.orchestra.run.smithi156.stdout:(21/131): ceph-common-18.0.0-4742.g79365bb3.el8 2.0 MB/s | 22 MB 00:11 2023-07-06T20:04:30.930 INFO:teuthology.orchestra.run.smithi098.stdout:(20/131): python3-cephfs-18.0.0-4742.g79365bb3. 1.1 MB/s | 197 kB 00:00 2023-07-06T20:04:30.932 INFO:teuthology.orchestra.run.smithi156.stdout:(22/131): python3-rados-18.0.0-4742.g79365bb3.e 1.0 MB/s | 370 kB 00:00 2023-07-06T20:04:31.032 INFO:teuthology.orchestra.run.smithi156.stdout:(23/131): python3-rgw-18.0.0-4742.g79365bb3.el8 1.0 MB/s | 107 kB 00:00 2023-07-06T20:04:31.130 INFO:teuthology.orchestra.run.smithi098.stdout:(21/131): python3-rados-18.0.0-4742.g79365bb3.e 1.8 MB/s | 370 kB 00:00 2023-07-06T20:04:31.132 INFO:teuthology.orchestra.run.smithi156.stdout:(24/131): python3-rbd-18.0.0-4742.g79365bb3.el8 1.6 MB/s | 364 kB 00:00 2023-07-06T20:04:31.158 INFO:teuthology.orchestra.run.smithi156.stdout:(25/131): rbd-fuse-18.0.0-4742.g79365bb3.el8.x8 754 kB/s | 94 kB 00:00 2023-07-06T20:04:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:(26/131): rbd-nbd-18.0.0-4742.g79365bb3.el8.x86 1.9 MB/s | 179 kB 00:00 2023-07-06T20:04:31.322 INFO:teuthology.orchestra.run.smithi098.stdout:(22/131): python3-rbd-18.0.0-4742.g79365bb3.el8 1.9 MB/s | 364 kB 00:00 2023-07-06T20:04:31.334 INFO:teuthology.orchestra.run.smithi156.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4742.g 276 kB/s | 23 kB 00:00 2023-07-06T20:04:31.425 INFO:teuthology.orchestra.run.smithi156.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4742.g79365bb 1.4 MB/s | 129 kB 00:00 2023-07-06T20:04:31.430 INFO:teuthology.orchestra.run.smithi098.stdout:(23/131): python3-rgw-18.0.0-4742.g79365bb3.el8 986 kB/s | 107 kB 00:00 2023-07-06T20:04:31.531 INFO:teuthology.orchestra.run.smithi098.stdout:(24/131): librgw2-18.0.0-4742.g79365bb3.el8.x86 3.7 MB/s | 6.7 MB 00:01 2023-07-06T20:04:31.589 INFO:teuthology.orchestra.run.smithi098.stdout:(25/131): rbd-fuse-18.0.0-4742.g79365bb3.el8.x8 595 kB/s | 94 kB 00:00 2023-07-06T20:04:31.698 INFO:teuthology.orchestra.run.smithi098.stdout:(26/131): rbd-nbd-18.0.0-4742.g79365bb3.el8.x86 1.6 MB/s | 179 kB 00:00 2023-07-06T20:04:31.806 INFO:teuthology.orchestra.run.smithi098.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4742.g 213 kB/s | 23 kB 00:00 2023-07-06T20:04:31.933 INFO:teuthology.orchestra.run.smithi098.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4742.g79365bb 1.0 MB/s | 129 kB 00:00 2023-07-06T20:04:31.967 INFO:teuthology.orchestra.run.smithi156.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4742.g79365 3.2 MB/s | 1.7 MB 00:00 2023-07-06T20:04:32.168 INFO:teuthology.orchestra.run.smithi156.stdout:(30/131): rbd-mirror-18.0.0-4742.g79365bb3.el8. 3.3 MB/s | 3.5 MB 00:01 2023-07-06T20:04:32.260 INFO:teuthology.orchestra.run.smithi156.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4742.g79 2.6 MB/s | 240 kB 00:00 2023-07-06T20:04:32.343 INFO:teuthology.orchestra.run.smithi156.stdout:(32/131): ceph-mgr-rook-18.0.0-4742.g79365bb3.e 569 kB/s | 47 kB 00:00 2023-07-06T20:04:32.468 INFO:teuthology.orchestra.run.smithi156.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4742.g7 116 kB/s | 15 kB 00:00 2023-07-06T20:04:32.532 INFO:teuthology.orchestra.run.smithi098.stdout:(29/131): rbd-mirror-18.0.0-4742.g79365bb3.el8. 3.4 MB/s | 3.5 MB 00:01 2023-07-06T20:04:32.595 INFO:teuthology.orchestra.run.smithi156.stdout:(34/131): ceph-volume-18.0.0-4742.g79365bb3.el8 2.0 MB/s | 258 kB 00:00 2023-07-06T20:04:32.641 INFO:teuthology.orchestra.run.smithi098.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4742.g79365 2.4 MB/s | 1.7 MB 00:00 2023-07-06T20:04:32.678 INFO:teuthology.orchestra.run.smithi156.stdout:(35/131): cephadm-18.0.0-4742.g79365bb3.el8.noa 2.0 MB/s | 166 kB 00:00 2023-07-06T20:04:32.744 INFO:teuthology.orchestra.run.smithi156.stdout:(36/131): cephfs-top-18.0.0-4742.g79365bb3.el8. 269 kB/s | 18 kB 00:00 2023-07-06T20:04:32.799 INFO:teuthology.orchestra.run.smithi098.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4742.g79 1.5 MB/s | 240 kB 00:00 2023-07-06T20:04:32.920 INFO:teuthology.orchestra.run.smithi098.stdout:(32/131): ceph-mgr-rook-18.0.0-4742.g79365bb3.e 394 kB/s | 47 kB 00:00 2023-07-06T20:04:33.042 INFO:teuthology.orchestra.run.smithi098.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4742.g7 120 kB/s | 15 kB 00:00 2023-07-06T20:04:33.292 INFO:teuthology.orchestra.run.smithi098.stdout:(34/131): ceph-volume-18.0.0-4742.g79365bb3.el8 1.0 MB/s | 258 kB 00:00 2023-07-06T20:04:33.587 INFO:teuthology.orchestra.run.smithi156.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 4.6 MB/s | 7.4 MB 00:01 2023-07-06T20:04:33.653 INFO:teuthology.orchestra.run.smithi156.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 337 kB/s | 306 kB 00:00 2023-07-06T20:04:33.667 INFO:teuthology.orchestra.run.smithi098.stdout:(35/131): cephadm-18.0.0-4742.g79365bb3.el8.noa 444 kB/s | 166 kB 00:00 2023-07-06T20:04:33.776 INFO:teuthology.orchestra.run.smithi098.stdout:(36/131): cephfs-top-18.0.0-4742.g79365bb3.el8. 166 kB/s | 18 kB 00:00 2023-07-06T20:04:33.812 INFO:teuthology.orchestra.run.smithi156.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 473 kB/s | 75 kB 00:00 2023-07-06T20:04:34.079 INFO:teuthology.orchestra.run.smithi156.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.8 MB/s | 476 kB 00:00 2023-07-06T20:04:34.105 INFO:teuthology.orchestra.run.smithi156.stdout:(41/131): liboath-2.6.2-3.el8.x86_64.rpm 114 kB/s | 59 kB 00:00 2023-07-06T20:04:34.188 INFO:teuthology.orchestra.run.smithi156.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 401 kB/s | 44 kB 00:00 2023-07-06T20:04:34.288 INFO:teuthology.orchestra.run.smithi156.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2023-07-06T20:04:34.334 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T20:04:34.335 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T20:04:34.350 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-06T20:04:34.351 INFO:teuthology.orchestra.run.smithi069.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-06T20:04:34.352 INFO:teuthology.orchestra.run.smithi069.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-06T20:04:34.353 INFO:teuthology.orchestra.run.smithi069.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T20:04:34.355 INFO:teuthology.orchestra.run.smithi156.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 740 kB/s | 185 kB 00:00 2023-07-06T20:04:34.356 INFO:teuthology.orchestra.run.smithi069.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T20:04:34.359 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T20:04:34.360 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T20:04:34.389 INFO:teuthology.orchestra.run.smithi156.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2023-07-06T20:04:34.498 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2023-07-06T20:04:34.511 INFO:teuthology.orchestra.run.smithi069.stdout:=========================================================================================================================== 2023-07-06T20:04:34.511 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2023-07-06T20:04:34.511 INFO:teuthology.orchestra.run.smithi069.stdout:=========================================================================================================================== 2023-07-06T20:04:34.511 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2023-07-06T20:04:34.511 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-06T20:04:34.511 INFO:teuthology.orchestra.run.smithi069.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-06T20:04:34.512 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.4 k 2023-07-06T20:04:34.512 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 5.4 M 2023-07-06T20:04:34.512 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 847 k 2023-07-06T20:04:34.512 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 165 k 2023-07-06T20:04:34.512 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 1.5 M 2023-07-06T20:04:34.513 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 129 k 2023-07-06T20:04:34.513 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 1.7 M 2023-07-06T20:04:34.514 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 7.4 M 2023-07-06T20:04:34.514 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 47 k 2023-07-06T20:04:34.514 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 13 M 2023-07-06T20:04:34.514 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 48 M 2023-07-06T20:04:34.514 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 258 k 2023-07-06T20:04:34.514 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 166 k 2023-07-06T20:04:34.514 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 258 k 2023-07-06T20:04:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 18 k 2023-07-06T20:04:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-06T20:04:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-06T20:04:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-06T20:04:34.515 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 31 k 2023-07-06T20:04:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 770 k 2023-07-06T20:04:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 139 k 2023-07-06T20:04:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-06T20:04:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-06T20:04:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-06T20:04:34.516 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-06T20:04:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-06T20:04:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-06T20:04:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 197 k 2023-07-06T20:04:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 370 k 2023-07-06T20:04:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 364 k 2023-07-06T20:04:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 107 k 2023-07-06T20:04:34.517 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 94 k 2023-07-06T20:04:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.5 M 2023-07-06T20:04:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 179 k 2023-07-06T20:04:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-06T20:04:34.518 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2023-07-06T20:04:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-06T20:04:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.7 M 2023-07-06T20:04:34.518 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.6 M 2023-07-06T20:04:34.518 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2023-07-06T20:04:34.519 INFO:teuthology.orchestra.run.smithi069.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-06T20:04:34.519 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 22 M 2023-07-06T20:04:34.519 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 23 k 2023-07-06T20:04:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 2.3 M 2023-07-06T20:04:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 240 k 2023-07-06T20:04:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 4.6 M 2023-07-06T20:04:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 18 M 2023-07-06T20:04:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 15 k 2023-07-06T20:04:34.520 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 24 k 2023-07-06T20:04:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-06T20:04:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T20:04:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-06T20:04:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-06T20:04:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-06T20:04:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 137 k 2023-07-06T20:04:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-06T20:04:34.521 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-06T20:04:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T20:04:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-06T20:04:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-06T20:04:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-06T20:04:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 475 k 2023-07-06T20:04:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-06T20:04:34.522 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.7 M 2023-07-06T20:04:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-06T20:04:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-06T20:04:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T20:04:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-06T20:04:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-06T20:04:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-06T20:04:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-06T20:04:34.523 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-06T20:04:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-06T20:04:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-06T20:04:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-06T20:04:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T20:04:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-06T20:04:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T20:04:34.524 INFO:teuthology.orchestra.run.smithi069.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T20:04:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T20:04:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 45 k 2023-07-06T20:04:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 119 k 2023-07-06T20:04:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T20:04:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-06T20:04:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T20:04:34.525 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T20:04:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-06T20:04:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T20:04:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T20:04:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T20:04:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-06T20:04:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-06T20:04:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-06T20:04:34.526 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T20:04:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T20:04:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T20:04:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-06T20:04:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-06T20:04:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T20:04:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T20:04:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-06T20:04:34.527 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-06T20:04:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-06T20:04:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T20:04:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T20:04:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-06T20:04:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-06T20:04:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-06T20:04:34.528 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-06T20:04:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-06T20:04:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-06T20:04:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T20:04:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T20:04:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T20:04:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T20:04:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-06T20:04:34.529 INFO:teuthology.orchestra.run.smithi069.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T20:04:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T20:04:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T20:04:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T20:04:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T20:04:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T20:04:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T20:04:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T20:04:34.530 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-06T20:04:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T20:04:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-06T20:04:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T20:04:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-06T20:04:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-06T20:04:34.531 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-06T20:04:34.531 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling module streams: 2023-07-06T20:04:34.532 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql 10 2023-07-06T20:04:34.532 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:04:34.532 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2023-07-06T20:04:34.532 INFO:teuthology.orchestra.run.smithi069.stdout:=========================================================================================================================== 2023-07-06T20:04:34.532 INFO:teuthology.orchestra.run.smithi069.stdout:Install 128 Packages 2023-07-06T20:04:34.532 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 3 Packages 2023-07-06T20:04:34.532 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:04:34.533 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 205 M 2023-07-06T20:04:34.533 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2023-07-06T20:04:34.535 INFO:teuthology.orchestra.run.smithi098.stdout:(37/131): ceph-test-18.0.0-4742.g79365bb3.el8.x 5.7 MB/s | 48 MB 00:08 2023-07-06T20:04:34.538 INFO:teuthology.orchestra.run.smithi156.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 946 kB/s | 173 kB 00:00 2023-07-06T20:04:34.573 INFO:teuthology.orchestra.run.smithi156.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.0 MB/s | 384 kB 00:00 2023-07-06T20:04:34.627 INFO:teuthology.orchestra.run.smithi098.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 3.5 MB/s | 7.4 MB 00:02 2023-07-06T20:04:34.657 INFO:teuthology.orchestra.run.smithi156.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 688 kB/s | 81 kB 00:00 2023-07-06T20:04:34.682 INFO:teuthology.orchestra.run.smithi156.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-07-06T20:04:34.683 INFO:teuthology.orchestra.run.smithi069.stdout:(1/131): ceph-18.0.0-4742.g79365bb3.el8.x86_64. 54 kB/s | 6.4 kB 00:00 2023-07-06T20:04:34.757 INFO:teuthology.orchestra.run.smithi156.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 176 kB/s | 18 kB 00:00 2023-07-06T20:04:34.866 INFO:teuthology.orchestra.run.smithi156.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 449 kB/s | 49 kB 00:00 2023-07-06T20:04:34.926 INFO:teuthology.orchestra.run.smithi069.stdout:(2/131): ceph-fuse-18.0.0-4742.g79365bb3.el8.x8 3.4 MB/s | 847 kB 00:00 2023-07-06T20:04:34.983 INFO:teuthology.orchestra.run.smithi156.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 504 kB/s | 59 kB 00:00 2023-07-06T20:04:34.993 INFO:teuthology.orchestra.run.smithi069.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 2.4 MB/s | 165 kB 00:00 2023-07-06T20:04:35.301 INFO:teuthology.orchestra.run.smithi069.stdout:(4/131): ceph-mds-18.0.0-4742.g79365bb3.el8.x86 7.4 MB/s | 2.3 MB 00:00 2023-07-06T20:04:35.408 INFO:teuthology.orchestra.run.smithi156.stdout:(53/131): ceph-test-18.0.0-4742.g79365bb3.el8.x 6.2 MB/s | 48 MB 00:07 2023-07-06T20:04:35.409 INFO:teuthology.orchestra.run.smithi069.stdout:(5/131): ceph-base-18.0.0-4742.g79365bb3.el8.x8 6.4 MB/s | 5.4 MB 00:00 2023-07-06T20:04:35.442 INFO:teuthology.orchestra.run.smithi156.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 1.5 MB/s | 1.2 MB 00:00 2023-07-06T20:04:35.467 INFO:teuthology.orchestra.run.smithi156.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 124 kB/s | 60 kB 00:00 2023-07-06T20:04:35.518 INFO:teuthology.orchestra.run.smithi069.stdout:(6/131): ceph-mgr-18.0.0-4742.g79365bb3.el8.x86 7.1 MB/s | 1.5 MB 00:00 2023-07-06T20:04:35.576 INFO:teuthology.orchestra.run.smithi156.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-07-06T20:04:35.601 INFO:teuthology.orchestra.run.smithi156.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 252 kB/s | 34 kB 00:00 2023-07-06T20:04:35.701 INFO:teuthology.orchestra.run.smithi156.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-07-06T20:04:35.718 INFO:teuthology.orchestra.run.smithi156.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 525 kB/s | 61 kB 00:00 2023-07-06T20:04:35.801 INFO:teuthology.orchestra.run.smithi156.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2023-07-06T20:04:35.818 INFO:teuthology.orchestra.run.smithi156.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 239 kB/s | 24 kB 00:00 2023-07-06T20:04:35.844 INFO:teuthology.orchestra.run.smithi069.stdout:(7/131): ceph-mon-18.0.0-4742.g79365bb3.el8.x86 11 MB/s | 4.6 MB 00:00 2023-07-06T20:04:35.919 INFO:teuthology.orchestra.run.smithi156.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2023-07-06T20:04:36.006 INFO:teuthology.orchestra.run.smithi156.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 147 kB/s | 27 kB 00:00 2023-07-06T20:04:36.203 INFO:teuthology.orchestra.run.smithi156.stdout:(64/131): python3-pecan-1.3.2-9.el8.noarch.rpm 356 kB/s | 283 kB 00:00 2023-07-06T20:04:36.345 INFO:teuthology.orchestra.run.smithi156.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 600 kB/s | 255 kB 00:00 2023-07-06T20:04:36.370 INFO:teuthology.orchestra.run.smithi156.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 690 kB/s | 251 kB 00:00 2023-07-06T20:04:36.395 INFO:teuthology.orchestra.run.smithi156.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 318 kB/s | 61 kB 00:00 2023-07-06T20:04:36.454 INFO:teuthology.orchestra.run.smithi156.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 782 kB/s | 85 kB 00:00 2023-07-06T20:04:36.479 INFO:teuthology.orchestra.run.smithi156.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 210 kB/s | 23 kB 00:00 2023-07-06T20:04:36.555 INFO:teuthology.orchestra.run.smithi156.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 694 kB/s | 70 kB 00:00 2023-07-06T20:04:36.655 INFO:teuthology.orchestra.run.smithi156.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-07-06T20:04:36.755 INFO:teuthology.orchestra.run.smithi156.stdout:(72/131): thrift-0.13.0-2.el8.x86_64.rpm 4.8 MB/s | 1.7 MB 00:00 2023-07-06T20:04:36.806 INFO:teuthology.orchestra.run.smithi156.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 361 kB/s | 54 kB 00:00 2023-07-06T20:04:37.056 INFO:teuthology.orchestra.run.smithi156.stdout:(74/131): python3-scipy-1.0.0-21.module+el8.5.0 24 MB/s | 14 MB 00:00 2023-07-06T20:04:37.081 INFO:teuthology.orchestra.run.smithi156.stdout:(75/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 796 kB/s | 259 kB 00:00 2023-07-06T20:04:37.094 INFO:teuthology.orchestra.run.smithi069.stdout:(8/131): ceph-radosgw-18.0.0-4742.g79365bb3.el8 11 MB/s | 13 MB 00:01 2023-07-06T20:04:37.115 INFO:teuthology.orchestra.run.smithi156.stdout:(76/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.0 MB/s | 323 kB 00:00 2023-07-06T20:04:37.140 INFO:teuthology.orchestra.run.smithi156.stdout:(77/131): python3-markupsafe-0.23-19.el8.x86_64 464 kB/s | 39 kB 00:00 2023-07-06T20:04:37.216 INFO:teuthology.orchestra.run.smithi156.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 5.0 MB/s | 688 kB 00:00 2023-07-06T20:04:37.274 INFO:teuthology.orchestra.run.smithi156.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.3 MB/s | 538 kB 00:00 2023-07-06T20:04:37.307 INFO:teuthology.orchestra.run.smithi156.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 939 kB/s | 157 kB 00:00 2023-07-06T20:04:37.311 INFO:teuthology.orchestra.run.smithi069.stdout:(9/131): ceph-osd-18.0.0-4742.g79365bb3.el8.x86 9.9 MB/s | 18 MB 00:01 2023-07-06T20:04:37.336 INFO:teuthology.orchestra.run.smithi069.stdout:(10/131): ceph-selinux-18.0.0-4742.g79365bb3.el 100 kB/s | 24 kB 00:00 2023-07-06T20:04:37.399 INFO:teuthology.orchestra.run.smithi156.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2023-07-06T20:04:37.403 INFO:teuthology.orchestra.run.smithi069.stdout:(11/131): cephfs-mirror-18.0.0-4742.g79365bb3.e 3.8 MB/s | 258 kB 00:00 2023-07-06T20:04:37.433 INFO:teuthology.orchestra.run.smithi156.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 4.4 MB/s | 713 kB 00:00 2023-07-06T20:04:37.453 INFO:teuthology.orchestra.run.smithi069.stdout:(12/131): libcephfs-devel-18.0.0-4742.g79365bb3 624 kB/s | 31 kB 00:00 2023-07-06T20:04:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 167 kB/s | 27 kB 00:00 2023-07-06T20:04:37.587 INFO:teuthology.orchestra.run.smithi069.stdout:(13/131): libcephfs2-18.0.0-4742.g79365bb3.el8. 5.6 MB/s | 770 kB 00:00 2023-07-06T20:04:37.634 INFO:teuthology.orchestra.run.smithi156.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2023-07-06T20:04:37.654 INFO:teuthology.orchestra.run.smithi069.stdout:(14/131): libcephsqlite-18.0.0-4742.g79365bb3.e 2.0 MB/s | 137 kB 00:00 2023-07-06T20:04:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 1.4 MB/s | 320 kB 00:00 2023-07-06T20:04:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:(86/131): python3-prettytable-0.7.2-14.el8.noar 350 kB/s | 44 kB 00:00 2023-07-06T20:04:37.721 INFO:teuthology.orchestra.run.smithi069.stdout:(15/131): librados-devel-18.0.0-4742.g79365bb3. 2.0 MB/s | 139 kB 00:00 2023-07-06T20:04:37.768 INFO:teuthology.orchestra.run.smithi156.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 4.1 MB/s | 457 kB 00:00 2023-07-06T20:04:37.805 INFO:teuthology.orchestra.run.smithi069.stdout:(16/131): libradosstriper1-18.0.0-4742.g79365bb 5.6 MB/s | 475 kB 00:00 2023-07-06T20:04:37.835 INFO:teuthology.orchestra.run.smithi156.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 19 MB/s | 3.7 MB 00:00 2023-07-06T20:04:37.860 INFO:teuthology.orchestra.run.smithi156.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 697 kB/s | 64 kB 00:00 2023-07-06T20:04:37.885 INFO:teuthology.orchestra.run.smithi156.stdout:(90/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.1 MB/s | 225 kB 00:00 2023-07-06T20:04:37.952 INFO:teuthology.orchestra.run.smithi156.stdout:(91/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-07-06T20:04:37.977 INFO:teuthology.orchestra.run.smithi156.stdout:(92/131): python3-jsonpointer-1.10-11.el8.noarc 171 kB/s | 20 kB 00:00 2023-07-06T20:04:38.036 INFO:teuthology.orchestra.run.smithi156.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 687 kB/s | 103 kB 00:00 2023-07-06T20:04:38.086 INFO:teuthology.orchestra.run.smithi156.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2023-07-06T20:04:38.145 INFO:teuthology.orchestra.run.smithi156.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.8 MB/s | 198 kB 00:00 2023-07-06T20:04:38.212 INFO:teuthology.orchestra.run.smithi156.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 21 MB/s | 4.9 MB 00:00 2023-07-06T20:04:38.354 INFO:teuthology.orchestra.run.smithi156.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-07-06T20:04:38.379 INFO:teuthology.orchestra.run.smithi156.stdout:(98/131): jq-1.6-3.el8.x86_64.rpm 864 kB/s | 202 kB 00:00 2023-07-06T20:04:38.413 INFO:teuthology.orchestra.run.smithi156.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.7 MB/s | 354 kB 00:00 2023-07-06T20:04:38.480 INFO:teuthology.orchestra.run.smithi069.stdout:(17/131): librgw2-18.0.0-4742.g79365bb3.el8.x86 10 MB/s | 6.7 MB 00:00 2023-07-06T20:04:38.530 INFO:teuthology.orchestra.run.smithi156.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.3 MB/s | 808 kB 00:00 2023-07-06T20:04:38.555 INFO:teuthology.orchestra.run.smithi069.stdout:(18/131): python3-ceph-argparse-18.0.0-4742.g79 598 kB/s | 45 kB 00:00 2023-07-06T20:04:38.597 INFO:teuthology.orchestra.run.smithi156.stdout:(101/131): postgresql-server-10.21-2.module+el8 21 MB/s | 5.1 MB 00:00 2023-07-06T20:04:38.630 INFO:teuthology.orchestra.run.smithi069.stdout:(19/131): python3-ceph-common-18.0.0-4742.g7936 1.6 MB/s | 119 kB 00:00 2023-07-06T20:04:38.639 INFO:teuthology.orchestra.run.smithi156.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 6.7 MB/s | 1.5 MB 00:00 2023-07-06T20:04:38.656 INFO:teuthology.orchestra.run.smithi156.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 373 kB/s | 47 kB 00:00 2023-07-06T20:04:38.698 INFO:teuthology.orchestra.run.smithi156.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.9 MB/s | 200 kB 00:00 2023-07-06T20:04:38.707 INFO:teuthology.orchestra.run.smithi069.stdout:(20/131): python3-cephfs-18.0.0-4742.g79365bb3. 2.6 MB/s | 197 kB 00:00 2023-07-06T20:04:38.773 INFO:teuthology.orchestra.run.smithi156.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-07-06T20:04:38.823 INFO:teuthology.orchestra.run.smithi156.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 384 kB/s | 48 kB 00:00 2023-07-06T20:04:38.874 INFO:teuthology.orchestra.run.smithi156.stdout:(107/131): python3-jwt-1.6.1-2.el8.noarch.rpm 181 kB/s | 43 kB 00:00 2023-07-06T20:04:38.907 INFO:teuthology.orchestra.run.smithi156.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 649 kB/s | 87 kB 00:00 2023-07-06T20:04:38.931 INFO:teuthology.orchestra.run.smithi069.stdout:(21/131): ceph-common-18.0.0-4742.g79365bb3.el8 5.1 MB/s | 22 MB 00:04 2023-07-06T20:04:38.965 INFO:teuthology.orchestra.run.smithi069.stdout:(22/131): python3-rados-18.0.0-4742.g79365bb3.e 1.4 MB/s | 370 kB 00:00 2023-07-06T20:04:38.997 INFO:teuthology.orchestra.run.smithi156.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 899 kB/s | 109 kB 00:00 2023-07-06T20:04:39.015 INFO:teuthology.orchestra.run.smithi069.stdout:(23/131): python3-rgw-18.0.0-4742.g79365bb3.el8 2.1 MB/s | 107 kB 00:00 2023-07-06T20:04:39.026 INFO:teuthology.orchestra.run.smithi156.stdout:(110/131): python3-requests-oauthlib-1.0.0-1.el 212 kB/s | 43 kB 00:00 2023-07-06T20:04:39.040 INFO:teuthology.orchestra.run.smithi069.stdout:(24/131): python3-rbd-18.0.0-4742.g79365bb3.el8 3.3 MB/s | 364 kB 00:00 2023-07-06T20:04:39.050 INFO:teuthology.orchestra.run.smithi156.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2023-07-06T20:04:39.082 INFO:teuthology.orchestra.run.smithi069.stdout:(25/131): rbd-fuse-18.0.0-4742.g79365bb3.el8.x8 1.4 MB/s | 94 kB 00:00 2023-07-06T20:04:39.125 INFO:teuthology.orchestra.run.smithi156.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.2 MB/s | 560 kB 00:00 2023-07-06T20:04:39.149 INFO:teuthology.orchestra.run.smithi069.stdout:(26/131): rbd-nbd-18.0.0-4742.g79365bb3.el8.x86 2.6 MB/s | 179 kB 00:00 2023-07-06T20:04:39.184 INFO:teuthology.orchestra.run.smithi156.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2023-07-06T20:04:39.217 INFO:teuthology.orchestra.run.smithi156.stdout:(114/131): libaio-devel-0.3.112-1.el8.x86_64.rp 97 kB/s | 19 kB 00:00 2023-07-06T20:04:39.220 INFO:teuthology.orchestra.run.smithi069.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4742.g 347 kB/s | 23 kB 00:00 2023-07-06T20:04:39.276 INFO:teuthology.orchestra.run.smithi156.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 3.6 MB/s | 559 kB 00:00 2023-07-06T20:04:39.291 INFO:teuthology.orchestra.run.smithi069.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4742.g79365bb 1.7 MB/s | 129 kB 00:00 2023-07-06T20:04:39.302 INFO:teuthology.orchestra.run.smithi156.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 152 kB/s | 18 kB 00:00 2023-07-06T20:04:39.401 INFO:teuthology.orchestra.run.smithi156.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.2 MB/s | 605 kB 00:00 2023-07-06T20:04:39.427 INFO:teuthology.orchestra.run.smithi156.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2023-07-06T20:04:39.510 INFO:teuthology.orchestra.run.smithi156.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 11 MB/s | 2.3 MB 00:00 2023-07-06T20:04:39.525 INFO:teuthology.orchestra.run.smithi069.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4742.g79365 7.3 MB/s | 1.7 MB 00:00 2023-07-06T20:04:39.527 INFO:teuthology.orchestra.run.smithi156.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 576 kB/s | 72 kB 00:00 2023-07-06T20:04:39.553 INFO:teuthology.orchestra.run.smithi156.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 658 kB/s | 83 kB 00:00 2023-07-06T20:04:39.595 INFO:teuthology.orchestra.run.smithi156.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 713 kB/s | 60 kB 00:00 2023-07-06T20:04:39.600 INFO:teuthology.orchestra.run.smithi069.stdout:(30/131): rbd-mirror-18.0.0-4742.g79365bb3.el8. 6.2 MB/s | 3.5 MB 00:00 2023-07-06T20:04:39.704 INFO:teuthology.orchestra.run.smithi156.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 383 kB/s | 58 kB 00:00 2023-07-06T20:04:39.709 INFO:teuthology.orchestra.run.smithi069.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4742.g79 2.2 MB/s | 240 kB 00:00 2023-07-06T20:04:39.746 INFO:teuthology.orchestra.run.smithi156.stdout:(124/131): libuuid-devel-2.32.1-35.el8.x86_64.r 451 kB/s | 98 kB 00:00 2023-07-06T20:04:39.771 INFO:teuthology.orchestra.run.smithi156.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 219 kB/s | 39 kB 00:00 2023-07-06T20:04:39.818 INFO:teuthology.orchestra.run.smithi069.stdout:(32/131): ceph-mgr-rook-18.0.0-4742.g79365bb3.e 438 kB/s | 47 kB 00:00 2023-07-06T20:04:39.855 INFO:teuthology.orchestra.run.smithi156.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 4.2 MB/s | 644 kB 00:00 2023-07-06T20:04:39.880 INFO:teuthology.orchestra.run.smithi156.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.2 MB/s | 171 kB 00:00 2023-07-06T20:04:39.905 INFO:teuthology.orchestra.run.smithi156.stdout:(128/131): NetworkManager-initscripts-updown-1. 1.0 MB/s | 138 kB 00:00 2023-07-06T20:04:39.918 INFO:teuthology.orchestra.run.smithi069.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4742.g7 146 kB/s | 15 kB 00:00 2023-07-06T20:04:40.076 INFO:teuthology.orchestra.run.smithi069.stdout:(34/131): ceph-volume-18.0.0-4742.g79365bb3.el8 1.6 MB/s | 258 kB 00:00 2023-07-06T20:04:40.097 INFO:teuthology.orchestra.run.smithi156.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.1 MB/s | 798 kB 00:00 2023-07-06T20:04:40.169 INFO:teuthology.orchestra.run.smithi069.stdout:(35/131): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-07-06T20:04:40.202 INFO:teuthology.orchestra.run.smithi069.stdout:(36/131): cephadm-18.0.0-4742.g79365bb3.el8.noa 1.3 MB/s | 166 kB 00:00 2023-07-06T20:04:40.236 INFO:teuthology.orchestra.run.smithi069.stdout:(37/131): cephfs-top-18.0.0-4742.g79365bb3.el8. 269 kB/s | 18 kB 00:00 2023-07-06T20:04:40.522 INFO:teuthology.orchestra.run.smithi156.stdout:(130/131): librbd1-18.0.0-4742.g79365bb3.el8.x8 5.5 MB/s | 3.6 MB 00:00 2023-07-06T20:04:40.614 INFO:teuthology.orchestra.run.smithi156.stdout:(131/131): librados2-18.0.0-4742.g79365bb3.el8. 4.8 MB/s | 3.7 MB 00:00 2023-07-06T20:04:40.619 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-07-06T20:04:40.619 INFO:teuthology.orchestra.run.smithi156.stdout:Total 9.8 MB/s | 205 MB 00:20 2023-07-06T20:04:40.745 INFO:teuthology.orchestra.run.smithi069.stdout:(38/131): ceph-test-18.0.0-4742.g79365bb3.el8.x 14 MB/s | 48 MB 00:03 2023-07-06T20:04:40.770 INFO:teuthology.orchestra.run.smithi069.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 110 kB/s | 59 kB 00:00 2023-07-06T20:04:40.995 INFO:teuthology.orchestra.run.smithi069.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 301 kB/s | 75 kB 00:00 2023-07-06T20:04:41.029 INFO:teuthology.orchestra.run.smithi069.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 370 kB/s | 306 kB 00:00 2023-07-06T20:04:41.113 INFO:teuthology.orchestra.run.smithi069.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 373 kB/s | 44 kB 00:00 2023-07-06T20:04:41.154 INFO:teuthology.orchestra.run.smithi069.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 799 kB/s | 33 kB 00:00 2023-07-06T20:04:41.188 INFO:teuthology.orchestra.run.smithi069.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2023-07-06T20:04:41.213 INFO:teuthology.orchestra.run.smithi069.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 279 kB/s | 16 kB 00:00 2023-07-06T20:04:41.239 INFO:teuthology.orchestra.run.smithi069.stdout:(46/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.0 MB/s | 476 kB 00:00 2023-07-06T20:04:41.289 INFO:teuthology.orchestra.run.smithi069.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-07-06T20:04:41.314 INFO:teuthology.orchestra.run.smithi069.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-07-06T20:04:41.341 INFO:teuthology.orchestra.run.smithi069.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2023-07-06T20:04:41.365 INFO:teuthology.orchestra.run.smithi069.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 352 kB/s | 18 kB 00:00 2023-07-06T20:04:41.390 INFO:teuthology.orchestra.run.smithi069.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-07-06T20:04:41.415 INFO:teuthology.orchestra.run.smithi069.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 978 kB/s | 49 kB 00:00 2023-07-06T20:04:41.429 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-07-06T20:04:41.449 INFO:teuthology.orchestra.run.smithi069.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-07-06T20:04:41.474 INFO:teuthology.orchestra.run.smithi069.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-07-06T20:04:41.516 INFO:teuthology.orchestra.run.smithi069.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 376 kB/s | 16 kB 00:00 2023-07-06T20:04:41.548 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-07-06T20:04:41.548 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-07-06T20:04:41.550 INFO:teuthology.orchestra.run.smithi069.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-07-06T20:04:41.600 INFO:teuthology.orchestra.run.smithi069.stdout:(57/131): python3-kubernetes-11.0.0-6.el8.noarc 4.5 MB/s | 1.2 MB 00:00 2023-07-06T20:04:41.625 INFO:teuthology.orchestra.run.smithi069.stdout:(58/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2023-07-06T20:04:41.651 INFO:teuthology.orchestra.run.smithi069.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2023-07-06T20:04:41.676 INFO:teuthology.orchestra.run.smithi069.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 375 kB/s | 19 kB 00:00 2023-07-06T20:04:41.701 INFO:teuthology.orchestra.run.smithi069.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 481 kB/s | 24 kB 00:00 2023-07-06T20:04:41.727 INFO:teuthology.orchestra.run.smithi069.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 586 kB/s | 29 kB 00:00 2023-07-06T20:04:41.751 INFO:teuthology.orchestra.run.smithi069.stdout:(63/131): python3-routes-2.4.1-12.el8.noarch.rp 977 kB/s | 196 kB 00:00 2023-07-06T20:04:41.777 INFO:teuthology.orchestra.run.smithi069.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 366 kB/s | 27 kB 00:00 2023-07-06T20:04:41.803 INFO:teuthology.orchestra.run.smithi069.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-07-06T20:04:41.828 INFO:teuthology.orchestra.run.smithi069.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-07-06T20:04:41.853 INFO:teuthology.orchestra.run.smithi069.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-07-06T20:04:41.900 INFO:teuthology.orchestra.run.smithi069.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 321 kB/s | 23 kB 00:00 2023-07-06T20:04:41.929 INFO:teuthology.orchestra.run.smithi069.stdout:(69/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.4 MB/s | 251 kB 00:00 2023-07-06T20:04:41.954 INFO:teuthology.orchestra.run.smithi069.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.3 MB/s | 70 kB 00:00 2023-07-06T20:04:42.104 INFO:teuthology.orchestra.run.smithi069.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 6.9 MB/s | 1.7 MB 00:00 2023-07-06T20:04:42.146 INFO:teuthology.orchestra.run.smithi069.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 575 kB/s | 110 kB 00:00 2023-07-06T20:04:42.280 INFO:teuthology.orchestra.run.smithi069.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2023-07-06T20:04:42.305 INFO:teuthology.orchestra.run.smithi069.stdout:(74/131): python3-pytz-2017.2-9.el8.noarch.rpm 271 kB/s | 54 kB 00:00 2023-07-06T20:04:42.505 INFO:teuthology.orchestra.run.smithi069.stdout:(75/131): python3-scipy-1.0.0-21.module+el8.5.0 24 MB/s | 14 MB 00:00 2023-07-06T20:04:42.539 INFO:teuthology.orchestra.run.smithi069.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 166 kB/s | 39 kB 00:00 2023-07-06T20:04:42.572 INFO:teuthology.orchestra.run.smithi069.stdout:(77/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.1 MB/s | 323 kB 00:00 2023-07-06T20:04:42.648 INFO:teuthology.orchestra.run.smithi069.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 4.7 MB/s | 688 kB 00:00 2023-07-06T20:04:42.666 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-07-06T20:04:42.670 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-07-06T20:04:42.673 INFO:teuthology.orchestra.run.smithi069.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.5 MB/s | 157 kB 00:00 2023-07-06T20:04:42.706 INFO:teuthology.orchestra.run.smithi069.stdout:(80/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.1 MB/s | 538 kB 00:00 2023-07-06T20:04:42.823 INFO:teuthology.orchestra.run.smithi069.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2023-07-06T20:04:42.899 INFO:teuthology.orchestra.run.smithi069.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 3.1 MB/s | 713 kB 00:00 2023-07-06T20:04:42.982 INFO:teuthology.orchestra.run.smithi069.stdout:(83/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-07-06T20:04:43.008 INFO:teuthology.orchestra.run.smithi069.stdout:(84/131): python3-jsonpatch-1.21-2.el8.noarch.r 144 kB/s | 27 kB 00:00 2023-07-06T20:04:43.091 INFO:teuthology.orchestra.run.smithi069.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2023-07-06T20:04:43.116 INFO:teuthology.orchestra.run.smithi069.stdout:(86/131): python3-prettytable-0.7.2-14.el8.noar 329 kB/s | 44 kB 00:00 2023-07-06T20:04:43.217 INFO:teuthology.orchestra.run.smithi069.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 18 MB/s | 3.7 MB 00:00 2023-07-06T20:04:43.242 INFO:teuthology.orchestra.run.smithi069.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.8 MB/s | 225 kB 00:00 2023-07-06T20:04:43.276 INFO:teuthology.orchestra.run.smithi069.stdout:(89/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.4 MB/s | 457 kB 00:00 2023-07-06T20:04:43.384 INFO:teuthology.orchestra.run.smithi069.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 383 kB/s | 64 kB 00:00 2023-07-06T20:04:43.409 INFO:teuthology.orchestra.run.smithi069.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 150 kB/s | 20 kB 00:00 2023-07-06T20:04:43.435 INFO:teuthology.orchestra.run.smithi069.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 657 kB/s | 126 kB 00:00 2023-07-06T20:04:43.543 INFO:teuthology.orchestra.run.smithi069.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 652 kB/s | 103 kB 00:00 2023-07-06T20:04:43.585 INFO:teuthology.orchestra.run.smithi069.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 3.9 MB/s | 709 kB 00:00 2023-07-06T20:04:43.711 INFO:teuthology.orchestra.run.smithi069.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 18 MB/s | 4.9 MB 00:00 2023-07-06T20:04:43.736 INFO:teuthology.orchestra.run.smithi069.stdout:(96/131): libpq-13.5-1.el8.x86_64.rpm 1.0 MB/s | 198 kB 00:00 2023-07-06T20:04:43.895 INFO:teuthology.orchestra.run.smithi069.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-07-06T20:04:43.920 INFO:teuthology.orchestra.run.smithi069.stdout:(98/131): jq-1.6-3.el8.x86_64.rpm 968 kB/s | 202 kB 00:00 2023-07-06T20:04:43.970 INFO:teuthology.orchestra.run.smithi069.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.5 MB/s | 354 kB 00:00 2023-07-06T20:04:44.121 INFO:teuthology.orchestra.run.smithi069.stdout:(100/131): postgresql-server-10.21-2.module+el8 22 MB/s | 5.1 MB 00:00 2023-07-06T20:04:44.155 INFO:teuthology.orchestra.run.smithi069.stdout:(101/131): postgresql-contrib-10.21-2.module+el 3.4 MB/s | 808 kB 00:00 2023-07-06T20:04:44.213 INFO:teuthology.orchestra.run.smithi069.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 6.2 MB/s | 1.5 MB 00:00 2023-07-06T20:04:44.263 INFO:teuthology.orchestra.run.smithi069.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 330 kB/s | 47 kB 00:00 2023-07-06T20:04:44.297 INFO:teuthology.orchestra.run.smithi069.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.4 MB/s | 200 kB 00:00 2023-07-06T20:04:44.339 INFO:teuthology.orchestra.run.smithi069.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 339 kB/s | 43 kB 00:00 2023-07-06T20:04:44.414 INFO:teuthology.orchestra.run.smithi069.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 412 kB/s | 48 kB 00:00 2023-07-06T20:04:44.448 INFO:teuthology.orchestra.run.smithi069.stdout:(107/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2023-07-06T20:04:44.473 INFO:teuthology.orchestra.run.smithi069.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 649 kB/s | 87 kB 00:00 2023-07-06T20:04:44.548 INFO:teuthology.orchestra.run.smithi069.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 318 kB/s | 43 kB 00:00 2023-07-06T20:04:44.615 INFO:teuthology.orchestra.run.smithi069.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2023-07-06T20:04:44.649 INFO:teuthology.orchestra.run.smithi069.stdout:(111/131): python3-pycparser-2.14-14.el8.noarch 543 kB/s | 109 kB 00:00 2023-07-06T20:04:44.682 INFO:teuthology.orchestra.run.smithi069.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-07-06T20:04:44.766 INFO:teuthology.orchestra.run.smithi069.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 124 kB/s | 19 kB 00:00 2023-07-06T20:04:44.791 INFO:teuthology.orchestra.run.smithi069.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2023-07-06T20:04:44.841 INFO:teuthology.orchestra.run.smithi069.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 3.4 MB/s | 559 kB 00:00 2023-07-06T20:04:44.883 INFO:teuthology.orchestra.run.smithi069.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 153 kB/s | 18 kB 00:00 2023-07-06T20:04:44.942 INFO:teuthology.orchestra.run.smithi069.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2023-07-06T20:04:44.984 INFO:teuthology.orchestra.run.smithi069.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.6 MB/s | 238 kB 00:00 2023-07-06T20:04:45.051 INFO:teuthology.orchestra.run.smithi069.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 664 kB/s | 72 kB 00:00 2023-07-06T20:04:45.126 INFO:teuthology.orchestra.run.smithi069.stdout:(120/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 9.6 MB/s | 2.3 MB 00:00 2023-07-06T20:04:45.160 INFO:teuthology.orchestra.run.smithi069.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 471 kB/s | 83 kB 00:00 2023-07-06T20:04:45.185 INFO:teuthology.orchestra.run.smithi069.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 448 kB/s | 60 kB 00:00 2023-07-06T20:04:45.269 INFO:teuthology.orchestra.run.smithi069.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 691 kB/s | 98 kB 00:00 2023-07-06T20:04:45.294 INFO:teuthology.orchestra.run.smithi069.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 430 kB/s | 58 kB 00:00 2023-07-06T20:04:45.328 INFO:teuthology.orchestra.run.smithi069.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 271 kB/s | 39 kB 00:00 2023-07-06T20:04:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 4.2 MB/s | 644 kB 00:00 2023-07-06T20:04:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.1 MB/s | 171 kB 00:00 2023-07-06T20:04:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:(128/131): NetworkManager-initscripts-updown-1. 968 kB/s | 138 kB 00:00 2023-07-06T20:04:45.620 INFO:teuthology.orchestra.run.smithi069.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.2 MB/s | 798 kB 00:00 2023-07-06T20:04:45.762 INFO:teuthology.orchestra.run.smithi069.stdout:(130/131): librados2-18.0.0-4742.g79365bb3.el8. 11 MB/s | 3.7 MB 00:00 2023-07-06T20:04:45.846 INFO:teuthology.orchestra.run.smithi069.stdout:(131/131): librbd1-18.0.0-4742.g79365bb3.el8.x8 8.9 MB/s | 3.6 MB 00:00 2023-07-06T20:04:45.850 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2023-07-06T20:04:45.850 INFO:teuthology.orchestra.run.smithi069.stdout:Total 18 MB/s | 205 MB 00:11 2023-07-06T20:04:47.464 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2023-07-06T20:04:47.571 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-07-06T20:04:47.597 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2023-07-06T20:04:47.598 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2023-07-06T20:04:47.753 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-06T20:04:47.952 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-06T20:04:48.153 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-06T20:04:48.305 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-06T20:04:48.392 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-06T20:04:48.647 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-06T20:04:49.044 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-06T20:04:49.168 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2023-07-06T20:04:49.172 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2023-07-06T20:04:49.247 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T20:04:49.640 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T20:04:49.880 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T20:04:50.006 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T20:04:50.188 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T20:04:50.286 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T20:04:50.480 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 9/134 2023-07-06T20:04:50.691 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-06T20:04:50.907 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-06T20:04:51.294 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-06T20:04:51.574 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-06T20:04:51.721 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T20:04:51.825 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T20:04:51.979 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T20:04:52.053 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T20:04:52.162 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-06T20:04:52.306 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-06T20:04:52.496 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-06T20:04:53.106 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-06T20:04:53.321 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-06T20:04:53.914 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-06T20:04:54.104 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-06T20:04:54.149 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2023-07-06T20:04:54.324 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-06T20:04:54.498 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-06T20:04:54.719 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-06T20:04:54.792 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-06T20:04:54.916 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-06T20:04:55.035 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-06T20:04:55.051 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-06T20:04:55.254 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-06T20:04:55.275 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-06T20:04:55.474 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-06T20:04:55.660 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-06T20:04:55.743 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-06T20:04:55.893 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-06T20:04:55.949 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T20:04:56.105 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-06T20:04:56.327 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 26/134 2023-07-06T20:04:56.406 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T20:04:56.457 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 27/134 2023-07-06T20:04:56.591 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T20:04:56.745 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T20:04:56.950 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T20:04:57.064 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T20:04:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 9/134 2023-07-06T20:04:57.343 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 28/134 2023-07-06T20:04:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-06T20:04:57.668 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-06T20:04:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-06T20:04:57.964 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-06T20:04:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-06T20:04:58.193 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-06T20:04:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-06T20:04:58.420 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-06T20:04:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T20:04:58.622 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T20:04:58.766 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T20:04:58.866 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T20:04:59.016 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-06T20:04:59.189 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-06T20:04:59.349 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-06T20:05:00.045 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-06T20:05:00.181 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-06T20:05:00.224 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-06T20:05:00.555 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-06T20:05:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-06T20:05:00.927 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-06T20:05:00.979 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-06T20:05:01.115 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-06T20:05:01.175 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 36/134 2023-07-06T20:05:01.381 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 37/134 2023-07-06T20:05:01.526 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-06T20:05:01.731 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-06T20:05:01.922 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-06T20:05:01.952 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-06T20:05:02.072 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-06T20:05:02.240 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-06T20:05:02.325 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-06T20:05:02.483 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-06T20:05:02.507 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-06T20:05:02.722 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-06T20:05:02.924 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-06T20:05:03.183 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-06T20:05:03.220 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-06T20:05:03.443 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-06T20:05:03.497 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-06T20:05:03.631 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 26/134 2023-07-06T20:05:03.720 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-06T20:05:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 27/134 2023-07-06T20:05:03.927 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-06T20:05:04.165 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-06T20:05:04.502 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-06T20:05:04.760 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-06T20:05:04.788 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 28/134 2023-07-06T20:05:04.953 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-06T20:05:05.141 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-06T20:05:05.195 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-06T20:05:05.348 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-06T20:05:05.472 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-06T20:05:05.514 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-06T20:05:05.784 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-06T20:05:05.844 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-06T20:05:05.973 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-06T20:05:06.099 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-06T20:05:06.381 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-06T20:05:06.617 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-06T20:05:06.818 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-06T20:05:06.962 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-06T20:05:07.177 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-06T20:05:07.241 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-06T20:05:07.481 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-06T20:05:07.722 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-06T20:05:07.860 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-06T20:05:07.936 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-06T20:05:08.052 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-06T20:05:08.205 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-06T20:05:08.385 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-06T20:05:08.398 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-06T20:05:08.542 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-06T20:05:08.625 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-06T20:05:08.745 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-06T20:05:08.827 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-06T20:05:09.001 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-06T20:05:09.021 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 36/134 2023-07-06T20:05:09.233 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-06T20:05:09.244 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 37/134 2023-07-06T20:05:09.380 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-06T20:05:09.406 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-06T20:05:09.528 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-06T20:05:09.625 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-06T20:05:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-06T20:05:09.825 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-06T20:05:09.971 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-06T20:05:09.978 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-06T20:05:10.187 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-06T20:05:10.210 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-06T20:05:10.363 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-06T20:05:10.436 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-06T20:05:10.507 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-06T20:05:10.748 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-06T20:05:10.846 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-06T20:05:10.981 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-06T20:05:11.130 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-06T20:05:11.216 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-06T20:05:11.346 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-06T20:05:11.467 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-06T20:05:11.592 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-06T20:05:11.692 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-06T20:05:11.857 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-06T20:05:11.859 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-06T20:05:12.086 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-06T20:05:12.204 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-06T20:05:12.423 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-06T20:05:12.592 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-06T20:05:12.822 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-06T20:05:13.004 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-06T20:05:13.183 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-06T20:05:13.198 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-06T20:05:13.521 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-06T20:05:13.528 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-06T20:05:13.808 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-06T20:05:13.817 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-06T20:05:14.090 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-06T20:05:14.120 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-06T20:05:14.379 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-06T20:05:14.388 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-06T20:05:14.556 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-06T20:05:14.564 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-06T20:05:14.757 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-06T20:05:14.946 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-06T20:05:15.040 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-06T20:05:15.295 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-06T20:05:15.374 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-06T20:05:15.518 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-06T20:05:15.525 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T20:05:15.616 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T20:05:15.680 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-06T20:05:15.776 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-06T20:05:15.793 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 93/134 2023-07-06T20:05:15.977 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-06T20:05:16.157 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-06T20:05:16.323 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-06T20:05:16.537 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-06T20:05:16.848 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-06T20:05:17.071 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-06T20:05:17.226 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-06T20:05:17.384 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-06T20:05:17.585 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-06T20:05:17.795 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-06T20:05:17.977 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-06T20:05:18.158 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-06T20:05:18.303 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-06T20:05:18.520 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-06T20:05:18.765 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-06T20:05:18.962 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-06T20:05:19.211 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-06T20:05:19.422 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-06T20:05:19.479 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T20:05:19.598 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-06T20:05:19.717 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T20:05:19.847 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-06T20:05:20.309 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T20:05:20.636 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T20:05:20.699 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T20:05:20.808 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T20:05:21.177 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-06T20:05:21.449 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-06T20:05:21.722 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-06T20:05:22.095 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-06T20:05:22.394 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-06T20:05:22.595 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-06T20:05:23.444 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-06T20:05:23.612 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T20:05:23.694 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T20:05:23.871 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 93/134 2023-07-06T20:05:28.648 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T20:05:28.895 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T20:05:29.587 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T20:05:29.869 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T20:05:29.997 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T20:05:30.120 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T20:05:39.592 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T20:05:39.592 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2023-07-06T20:05:39.593 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2023-07-06T20:05:39.594 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-07-06T20:05:39.594 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /run 2023-07-06T20:05:39.594 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2023-07-06T20:05:39.594 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2023-07-06T20:05:39.594 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2023-07-06T20:05:39.594 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2023-07-06T20:05:39.594 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-07-06T20:05:39.595 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:05:39.939 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T20:05:41.521 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T20:05:41.741 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T20:05:41.958 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T20:05:42.172 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T20:05:42.625 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T20:05:42.780 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T20:05:42.797 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T20:05:44.444 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T20:05:44.668 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T20:05:44.794 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 102/134 2023-07-06T20:05:45.043 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 103/134 2023-07-06T20:05:45.238 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T20:05:45.454 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T20:05:45.698 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T20:05:46.673 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T20:05:46.829 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T20:05:46.920 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T20:05:47.319 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 107/134 2023-07-06T20:05:47.560 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T20:05:47.596 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T20:05:47.745 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T20:05:47.756 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T20:05:49.032 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 110/134 2023-07-06T20:05:49.244 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T20:05:49.278 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T20:05:49.429 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T20:05:49.470 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T20:05:49.628 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T20:05:49.978 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T20:05:50.145 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T20:05:50.496 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T20:05:50.497 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2023-07-06T20:05:50.497 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2023-07-06T20:05:50.497 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /dev 2023-07-06T20:05:50.497 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /run 2023-07-06T20:05:50.497 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2023-07-06T20:05:50.497 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2023-07-06T20:05:50.497 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2023-07-06T20:05:50.498 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2023-07-06T20:05:50.498 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /dev 2023-07-06T20:05:50.498 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:05:50.900 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T20:05:52.637 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T20:05:52.808 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T20:05:53.074 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T20:05:53.258 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T20:05:53.783 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T20:05:53.982 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T20:05:54.023 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T20:05:54.675 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T20:05:54.990 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 115/134 2023-07-06T20:05:55.144 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-06T20:05:55.340 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-06T20:05:56.236 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-06T20:05:56.584 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-06T20:05:56.672 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T20:05:56.781 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 119/134 2023-07-06T20:05:56.852 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T20:05:56.996 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 102/134 2023-07-06T20:05:57.006 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-06T20:05:57.182 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 121/134 2023-07-06T20:05:57.196 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 103/134 2023-07-06T20:05:57.370 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T20:05:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T20:05:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T20:05:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T20:05:58.995 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T20:05:59.165 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T20:05:59.292 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T20:05:59.482 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T20:05:59.665 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 107/134 2023-07-06T20:05:59.710 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T20:05:59.728 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T20:05:59.895 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T20:05:59.911 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 124/134 2023-07-06T20:05:59.959 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T20:06:00.131 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T20:06:00.159 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T20:06:00.210 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 125/134 2023-07-06T20:06:00.548 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 126/134 2023-07-06T20:06:00.870 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-06T20:06:01.019 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T20:06:01.027 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T20:06:01.205 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-06T20:06:01.427 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-06T20:06:01.514 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-06T20:06:01.603 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 110/134 2023-07-06T20:06:01.710 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-06T20:06:01.723 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-06T20:06:01.724 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T20:06:01.801 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T20:06:01.862 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T20:06:01.930 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T20:06:01.930 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T20:06:02.006 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T20:06:02.077 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T20:06:02.149 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T20:06:02.248 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T20:06:02.376 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T20:06:02.655 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T20:06:02.823 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T20:06:02.885 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T20:06:02.885 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 1/134 2023-07-06T20:06:02.885 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2/134 2023-07-06T20:06:02.885 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 3/134 2023-07-06T20:06:02.885 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 4/134 2023-07-06T20:06:02.885 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4742.g79365 5/134 2023-07-06T20:06:02.886 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T20:06:02.886 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T20:06:02.886 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T20:06:02.886 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 9/134 2023-07-06T20:06:02.886 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 10/134 2023-07-06T20:06:02.886 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 11/134 2023-07-06T20:06:02.886 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 12/134 2023-07-06T20:06:02.887 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 13/134 2023-07-06T20:06:02.887 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 14/134 2023-07-06T20:06:02.887 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 15/134 2023-07-06T20:06:02.887 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 16/134 2023-07-06T20:06:02.887 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 17/134 2023-07-06T20:06:02.887 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 18/134 2023-07-06T20:06:02.887 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 19/134 2023-07-06T20:06:02.887 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 20/134 2023-07-06T20:06:02.888 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 21/134 2023-07-06T20:06:02.888 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 22/134 2023-07-06T20:06:02.888 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 23/134 2023-07-06T20:06:02.888 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 24/134 2023-07-06T20:06:02.888 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 25/134 2023-07-06T20:06:02.888 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 26/134 2023-07-06T20:06:02.888 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 27/134 2023-07-06T20:06:02.889 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 28/134 2023-07-06T20:06:02.889 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 29/134 2023-07-06T20:06:02.889 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 30/134 2023-07-06T20:06:02.896 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 31/134 2023-07-06T20:06:02.896 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 32/134 2023-07-06T20:06:02.896 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 33/134 2023-07-06T20:06:02.897 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 34/134 2023-07-06T20:06:02.897 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 35/134 2023-07-06T20:06:02.897 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 36/134 2023-07-06T20:06:02.897 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 37/134 2023-07-06T20:06:02.897 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 38/134 2023-07-06T20:06:02.897 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-06T20:06:02.897 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-06T20:06:02.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-06T20:06:02.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-06T20:06:02.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-06T20:06:02.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-06T20:06:02.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-06T20:06:02.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-06T20:06:02.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-06T20:06:02.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-06T20:06:02.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-06T20:06:02.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-06T20:06:02.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-06T20:06:02.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-06T20:06:02.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-06T20:06:02.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-06T20:06:02.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-06T20:06:02.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-06T20:06:02.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-06T20:06:02.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-06T20:06:02.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-06T20:06:02.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-06T20:06:02.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-06T20:06:02.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-06T20:06:02.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-06T20:06:02.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-06T20:06:02.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-06T20:06:02.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-06T20:06:02.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-06T20:06:02.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-06T20:06:02.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-06T20:06:02.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-06T20:06:02.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-06T20:06:02.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-06T20:06:02.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-06T20:06:02.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-06T20:06:02.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-06T20:06:02.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-06T20:06:02.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-06T20:06:02.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-06T20:06:02.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-06T20:06:02.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-06T20:06:02.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-06T20:06:02.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-06T20:06:02.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-06T20:06:02.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-06T20:06:02.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-06T20:06:02.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-06T20:06:02.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-06T20:06:02.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-06T20:06:02.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-06T20:06:02.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-06T20:06:02.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-06T20:06:02.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-06T20:06:02.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-06T20:06:02.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-06T20:06:02.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-06T20:06:02.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-06T20:06:02.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-06T20:06:02.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-06T20:06:02.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-06T20:06:02.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-06T20:06:02.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-06T20:06:02.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-06T20:06:02.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-06T20:06:02.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-06T20:06:02.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-06T20:06:02.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-06T20:06:02.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-06T20:06:02.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-06T20:06:02.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-06T20:06:02.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-06T20:06:02.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-06T20:06:02.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-06T20:06:02.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-06T20:06:02.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-06T20:06:02.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-06T20:06:02.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-06T20:06:02.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-06T20:06:02.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-06T20:06:02.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-06T20:06:02.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-06T20:06:02.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-06T20:06:02.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T20:06:02.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-06T20:06:02.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-06T20:06:02.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-06T20:06:02.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-06T20:06:02.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-06T20:06:02.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T20:06:02.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 129/134 2023-07-06T20:06:02.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-06T20:06:02.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 131/134 2023-07-06T20:06:02.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T20:06:02.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-06T20:06:05.409 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T20:06:05.410 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:58 ago on Thu 06 Jul 2023 08:04:07 PM UTC. 2023-07-06T20:06:07.791 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T20:06:08.093 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 115/134 2023-07-06T20:06:08.255 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-06T20:06:08.504 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-06T20:06:09.737 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-06T20:06:10.067 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-06T20:06:10.285 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 119/134 2023-07-06T20:06:10.476 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-06T20:06:10.685 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 121/134 2023-07-06T20:06:10.847 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T20:06:14.092 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T20:06:14.379 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T20:06:14.417 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T20:06:14.622 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 124/134 2023-07-06T20:06:14.847 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 125/134 2023-07-06T20:06:15.188 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 126/134 2023-07-06T20:06:15.506 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-06T20:06:15.695 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T20:06:15.717 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T20:06:15.936 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-06T20:06:16.137 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-06T20:06:16.248 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-06T20:06:16.445 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-06T20:06:16.474 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-06T20:06:16.474 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T20:06:16.647 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-07-06T20:06:16.693 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T20:06:16.693 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T20:06:16.860 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:06:16.861 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2023-07-06T20:06:16.861 INFO:teuthology.orchestra.run.smithi156.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-06T20:06:16.861 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.861 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.861 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-07-06T20:06:16.862 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-06T20:06:16.862 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T20:06:16.862 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T20:06:16.862 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.862 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.862 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.862 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.863 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:16.863 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.863 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.863 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.863 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:16.863 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:16.863 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:16.864 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:16.864 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:16.864 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.864 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.864 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:16.865 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.865 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.865 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.865 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:16.865 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:16.865 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.866 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:16.866 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-06T20:06:16.866 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T20:06:16.866 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-06T20:06:16.866 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T20:06:16.866 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-3.el8.x86_64 2023-07-06T20:06:16.866 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T20:06:16.867 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-06T20:06:16.867 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T20:06:16.867 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.867 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.867 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.867 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-06T20:06:16.867 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-06T20:06:16.867 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T20:06:16.868 INFO:teuthology.orchestra.run.smithi156.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T20:06:16.868 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-06T20:06:16.868 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-06T20:06:16.868 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.868 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.869 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T20:06:16.869 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.869 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-06T20:06:16.869 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T20:06:16.869 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T20:06:16.869 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T20:06:16.869 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-06T20:06:16.869 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-06T20:06:16.870 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T20:06:16.870 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T20:06:16.870 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-06T20:06:16.870 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-06T20:06:16.870 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-06T20:06:16.870 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-06T20:06:16.870 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-06T20:06:16.871 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-06T20:06:16.871 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T20:06:16.871 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T20:06:16.871 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T20:06:16.872 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T20:06:16.872 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T20:06:16.872 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T20:06:16.873 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T20:06:16.873 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T20:06:16.873 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.873 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.874 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.874 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T20:06:16.874 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-06T20:06:16.874 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T20:06:16.875 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T20:06:16.875 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-06T20:06:16.875 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T20:06:16.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T20:06:16.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T20:06:16.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T20:06:16.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T20:06:16.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T20:06:16.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T20:06:16.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T20:06:16.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T20:06:16.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-06T20:06:16.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T20:06:16.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T20:06:16.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T20:06:16.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T20:06:16.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T20:06:16.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T20:06:16.880 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T20:06:16.880 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T20:06:16.880 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T20:06:16.880 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T20:06:16.881 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T20:06:16.881 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T20:06:16.881 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T20:06:16.881 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T20:06:16.882 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.882 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.882 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T20:06:16.882 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T20:06:16.883 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.883 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T20:06:16.883 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T20:06:16.883 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-06T20:06:16.883 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T20:06:16.883 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T20:06:16.883 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T20:06:16.884 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T20:06:16.884 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T20:06:16.884 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T20:06:16.884 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T20:06:16.884 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T20:06:16.884 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T20:06:16.884 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T20:06:16.885 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.885 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.885 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:16.885 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T20:06:16.885 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T20:06:16.885 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T20:06:16.885 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-06T20:06:16.886 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T20:06:16.886 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-06T20:06:16.886 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:06:16.886 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-07-06T20:06:16.935 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T20:06:17.036 DEBUG:teuthology.parallel:result is None 2023-07-06T20:06:17.153 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T20:06:17.423 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T20:06:17.423 INFO:teuthology.orchestra.run.smithi069.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-06T20:06:17.423 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:06:18.065 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 1/134 2023-07-06T20:06:18.065 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2/134 2023-07-06T20:06:18.065 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 3/134 2023-07-06T20:06:18.065 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 4/134 2023-07-06T20:06:18.065 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4742.g79365 5/134 2023-07-06T20:06:18.066 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T20:06:18.066 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T20:06:18.066 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T20:06:18.066 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 9/134 2023-07-06T20:06:18.066 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 10/134 2023-07-06T20:06:18.066 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 11/134 2023-07-06T20:06:18.066 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 12/134 2023-07-06T20:06:18.067 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 13/134 2023-07-06T20:06:18.067 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 14/134 2023-07-06T20:06:18.067 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 15/134 2023-07-06T20:06:18.067 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 16/134 2023-07-06T20:06:18.067 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 17/134 2023-07-06T20:06:18.067 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 18/134 2023-07-06T20:06:18.067 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 19/134 2023-07-06T20:06:18.068 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 20/134 2023-07-06T20:06:18.068 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 21/134 2023-07-06T20:06:18.068 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 22/134 2023-07-06T20:06:18.068 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 23/134 2023-07-06T20:06:18.068 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 24/134 2023-07-06T20:06:18.069 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 25/134 2023-07-06T20:06:18.078 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 26/134 2023-07-06T20:06:18.078 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 27/134 2023-07-06T20:06:18.078 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 28/134 2023-07-06T20:06:18.078 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 29/134 2023-07-06T20:06:18.078 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 30/134 2023-07-06T20:06:18.079 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 31/134 2023-07-06T20:06:18.079 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 32/134 2023-07-06T20:06:18.079 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 33/134 2023-07-06T20:06:18.079 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 34/134 2023-07-06T20:06:18.079 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 35/134 2023-07-06T20:06:18.079 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 36/134 2023-07-06T20:06:18.080 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 37/134 2023-07-06T20:06:18.080 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 38/134 2023-07-06T20:06:18.080 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-06T20:06:18.080 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-06T20:06:18.080 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-06T20:06:18.081 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-06T20:06:18.081 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-06T20:06:18.081 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-06T20:06:18.081 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-06T20:06:18.081 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-06T20:06:18.081 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-06T20:06:18.081 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-06T20:06:18.082 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-06T20:06:18.082 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-06T20:06:18.082 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-06T20:06:18.082 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-06T20:06:18.083 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-06T20:06:18.083 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-06T20:06:18.083 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-06T20:06:18.083 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-06T20:06:18.083 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-06T20:06:18.084 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-06T20:06:18.084 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-06T20:06:18.084 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-06T20:06:18.084 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-06T20:06:18.084 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-06T20:06:18.084 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-06T20:06:18.085 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-06T20:06:18.085 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-06T20:06:18.085 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-06T20:06:18.085 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-06T20:06:18.085 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-06T20:06:18.085 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-06T20:06:18.086 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-06T20:06:18.086 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-06T20:06:18.086 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-06T20:06:18.086 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-06T20:06:18.086 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-06T20:06:18.086 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-06T20:06:18.086 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-06T20:06:18.087 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-06T20:06:18.087 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-06T20:06:18.087 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-06T20:06:18.087 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-06T20:06:18.087 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-06T20:06:18.087 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-06T20:06:18.087 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-06T20:06:18.087 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-06T20:06:18.088 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-06T20:06:18.088 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-06T20:06:18.088 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-06T20:06:18.088 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-06T20:06:18.088 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-06T20:06:18.088 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-06T20:06:18.088 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-06T20:06:18.089 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-06T20:06:18.089 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-06T20:06:18.089 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-06T20:06:18.089 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-06T20:06:18.089 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-06T20:06:18.089 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-06T20:06:18.090 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-06T20:06:18.090 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-06T20:06:18.090 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-06T20:06:18.090 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-06T20:06:18.090 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-06T20:06:18.090 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-06T20:06:18.090 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-06T20:06:18.091 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-06T20:06:18.091 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-06T20:06:18.091 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-06T20:06:18.091 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-06T20:06:18.091 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-06T20:06:18.092 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-06T20:06:18.092 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-06T20:06:18.092 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-06T20:06:18.092 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-06T20:06:18.092 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-06T20:06:18.092 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-06T20:06:18.092 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-06T20:06:18.093 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-06T20:06:18.093 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-06T20:06:18.093 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-06T20:06:18.093 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-06T20:06:18.093 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-06T20:06:18.093 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T20:06:18.093 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-06T20:06:18.093 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-06T20:06:18.094 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-06T20:06:18.094 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-06T20:06:18.094 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-06T20:06:18.094 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T20:06:18.094 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 129/134 2023-07-06T20:06:18.094 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-06T20:06:18.094 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 131/134 2023-07-06T20:06:18.094 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T20:06:18.095 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-06T20:06:20.714 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T20:06:20.715 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:58 ago on Thu 06 Jul 2023 08:04:22 PM UTC. 2023-07-06T20:06:31.623 INFO:teuthology.orchestra.run.smithi069.stdout:Installed products updated. 2023-07-06T20:06:31.851 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:06:31.851 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2023-07-06T20:06:31.851 INFO:teuthology.orchestra.run.smithi069.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-06T20:06:31.851 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.851 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.852 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2023-07-06T20:06:31.852 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-06T20:06:31.852 INFO:teuthology.orchestra.run.smithi069.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T20:06:31.852 INFO:teuthology.orchestra.run.smithi069.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T20:06:31.852 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.852 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.852 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.853 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.853 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:31.853 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.853 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.853 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.854 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:31.854 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:31.855 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:31.855 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:31.855 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:31.855 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.856 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.856 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:31.856 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.856 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.856 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.856 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:31.856 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:31.857 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.857 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:06:31.857 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-06T20:06:31.858 INFO:teuthology.orchestra.run.smithi069.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T20:06:31.858 INFO:teuthology.orchestra.run.smithi069.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-06T20:06:31.858 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T20:06:31.858 INFO:teuthology.orchestra.run.smithi069.stdout: jq-1.6-3.el8.x86_64 2023-07-06T20:06:31.858 INFO:teuthology.orchestra.run.smithi069.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T20:06:31.859 INFO:teuthology.orchestra.run.smithi069.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-06T20:06:31.859 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T20:06:31.859 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.859 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.859 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.860 INFO:teuthology.orchestra.run.smithi069.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-06T20:06:31.860 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-06T20:06:31.860 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T20:06:31.860 INFO:teuthology.orchestra.run.smithi069.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T20:06:31.860 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-06T20:06:31.861 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-06T20:06:31.861 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.861 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.861 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T20:06:31.861 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.861 INFO:teuthology.orchestra.run.smithi069.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-06T20:06:31.861 INFO:teuthology.orchestra.run.smithi069.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T20:06:31.862 INFO:teuthology.orchestra.run.smithi069.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T20:06:31.862 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T20:06:31.862 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-06T20:06:31.862 INFO:teuthology.orchestra.run.smithi069.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-06T20:06:31.862 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T20:06:31.862 INFO:teuthology.orchestra.run.smithi069.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T20:06:31.862 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-06T20:06:31.862 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-06T20:06:31.863 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-06T20:06:31.863 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-06T20:06:31.863 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-06T20:06:31.863 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-06T20:06:31.863 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T20:06:31.863 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T20:06:31.863 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T20:06:31.864 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T20:06:31.864 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T20:06:31.865 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T20:06:31.865 INFO:teuthology.orchestra.run.smithi069.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T20:06:31.865 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T20:06:31.866 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.866 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.866 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.866 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T20:06:31.866 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-06T20:06:31.866 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T20:06:31.867 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T20:06:31.867 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-06T20:06:31.867 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T20:06:31.868 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T20:06:31.868 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T20:06:31.868 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T20:06:31.868 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T20:06:31.869 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T20:06:31.869 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T20:06:31.869 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T20:06:31.869 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T20:06:31.870 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-06T20:06:31.870 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T20:06:31.870 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T20:06:31.870 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T20:06:31.870 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T20:06:31.870 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T20:06:31.871 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T20:06:31.871 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T20:06:31.871 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T20:06:31.871 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T20:06:31.871 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T20:06:31.872 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T20:06:31.872 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T20:06:31.872 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T20:06:31.872 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T20:06:31.872 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.873 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.873 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T20:06:31.873 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T20:06:31.873 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.874 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T20:06:31.874 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T20:06:31.874 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-06T20:06:31.874 INFO:teuthology.orchestra.run.smithi069.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T20:06:31.874 INFO:teuthology.orchestra.run.smithi069.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T20:06:31.875 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T20:06:31.875 INFO:teuthology.orchestra.run.smithi069.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T20:06:31.875 INFO:teuthology.orchestra.run.smithi069.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T20:06:31.875 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T20:06:31.876 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T20:06:31.876 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T20:06:31.876 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T20:06:31.876 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T20:06:31.876 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.877 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.877 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:06:31.877 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T20:06:31.877 INFO:teuthology.orchestra.run.smithi069.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T20:06:31.877 INFO:teuthology.orchestra.run.smithi069.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T20:06:31.877 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-06T20:06:31.878 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T20:06:31.878 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-06T20:06:31.878 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:06:31.879 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2023-07-06T20:06:32.142 DEBUG:teuthology.parallel:result is None 2023-07-06T20:06:45.075 INFO:teuthology.orchestra.run.smithi098.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-06T20:06:45.076 INFO:teuthology.orchestra.run.smithi098.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-06T20:06:45.076 INFO:teuthology.orchestra.run.smithi098.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-06T20:06:45.454 INFO:teuthology.orchestra.run.smithi098.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 460 B/s | 59 kB 02:10 2023-07-06T20:06:45.479 INFO:teuthology.orchestra.run.smithi098.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 587 B/s | 75 kB 02:10 2023-07-06T20:06:45.609 INFO:teuthology.orchestra.run.smithi098.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 338 kB/s | 44 kB 00:00 2023-07-06T20:06:45.639 INFO:teuthology.orchestra.run.smithi098.stdout:(42/131): gperftools-libs-2.7-9.el8.x86_64.rpm 2.3 kB/s | 306 kB 02:11 2023-07-06T20:06:45.764 INFO:teuthology.orchestra.run.smithi098.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 264 kB/s | 33 kB 00:00 2023-07-06T20:06:45.847 INFO:teuthology.orchestra.run.smithi098.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 778 kB/s | 185 kB 00:00 2023-07-06T20:06:45.889 INFO:teuthology.orchestra.run.smithi098.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-07-06T20:06:46.023 INFO:teuthology.orchestra.run.smithi098.stdout:(46/131): python3-asyncssh-2.7.0-2.el8.noarch.r 837 kB/s | 476 kB 00:00 2023-07-06T20:06:46.048 INFO:teuthology.orchestra.run.smithi098.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 860 kB/s | 173 kB 00:00 2023-07-06T20:06:46.157 INFO:teuthology.orchestra.run.smithi098.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 609 kB/s | 81 kB 00:00 2023-07-06T20:06:46.182 INFO:teuthology.orchestra.run.smithi098.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-07-06T20:06:46.307 INFO:teuthology.orchestra.run.smithi098.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 117 kB/s | 18 kB 00:00 2023-07-06T20:06:46.358 INFO:teuthology.orchestra.run.smithi098.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 821 kB/s | 384 kB 00:00 2023-07-06T20:06:46.391 INFO:teuthology.orchestra.run.smithi098.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 582 kB/s | 49 kB 00:00 2023-07-06T20:06:46.475 INFO:teuthology.orchestra.run.smithi098.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 724 kB/s | 60 kB 00:00 2023-07-06T20:06:46.500 INFO:teuthology.orchestra.run.smithi098.stdout:(54/131): python3-more-itertools-7.2.0-3.el8.no 413 kB/s | 59 kB 00:00 2023-07-06T20:06:46.584 INFO:teuthology.orchestra.run.smithi098.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 187 kB/s | 16 kB 00:00 2023-07-06T20:06:46.667 INFO:teuthology.orchestra.run.smithi098.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 404 kB/s | 34 kB 00:00 2023-07-06T20:06:46.693 INFO:teuthology.orchestra.run.smithi098.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2023-07-06T20:06:46.760 INFO:teuthology.orchestra.run.smithi098.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 924 kB/s | 61 kB 00:00 2023-07-06T20:06:46.835 INFO:teuthology.orchestra.run.smithi098.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 250 kB/s | 19 kB 00:00 2023-07-06T20:06:46.919 INFO:teuthology.orchestra.run.smithi098.stdout:(60/131): python3-kubernetes-11.0.0-6.el8.noarc 1.6 MB/s | 1.2 MB 00:00 2023-07-06T20:06:46.983 INFO:teuthology.orchestra.run.smithi098.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 161 kB/s | 24 kB 00:00 2023-07-06T20:06:47.011 INFO:teuthology.orchestra.run.smithi098.stdout:(62/131): python3-routes-2.4.1-12.el8.noarch.rp 572 kB/s | 196 kB 00:00 2023-07-06T20:06:47.036 INFO:teuthology.orchestra.run.smithi098.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2023-07-06T20:06:47.061 INFO:teuthology.orchestra.run.smithi098.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 353 kB/s | 27 kB 00:00 2023-07-06T20:06:47.137 INFO:teuthology.orchestra.run.smithi098.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 816 kB/s | 61 kB 00:00 2023-07-06T20:06:47.212 INFO:teuthology.orchestra.run.smithi098.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 1.2 MB/s | 255 kB 00:00 2023-07-06T20:06:47.237 INFO:teuthology.orchestra.run.smithi098.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 853 kB/s | 85 kB 00:00 2023-07-06T20:06:47.323 INFO:teuthology.orchestra.run.smithi098.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 208 kB/s | 23 kB 00:00 2023-07-06T20:06:47.354 INFO:teuthology.orchestra.run.smithi098.stdout:(69/131): python3-webob-1.8.5-1.el8.1.noarch.rp 790 kB/s | 251 kB 00:00 2023-07-06T20:06:47.380 INFO:teuthology.orchestra.run.smithi098.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-07-06T20:06:47.505 INFO:teuthology.orchestra.run.smithi098.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 311 kB/s | 47 kB 00:00 2023-07-06T20:06:47.564 INFO:teuthology.orchestra.run.smithi098.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.1 MB/s | 200 kB 00:00 2023-07-06T20:06:47.681 INFO:teuthology.orchestra.run.smithi098.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 243 kB/s | 43 kB 00:00 2023-07-06T20:06:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2023-07-06T20:06:47.790 INFO:teuthology.orchestra.run.smithi098.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 443 kB/s | 48 kB 00:00 2023-07-06T20:06:47.890 INFO:teuthology.orchestra.run.smithi098.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 473 kB/s | 87 kB 00:00 2023-07-06T20:06:47.940 INFO:teuthology.orchestra.run.smithi098.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 284 kB/s | 43 kB 00:00 2023-07-06T20:06:48.058 INFO:teuthology.orchestra.run.smithi098.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 654 kB/s | 109 kB 00:00 2023-07-06T20:06:48.191 INFO:teuthology.orchestra.run.smithi098.stdout:(79/131): thrift-0.13.0-2.el8.x86_64.rpm 1.8 MB/s | 1.7 MB 00:00 2023-07-06T20:06:48.208 INFO:teuthology.orchestra.run.smithi098.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 856 kB/s | 229 kB 00:00 2023-07-06T20:06:48.234 INFO:teuthology.orchestra.run.smithi098.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.1 MB/s | 560 kB 00:00 2023-07-06T20:06:48.377 INFO:teuthology.orchestra.run.smithi098.stdout:(82/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 102 kB/s | 19 kB 00:00 2023-07-06T20:06:48.434 INFO:teuthology.orchestra.run.smithi098.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 687 kB/s | 155 kB 00:00 2023-07-06T20:06:48.485 INFO:teuthology.orchestra.run.smithi098.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 164 kB/s | 18 kB 00:00 2023-07-06T20:06:48.518 INFO:teuthology.orchestra.run.smithi098.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 2.0 MB/s | 559 kB 00:00 2023-07-06T20:06:48.569 INFO:teuthology.orchestra.run.smithi098.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-07-06T20:06:48.636 INFO:teuthology.orchestra.run.smithi098.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2023-07-06T20:06:48.719 INFO:teuthology.orchestra.run.smithi098.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 12 MB/s | 2.3 MB 00:00 2023-07-06T20:06:48.745 INFO:teuthology.orchestra.run.smithi098.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 762 kB/s | 83 kB 00:00 2023-07-06T20:06:48.803 INFO:teuthology.orchestra.run.smithi098.stdout:(90/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 308 kB/s | 72 kB 00:00 2023-07-06T20:06:48.870 INFO:teuthology.orchestra.run.smithi098.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 398 kB/s | 60 kB 00:00 2023-07-06T20:06:48.904 INFO:teuthology.orchestra.run.smithi098.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 617 kB/s | 98 kB 00:00 2023-07-06T20:06:49.028 INFO:teuthology.orchestra.run.smithi098.stdout:(93/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 257 kB/s | 58 kB 00:00 2023-07-06T20:06:49.063 INFO:teuthology.orchestra.run.smithi098.stdout:(94/131): libcom_err-devel-1.45.6-4.el8.x86_64. 201 kB/s | 39 kB 00:00 2023-07-06T20:06:49.188 INFO:teuthology.orchestra.run.smithi098.stdout:(95/131): NetworkManager-initscripts-updown-1.3 1.1 MB/s | 138 kB 00:00 2023-07-06T20:06:49.222 INFO:teuthology.orchestra.run.smithi098.stdout:(96/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 2.0 MB/s | 644 kB 00:00 2023-07-06T20:06:49.414 INFO:teuthology.orchestra.run.smithi098.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 574 kB/s | 110 kB 00:00 2023-07-06T20:06:49.430 INFO:teuthology.orchestra.run.smithi098.stdout:(98/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 425 kB/s | 171 kB 00:00 2023-07-06T20:06:49.606 INFO:teuthology.orchestra.run.smithi098.stdout:(99/131): python3-scipy-1.0.0-21.module+el8.5.0 33 MB/s | 14 MB 00:00 2023-07-06T20:06:49.632 INFO:teuthology.orchestra.run.smithi098.stdout:(100/131): python3-pytz-2017.2-9.el8.noarch.rpm 249 kB/s | 54 kB 00:00 2023-07-06T20:06:49.656 INFO:teuthology.orchestra.run.smithi098.stdout:(101/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.1 MB/s | 259 kB 00:00 2023-07-06T20:06:49.774 INFO:teuthology.orchestra.run.smithi098.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 273 kB/s | 39 kB 00:00 2023-07-06T20:06:49.807 INFO:teuthology.orchestra.run.smithi098.stdout:(103/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2023-07-06T20:06:49.841 INFO:teuthology.orchestra.run.smithi098.stdout:(104/131): bison-3.0.4-10.el8.x86_64.rpm 3.7 MB/s | 688 kB 00:00 2023-07-06T20:06:49.908 INFO:teuthology.orchestra.run.smithi098.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.5 MB/s | 157 kB 00:00 2023-07-06T20:06:49.941 INFO:teuthology.orchestra.run.smithi098.stdout:(106/131): python3-jinja2-2.10.1-3.el8.noarch.r 3.1 MB/s | 538 kB 00:00 2023-07-06T20:06:50.038 INFO:teuthology.orchestra.run.smithi098.stdout:(107/131): oniguruma-6.8.2-2.el8.x86_64.rpm 952 kB/s | 187 kB 00:00 2023-07-06T20:06:50.084 INFO:teuthology.orchestra.run.smithi098.stdout:(108/131): automake-1.16.1-7.el8.noarch.rpm 4.0 MB/s | 713 kB 00:00 2023-07-06T20:06:50.209 INFO:teuthology.orchestra.run.smithi098.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-07-06T20:06:50.234 INFO:teuthology.orchestra.run.smithi098.stdout:(110/131): python3-jsonpatch-1.21-2.el8.noarch. 135 kB/s | 27 kB 00:00 2023-07-06T20:06:50.259 INFO:teuthology.orchestra.run.smithi098.stdout:(111/131): flex-2.6.1-9.el8.x86_64.rpm 1.8 MB/s | 320 kB 00:00 2023-07-06T20:06:50.318 INFO:teuthology.orchestra.run.smithi098.stdout:(112/131): python3-prettytable-0.7.2-14.el8.noa 404 kB/s | 44 kB 00:00 2023-07-06T20:06:50.477 INFO:teuthology.orchestra.run.smithi098.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-07-06T20:06:50.519 INFO:teuthology.orchestra.run.smithi098.stdout:(114/131): python3-werkzeug-0.12.2-4.el8.noarch 1.7 MB/s | 457 kB 00:00 2023-07-06T20:06:50.544 INFO:teuthology.orchestra.run.smithi098.stdout:(115/131): python3-numpy-f2py-1.14.3-10.el8.x86 996 kB/s | 225 kB 00:00 2023-07-06T20:06:50.611 INFO:teuthology.orchestra.run.smithi098.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 478 kB/s | 64 kB 00:00 2023-07-06T20:06:50.717 INFO:teuthology.orchestra.run.smithi098.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 116 kB/s | 20 kB 00:00 2023-07-06T20:06:50.753 INFO:teuthology.orchestra.run.smithi098.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 729 kB/s | 103 kB 00:00 2023-07-06T20:06:50.779 INFO:teuthology.orchestra.run.smithi098.stdout:(119/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 487 kB/s | 126 kB 00:00 2023-07-06T20:06:50.837 INFO:teuthology.orchestra.run.smithi098.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 5.8 MB/s | 709 kB 00:00 2023-07-06T20:06:50.946 INFO:teuthology.orchestra.run.smithi098.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 25 MB/s | 4.9 MB 00:00 2023-07-06T20:06:50.972 INFO:teuthology.orchestra.run.smithi098.stdout:(122/131): libpq-13.5-1.el8.x86_64.rpm 1.0 MB/s | 198 kB 00:00 2023-07-06T20:06:51.065 INFO:teuthology.orchestra.run.smithi098.stdout:(123/131): jq-1.6-3.el8.x86_64.rpm 1.7 MB/s | 202 kB 00:00 2023-07-06T20:06:51.180 INFO:teuthology.orchestra.run.smithi098.stdout:(124/131): openblas-0.3.15-3.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2023-07-06T20:06:51.206 INFO:teuthology.orchestra.run.smithi098.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.5 MB/s | 354 kB 00:00 2023-07-06T20:06:51.281 INFO:teuthology.orchestra.run.smithi098.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.9 MB/s | 808 kB 00:00 2023-07-06T20:06:51.365 INFO:teuthology.orchestra.run.smithi098.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 9.5 MB/s | 1.5 MB 00:00 2023-07-06T20:06:51.457 INFO:teuthology.orchestra.run.smithi098.stdout:(128/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-07-06T20:06:51.665 INFO:teuthology.orchestra.run.smithi098.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 3.8 MB/s | 798 kB 00:00 2023-07-06T20:06:52.832 INFO:teuthology.orchestra.run.smithi098.stdout:(130/131): librados2-18.0.0-4742.g79365bb3.el8. 2.4 MB/s | 3.7 MB 00:01 2023-07-06T20:06:54.049 INFO:teuthology.orchestra.run.smithi098.stdout:(131/131): librbd1-18.0.0-4742.g79365bb3.el8.x8 1.3 MB/s | 3.6 MB 00:02 2023-07-06T20:06:54.053 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2023-07-06T20:06:54.053 INFO:teuthology.orchestra.run.smithi098.stdout:Total 1.3 MB/s | 205 MB 02:35 2023-07-06T20:06:54.836 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2023-07-06T20:06:54.957 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2023-07-06T20:06:54.958 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2023-07-06T20:06:56.093 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2023-07-06T20:06:56.097 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2023-07-06T20:07:01.320 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2023-07-06T20:07:01.534 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-06T20:07:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-06T20:07:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-06T20:07:02.066 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-06T20:07:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-06T20:07:02.735 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-06T20:07:02.944 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 5/134 2023-07-06T20:07:03.378 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 5/134 2023-07-06T20:07:03.577 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T20:07:03.699 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T20:07:03.910 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T20:07:04.004 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T20:07:04.215 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T20:07:04.438 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-06T20:07:04.672 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-06T20:07:04.905 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-06T20:07:05.346 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-06T20:07:05.622 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-06T20:07:05.827 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T20:07:05.927 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T20:07:06.085 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T20:07:06.261 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T20:07:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-06T20:07:06.732 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-06T20:07:06.960 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-06T20:07:07.118 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-06T20:07:07.294 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-06T20:07:07.460 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-06T20:07:08.109 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-06T20:07:08.327 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-06T20:07:08.918 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-06T20:07:09.118 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-06T20:07:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-06T20:07:10.141 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-06T20:07:10.391 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-06T20:07:10.619 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-06T20:07:10.824 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 26/134 2023-07-06T20:07:11.021 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 27/134 2023-07-06T20:07:11.248 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 28/134 2023-07-06T20:07:11.484 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-06T20:07:13.226 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-06T20:07:14.261 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-06T20:07:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-06T20:07:14.920 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-06T20:07:15.210 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-06T20:07:15.437 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-06T20:07:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-06T20:07:15.873 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 36/134 2023-07-06T20:07:16.040 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 37/134 2023-07-06T20:07:16.249 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-06T20:07:16.418 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-06T20:07:16.634 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-06T20:07:16.698 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-06T20:07:16.970 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-06T20:07:17.245 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-06T20:07:17.392 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-06T20:07:17.468 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-06T20:07:17.737 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-06T20:07:17.954 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-06T20:07:18.116 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-06T20:07:18.321 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-06T20:07:18.486 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-06T20:07:18.678 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-06T20:07:18.888 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-06T20:07:19.130 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-06T20:07:19.535 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-06T20:07:19.902 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-06T20:07:20.183 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-06T20:07:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-06T20:07:20.748 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-06T20:07:21.063 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-06T20:07:21.291 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-06T20:07:21.535 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-06T20:07:21.813 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-06T20:07:21.995 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-06T20:07:22.211 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-06T20:07:22.547 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-06T20:07:22.845 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-06T20:07:23.129 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-06T20:07:23.404 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-06T20:07:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-06T20:07:23.819 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-06T20:07:24.100 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-06T20:07:24.348 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-06T20:07:24.537 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-06T20:07:24.677 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-06T20:07:24.925 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-06T20:07:25.212 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-06T20:07:25.411 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-06T20:07:25.644 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-06T20:07:25.847 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-06T20:07:26.072 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-06T20:07:26.345 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-06T20:07:26.580 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-06T20:07:26.822 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-06T20:07:27.090 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-06T20:07:27.308 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-06T20:07:27.541 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-06T20:07:28.670 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-06T20:07:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-06T20:07:29.315 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-06T20:07:29.633 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-06T20:07:29.907 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-06T20:07:30.162 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-06T20:07:30.995 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-06T20:07:31.165 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T20:07:31.237 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T20:07:31.449 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 93/134 2023-07-06T20:07:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T20:07:36.025 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T20:07:36.616 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T20:07:36.916 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T20:07:37.011 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T20:07:37.181 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T20:07:56.001 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T20:07:56.001 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2023-07-06T20:07:56.001 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2023-07-06T20:07:56.001 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /dev 2023-07-06T20:07:56.002 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /run 2023-07-06T20:07:56.002 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2023-07-06T20:07:56.002 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2023-07-06T20:07:56.002 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2023-07-06T20:07:56.002 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2023-07-06T20:07:56.002 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /dev 2023-07-06T20:07:56.002 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T20:07:56.387 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T20:07:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T20:07:58.186 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T20:07:58.402 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T20:07:58.629 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T20:07:59.080 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T20:07:59.253 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T20:07:59.270 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T20:08:00.955 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T20:08:01.188 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T20:08:01.334 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 102/134 2023-07-06T20:08:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 103/134 2023-07-06T20:08:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T20:08:02.008 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T20:08:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T20:08:03.227 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T20:08:03.403 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T20:08:03.492 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T20:08:03.913 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 107/134 2023-07-06T20:08:04.123 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T20:08:04.156 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T20:08:04.344 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T20:08:04.355 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T20:08:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 110/134 2023-07-06T20:08:05.842 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T20:08:05.875 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T20:08:06.069 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T20:08:06.110 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T20:08:06.269 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T20:08:06.613 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T20:08:06.819 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T20:08:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T20:08:12.225 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 115/134 2023-07-06T20:08:12.566 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-06T20:08:12.754 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 117/134 2023-07-06T20:08:13.048 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-06T20:08:13.243 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-06T20:08:13.438 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-06T20:08:13.716 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-06T20:08:13.914 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 121/134 2023-07-06T20:08:14.078 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T20:08:16.355 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T20:08:16.651 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T20:08:16.668 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T20:08:16.932 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 124/134 2023-07-06T20:08:17.183 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 125/134 2023-07-06T20:08:17.531 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 126/134 2023-07-06T20:08:17.933 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-06T20:08:18.167 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-06T20:08:18.246 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-06T20:08:18.459 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-06T20:08:18.471 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-06T20:08:18.626 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-06T20:08:18.633 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-06T20:08:18.819 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-06T20:08:18.820 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T20:08:19.020 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T20:08:19.021 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T20:08:19.298 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T20:08:19.550 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T20:08:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T20:08:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 1/134 2023-07-06T20:08:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2/134 2023-07-06T20:08:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 3/134 2023-07-06T20:08:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 4/134 2023-07-06T20:08:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4742.g79365 5/134 2023-07-06T20:08:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T20:08:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T20:08:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T20:08:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 9/134 2023-07-06T20:08:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 10/134 2023-07-06T20:08:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 11/134 2023-07-06T20:08:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 12/134 2023-07-06T20:08:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 13/134 2023-07-06T20:08:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 14/134 2023-07-06T20:08:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 15/134 2023-07-06T20:08:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 16/134 2023-07-06T20:08:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 17/134 2023-07-06T20:08:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 18/134 2023-07-06T20:08:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 19/134 2023-07-06T20:08:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 20/134 2023-07-06T20:08:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 21/134 2023-07-06T20:08:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 22/134 2023-07-06T20:08:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 23/134 2023-07-06T20:08:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 24/134 2023-07-06T20:08:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 25/134 2023-07-06T20:08:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 26/134 2023-07-06T20:08:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 27/134 2023-07-06T20:08:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 28/134 2023-07-06T20:08:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 29/134 2023-07-06T20:08:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 30/134 2023-07-06T20:08:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 31/134 2023-07-06T20:08:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 32/134 2023-07-06T20:08:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 33/134 2023-07-06T20:08:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 34/134 2023-07-06T20:08:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 35/134 2023-07-06T20:08:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 36/134 2023-07-06T20:08:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 37/134 2023-07-06T20:08:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 38/134 2023-07-06T20:08:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-06T20:08:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-06T20:08:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-06T20:08:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-06T20:08:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-06T20:08:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-06T20:08:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-06T20:08:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-06T20:08:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-06T20:08:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-06T20:08:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-06T20:08:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-06T20:08:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-06T20:08:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-06T20:08:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-06T20:08:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-06T20:08:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-06T20:08:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-06T20:08:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-06T20:08:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-06T20:08:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-06T20:08:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-06T20:08:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-06T20:08:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-06T20:08:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-06T20:08:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-06T20:08:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-06T20:08:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-06T20:08:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-06T20:08:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-06T20:08:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-06T20:08:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-06T20:08:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-06T20:08:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-06T20:08:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-06T20:08:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-06T20:08:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-06T20:08:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-06T20:08:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-06T20:08:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-06T20:08:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-06T20:08:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-06T20:08:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-06T20:08:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-06T20:08:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-06T20:08:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-06T20:08:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-06T20:08:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-06T20:08:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-06T20:08:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-06T20:08:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-06T20:08:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-06T20:08:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-06T20:08:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-06T20:08:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-06T20:08:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-06T20:08:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-06T20:08:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-06T20:08:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-06T20:08:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-06T20:08:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-06T20:08:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-06T20:08:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-06T20:08:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-06T20:08:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-06T20:08:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-06T20:08:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-06T20:08:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-06T20:08:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-06T20:08:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-06T20:08:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-06T20:08:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-06T20:08:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-06T20:08:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-06T20:08:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-06T20:08:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-06T20:08:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-06T20:08:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-06T20:08:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-06T20:08:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-06T20:08:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-06T20:08:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-06T20:08:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-06T20:08:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-06T20:08:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-06T20:08:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-06T20:08:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-06T20:08:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-06T20:08:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-06T20:08:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-06T20:08:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 129/134 2023-07-06T20:08:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-06T20:08:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 131/134 2023-07-06T20:08:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T20:08:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-06T20:08:22.699 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T20:08:22.699 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:04:15 ago on Thu 06 Jul 2023 08:04:07 PM UTC. 2023-07-06T20:08:33.660 INFO:teuthology.orchestra.run.smithi098.stdout:Installed products updated. 2023-07-06T20:08:33.874 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T20:08:33.874 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2023-07-06T20:08:33.874 INFO:teuthology.orchestra.run.smithi098.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-06T20:08:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.875 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2023-07-06T20:08:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-06T20:08:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T20:08:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T20:08:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.876 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.876 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:08:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:08:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:08:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:08:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:08:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:08:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:08:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:08:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:08:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T20:08:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-06T20:08:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T20:08:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-06T20:08:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T20:08:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: jq-1.6-3.el8.x86_64 2023-07-06T20:08:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T20:08:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-06T20:08:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T20:08:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-06T20:08:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-06T20:08:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T20:08:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T20:08:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-06T20:08:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-06T20:08:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T20:08:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-06T20:08:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T20:08:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T20:08:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T20:08:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-06T20:08:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-06T20:08:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T20:08:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T20:08:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-06T20:08:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-06T20:08:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-06T20:08:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-06T20:08:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-06T20:08:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-06T20:08:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T20:08:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T20:08:33.886 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T20:08:33.886 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T20:08:33.886 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T20:08:33.886 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T20:08:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T20:08:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T20:08:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T20:08:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-06T20:08:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T20:08:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T20:08:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-06T20:08:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T20:08:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T20:08:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T20:08:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T20:08:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T20:08:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T20:08:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T20:08:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T20:08:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T20:08:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-06T20:08:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T20:08:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T20:08:33.890 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T20:08:33.890 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T20:08:33.890 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T20:08:33.890 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T20:08:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T20:08:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T20:08:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T20:08:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T20:08:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T20:08:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T20:08:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T20:08:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T20:08:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T20:08:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T20:08:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T20:08:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T20:08:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-06T20:08:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T20:08:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T20:08:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T20:08:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T20:08:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T20:08:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T20:08:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T20:08:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T20:08:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T20:08:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T20:08:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T20:08:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T20:08:33.895 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T20:08:33.895 INFO:teuthology.orchestra.run.smithi098.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T20:08:33.895 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-06T20:08:33.895 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T20:08:33.896 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-06T20:08:33.896 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-06T20:08:33.896 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2023-07-06T20:08:34.064 DEBUG:teuthology.parallel:result is None 2023-07-06T20:08:34.065 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:08:34.203 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T20:08:34.262 INFO:teuthology.orchestra.run.smithi069.stdout:18.0.0-4742.g79365bb3.el8 2023-07-06T20:08:34.263 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4742.g79365bb3.el8 2023-07-06T20:08:34.263 INFO:teuthology.task.install:The correct ceph version 18.0.0-4742.g79365bb3 is installed. 2023-07-06T20:08:34.264 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:08:34.400 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T20:08:34.424 INFO:teuthology.orchestra.run.smithi098.stdout:18.0.0-4742.g79365bb3.el8 2023-07-06T20:08:34.424 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4742.g79365bb3.el8 2023-07-06T20:08:34.424 INFO:teuthology.task.install:The correct ceph version 18.0.0-4742.g79365bb3 is installed. 2023-07-06T20:08:34.425 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:08:34.778 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T20:08:34.808 INFO:teuthology.orchestra.run.smithi156.stdout:18.0.0-4742.g79365bb3.el8 2023-07-06T20:08:34.808 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4742.g79365bb3.el8 2023-07-06T20:08:34.808 INFO:teuthology.task.install:The correct ceph version 18.0.0-4742.g79365bb3 is installed. 2023-07-06T20:08:34.809 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-06T20:08:34.809 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:08:34.810 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T20:08:34.905 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-07-06T20:08:34.906 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T20:08:34.952 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-06T20:08:34.953 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T20:08:34.999 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-06T20:08:34.999 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:08:35.000 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T20:08:35.069 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T20:08:35.174 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-07-06T20:08:35.175 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T20:08:35.208 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T20:08:35.277 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-06T20:08:35.278 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T20:08:35.311 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T20:08:35.381 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-06T20:08:35.382 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:08:35.382 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T20:08:35.449 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T20:08:35.554 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-07-06T20:08:35.555 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T20:08:35.586 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T20:08:35.656 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-06T20:08:35.656 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T20:08:35.688 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T20:08:35.759 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-06T20:08:35.759 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:08:35.760 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T20:08:35.826 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T20:08:35.932 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-07-06T20:08:35.933 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T20:08:35.964 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T20:08:36.034 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-06T20:08:36.035 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T20:08:36.066 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T20:08:36.136 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-06T20:08:36.239 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-07-06T20:08:36.240 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:08:36.250 INFO:tasks.cephadm:Cluster fsid is e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:08:36.251 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-06T20:08:36.251 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.69', 'mon.b': '172.21.15.98', 'mon.c': '172.21.15.156'} 2023-07-06T20:08:36.252 INFO:tasks.cephadm:First mon is mon.a on smithi069 2023-07-06T20:08:36.252 INFO:tasks.cephadm:First mgr is x 2023-07-06T20:08:36.252 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-06T20:08:36.252 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname $(hostname -s) 2023-07-06T20:08:36.323 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname $(hostname -s) 2023-07-06T20:08:36.354 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname $(hostname -s) 2023-07-06T20:08:36.385 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-06T20:08:36.386 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:08:36.528 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71081/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4742-g79365bb3', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4742.g79365bb3'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-05 20:47:57.573203', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Jul5', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-07-06T20:08:36.668 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-rishabh-2023Jul5, sha1 79365bb3b803780276d69efb2cc4afe5b160de9e from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T20:08:36.669 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/x86_64/flavors/default/cephadm 2023-07-06T20:08:36.669 DEBUG:teuthology.orchestra.run.smithi069:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-06T20:08:37.099 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 6 20:08 /home/ubuntu/cephtest/cephadm 2023-07-06T20:08:37.100 DEBUG:teuthology.orchestra.run.smithi098:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-06T20:08:37.544 INFO:teuthology.orchestra.run.smithi098.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 6 20:08 /home/ubuntu/cephtest/cephadm 2023-07-06T20:08:37.544 DEBUG:teuthology.orchestra.run.smithi156:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-06T20:08:38.296 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 6 20:08 /home/ubuntu/cephtest/cephadm 2023-07-06T20:08:38.297 DEBUG:teuthology.orchestra.run.smithi069:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-06T20:08:38.347 DEBUG:teuthology.orchestra.run.smithi098:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-06T20:08:38.369 DEBUG:teuthology.orchestra.run.smithi156:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-06T20:08:38.404 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e on all hosts... 2023-07-06T20:08:38.404 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e pull 2023-07-06T20:08:38.409 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e pull 2023-07-06T20:08:38.411 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e pull 2023-07-06T20:08:39.681 INFO:teuthology.orchestra.run.smithi156.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e... 2023-07-06T20:08:39.809 INFO:teuthology.orchestra.run.smithi069.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e... 2023-07-06T20:08:42.067 INFO:teuthology.orchestra.run.smithi098.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e... 2023-07-06T20:09:05.159 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-07-06T20:09:05.160 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T20:09:05.160 INFO:teuthology.orchestra.run.smithi156.stdout: "image_id": "d8383422d5366e195610b0e81cb1ffd2bf2ab86f12ae1865a8c4845ec54e441e", 2023-07-06T20:09:05.160 INFO:teuthology.orchestra.run.smithi156.stdout: "repo_digests": [ 2023-07-06T20:09:05.160 INFO:teuthology.orchestra.run.smithi156.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4e3032bedf9b139cce9fcfda7c8eb085561dde3fd4e7fbc189c7bdfb738dd2e8" 2023-07-06T20:09:05.160 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2023-07-06T20:09:05.161 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-07-06T20:09:46.595 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-07-06T20:09:46.595 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T20:09:46.595 INFO:teuthology.orchestra.run.smithi098.stdout: "image_id": "d8383422d5366e195610b0e81cb1ffd2bf2ab86f12ae1865a8c4845ec54e441e", 2023-07-06T20:09:46.595 INFO:teuthology.orchestra.run.smithi098.stdout: "repo_digests": [ 2023-07-06T20:09:46.595 INFO:teuthology.orchestra.run.smithi098.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4e3032bedf9b139cce9fcfda7c8eb085561dde3fd4e7fbc189c7bdfb738dd2e8" 2023-07-06T20:09:46.596 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2023-07-06T20:09:46.596 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-07-06T20:09:47.051 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:09:47.051 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T20:09:47.051 INFO:teuthology.orchestra.run.smithi069.stdout: "image_id": "d8383422d5366e195610b0e81cb1ffd2bf2ab86f12ae1865a8c4845ec54e441e", 2023-07-06T20:09:47.051 INFO:teuthology.orchestra.run.smithi069.stdout: "repo_digests": [ 2023-07-06T20:09:47.051 INFO:teuthology.orchestra.run.smithi069.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4e3032bedf9b139cce9fcfda7c8eb085561dde3fd4e7fbc189c7bdfb738dd2e8" 2023-07-06T20:09:47.052 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2023-07-06T20:09:47.052 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:09:47.078 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph 2023-07-06T20:09:47.150 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph 2023-07-06T20:09:47.183 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph 2023-07-06T20:09:47.221 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /etc/ceph 2023-07-06T20:09:47.288 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /etc/ceph 2023-07-06T20:09:47.316 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /etc/ceph 2023-07-06T20:09:47.347 INFO:tasks.cephadm:Writing seed config... 2023-07-06T20:09:47.349 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-06T20:09:47.349 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-06T20:09:47.350 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-06T20:09:47.350 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-06T20:09:47.350 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-06T20:09:47.351 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-07-06T20:09:47.351 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-06T20:09:47.352 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-06T20:09:47.352 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-06T20:09:47.352 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-06T20:09:47.352 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-06T20:09:47.352 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-06T20:09:47.353 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-06T20:09:47.353 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-06T20:09:47.353 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-06T20:09:47.353 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-06T20:09:47.354 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-06T20:09:47.354 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-07-06T20:09:47.354 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-06T20:09:47.354 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-06T20:09:47.354 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-06T20:09:47.355 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-06T20:09:47.355 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-06T20:09:47.355 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-06T20:09:47.355 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-06T20:09:47.355 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-06T20:09:47.356 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-06T20:09:47.356 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-06T20:09:47.356 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-07-06T20:09:47.356 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-07-06T20:09:47.357 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-07-06T20:09:47.357 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-06T20:09:47.357 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-06T20:09:47.357 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-07-06T20:09:47.357 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-07-06T20:09:47.358 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-06T20:09:47.358 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-06T20:09:47.358 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-07-06T20:09:47.358 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-06T20:09:47.359 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-06T20:09:47.359 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-06T20:09:47.359 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-06T20:09:47.359 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-06T20:09:47.360 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-06T20:09:47.360 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-06T20:09:47.360 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-07-06T20:09:47.361 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:09:47.361 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-06T20:09:47.404 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e3e8ec0c-1c38-11ee-9b2f-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-06T20:09:47.405 DEBUG:teuthology.orchestra.run.smithi069:mon.a> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.a.service 2023-07-06T20:09:47.448 DEBUG:teuthology.orchestra.run.smithi069:mgr.x> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.x.service 2023-07-06T20:09:47.493 INFO:tasks.cephadm:Bootstrapping... 2023-07-06T20:09:47.494 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e -v bootstrap --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.69 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-06T20:09:47.514 INFO:journalctl@ceph.mon.a.smithi069.stdout:-- Logs begin at Thu 2023-07-06 19:46:03 UTC. -- 2023-07-06T20:09:47.562 INFO:journalctl@ceph.mgr.x.smithi069.stdout:-- Logs begin at Thu 2023-07-06 19:46:03 UTC. -- 2023-07-06T20:09:47.714 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2023-07-06T20:09:47.715 INFO:teuthology.orchestra.run.smithi069.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e', '-v', 'bootstrap', '--fsid', 'e3e8ec0c-1c38-11ee-9b2f-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.69', '--skip-admin-label'] 2023-07-06T20:09:47.888 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 4.1.1 2023-07-06T20:09:47.889 INFO:teuthology.orchestra.run.smithi069.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-06T20:09:47.889 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying podman|docker is present... 2023-07-06T20:09:48.059 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 4.1.1 2023-07-06T20:09:48.060 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying lvm2 is present... 2023-07-06T20:09:48.060 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying time synchronization is in place... 2023-07-06T20:09:48.074 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-06T20:09:48.074 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-06T20:09:48.086 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-06T20:09:48.086 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2023-07-06T20:09:48.099 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout enabled 2023-07-06T20:09:48.112 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout active 2023-07-06T20:09:48.112 INFO:teuthology.orchestra.run.smithi069.stdout:Unit chronyd.service is enabled and running 2023-07-06T20:09:48.112 INFO:teuthology.orchestra.run.smithi069.stdout:Repeating the final host check... 2023-07-06T20:09:48.281 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 4.1.1 2023-07-06T20:09:48.281 INFO:teuthology.orchestra.run.smithi069.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-06T20:09:48.281 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl is present 2023-07-06T20:09:48.281 INFO:teuthology.orchestra.run.smithi069.stdout:lvcreate is present 2023-07-06T20:09:48.293 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-06T20:09:48.294 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-06T20:09:48.306 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-06T20:09:48.306 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2023-07-06T20:09:48.319 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout enabled 2023-07-06T20:09:48.332 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout active 2023-07-06T20:09:48.332 INFO:teuthology.orchestra.run.smithi069.stdout:Unit chronyd.service is enabled and running 2023-07-06T20:09:48.332 INFO:teuthology.orchestra.run.smithi069.stdout:Host looks OK 2023-07-06T20:09:48.332 INFO:teuthology.orchestra.run.smithi069.stdout:Cluster fsid: e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:09:48.333 INFO:teuthology.orchestra.run.smithi069.stdout:Acquiring lock 139797770041832 on /run/cephadm/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c.lock 2023-07-06T20:09:48.333 INFO:teuthology.orchestra.run.smithi069.stdout:Lock 139797770041832 acquired on /run/cephadm/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c.lock 2023-07-06T20:09:48.333 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying IP 172.21.15.69 port 3300 ... 2023-07-06T20:09:48.333 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying IP 172.21.15.69 port 6789 ... 2023-07-06T20:09:48.334 INFO:teuthology.orchestra.run.smithi069.stdout:Base mon IP(s) is [172.21.15.69:3300, 172.21.15.69:6789], mon addrv is [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2023-07-06T20:09:48.341 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-07-06T20:09:48.341 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.69 metric 100 2023-07-06T20:09:48.348 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-06T20:09:48.348 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-07-06T20:09:48.348 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-07-06T20:09:48.354 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-06T20:09:48.354 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-06T20:09:48.354 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-06T20:09:48.355 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-07-06T20:09:48.355 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout inet6 fe80::9c26:d5af:e804:49a4/64 scope link noprefixroute 2023-07-06T20:09:48.355 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-06T20:09:48.355 INFO:teuthology.orchestra.run.smithi069.stdout:Mon IP `172.21.15.69` is in CIDR network `172.21.0.0/20` 2023-07-06T20:09:48.356 INFO:teuthology.orchestra.run.smithi069.stdout:Mon IP `172.21.15.69` is in CIDR network `172.21.0.0/20` 2023-07-06T20:09:48.356 INFO:teuthology.orchestra.run.smithi069.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-06T20:09:48.356 INFO:teuthology.orchestra.run.smithi069.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-06T20:09:48.357 INFO:teuthology.orchestra.run.smithi069.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e... 2023-07-06T20:09:48.930 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout d8383422d5366e195610b0e81cb1ffd2bf2ab86f12ae1865a8c4845ec54e441e 2023-07-06T20:09:48.931 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e... 2023-07-06T20:09:48.931 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Getting image source signatures 2023-07-06T20:09:48.931 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying blob sha256:44d1623e31d57d9d5d27727eb0f9f254b284564b0aad84d1cc5b192b3aa4a232 2023-07-06T20:09:48.931 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying blob sha256:f00ae3d4c98d38209fd572dc4ae396d2ed41ebdd4e42e81d94310d564cff29ab 2023-07-06T20:09:48.931 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying config sha256:d8383422d5366e195610b0e81cb1ffd2bf2ab86f12ae1865a8c4845ec54e441e 2023-07-06T20:09:48.931 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-06T20:09:48.931 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Storing signatures 2023-07-06T20:09:49.802 INFO:teuthology.orchestra.run.smithi069.stdout:ceph: stdout ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T20:09:49.803 INFO:teuthology.orchestra.run.smithi069.stdout:Ceph version: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T20:09:49.803 INFO:teuthology.orchestra.run.smithi069.stdout:Extracting ceph user uid/gid from container image... 2023-07-06T20:09:50.579 INFO:teuthology.orchestra.run.smithi069.stdout:stat: stdout 167 167 2023-07-06T20:09:50.579 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial keys... 2023-07-06T20:09:51.352 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQCOH6dkWTgyOhAAGk09c2aCLkIaQieyNZvCaA== 2023-07-06T20:09:52.127 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQCPH6dknME0LRAAchKAIRKJ7tYwtzZVoXoVmg== 2023-07-06T20:09:52.911 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQCQH6dkgI9CIBAAd61y4ybS7fZmcqqb+KZ1Sw== 2023-07-06T20:09:52.912 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial monmap... 2023-07-06T20:09:53.684 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-06T20:09:53.685 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-06T20:09:53.685 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:09:53.685 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-06T20:09:53.686 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool for a [v2:172.21.15.69:3300,v1:172.21.15.69:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-06T20:09:53.686 INFO:teuthology.orchestra.run.smithi069.stdout:setting min_mon_release = pacific 2023-07-06T20:09:53.686 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: set fsid to e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:09:53.686 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-06T20:09:53.686 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:09:53.686 INFO:teuthology.orchestra.run.smithi069.stdout:Creating mon... 2023-07-06T20:09:54.566 INFO:teuthology.orchestra.run.smithi069.stdout:create mon.a on 2023-07-06T20:09:54.807 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-06T20:09:55.005 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-06T20:09:55.239 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c.target → /etc/systemd/system/ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c.target. 2023-07-06T20:09:55.240 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c.target → /etc/systemd/system/ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c.target. 2023-07-06T20:09:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.a 2023-07-06T20:09:55.567 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.a.service: Unit ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.a.service not loaded. 2023-07-06T20:09:55.760 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c.target.wants/ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@.service. 2023-07-06T20:09:56.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:55 smithi069 systemd[1]: Starting Ceph mon.a for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:09:56.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:56 smithi069 podman[125852]: 2023-07-06T20:09:56.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:56 smithi069 bash[125852]: b7dbc38345b716cc118df17404e6a34274868a37641e247f311b0aa5bdbcdf4e 2023-07-06T20:09:56.569 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-06T20:09:56.569 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2023-07-06T20:09:56.585 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-06T20:09:56.585 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2023-07-06T20:09:56.585 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2023-07-06T20:09:56.585 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to enable service . firewalld.service is not available 2023-07-06T20:09:56.586 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mon to start... 2023-07-06T20:09:56.586 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mon... 2023-07-06T20:09:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:56 smithi069 ceph-mon[125892]: mkfs e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:09:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:56 smithi069 ceph-mon[125892]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-06T20:09:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:56 smithi069 systemd[1]: Started Ceph mon.a for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:09:57.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:57 smithi069 ceph-mon[125892]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-06T20:09:57.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:57 smithi069 ceph-mon[125892]: monmap e1: 1 mons at {a=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0]} removed_ranks: {} 2023-07-06T20:09:57.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:57 smithi069 ceph-mon[125892]: fsmap 2023-07-06T20:09:57.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:57 smithi069 ceph-mon[125892]: osdmap e1: 0 total, 0 up, 0 in 2023-07-06T20:09:57.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:57 smithi069 ceph-mon[125892]: mgrmap e1: no daemons active 2023-07-06T20:09:57.967 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout cluster: 2023-07-06T20:09:57.976 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout id: e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:09:57.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-06T20:09:57.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:09:57.977 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout services: 2023-07-06T20:09:57.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 1.08964s) 2023-07-06T20:09:57.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-06T20:09:57.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-06T20:09:57.978 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:09:57.979 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout data: 2023-07-06T20:09:57.979 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-06T20:09:57.979 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-06T20:09:57.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-06T20:09:57.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout pgs: 2023-07-06T20:09:57.980 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:09:57.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.512+0000 7fe9c68b6700 1 Processor -- start 2023-07-06T20:09:57.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.512+0000 7fe9c68b6700 1 -- start start 2023-07-06T20:09:57.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.512+0000 7fe9c68b6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 0x7fe9c0079160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:09:57.981 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.512+0000 7fe9c68b6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00796a0 con 0x7fe9c007ac60 2023-07-06T20:09:57.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.513+0000 7fe9bffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 0x7fe9c0079160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:09:57.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.513+0000 7fe9bffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 0x7fe9c0079160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56986/0 (socket says 172.21.15.69:56986) 2023-07-06T20:09:57.982 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.513+0000 7fe9bffff700 1 -- 172.21.15.69:0/109766910 learned_addr learned my addr 172.21.15.69:0/109766910 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:09:57.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.514+0000 7fe9bffff700 1 -- 172.21.15.69:0/109766910 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c00797e0 con 0x7fe9c007ac60 2023-07-06T20:09:57.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.514+0000 7fe9bffff700 1 --2- 172.21.15.69:0/109766910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 0x7fe9c0079160 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00098a0 tx=0x7fe9b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8cbb79f0a06bcf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:09:57.983 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.514+0000 7fe9beffd700 1 -- 172.21.15.69:0/109766910 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9b0004160 con 0x7fe9c007ac60 2023-07-06T20:09:57.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.514+0000 7fe9beffd700 1 -- 172.21.15.69:0/109766910 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b00042c0 con 0x7fe9c007ac60 2023-07-06T20:09:57.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.514+0000 7fe9beffd700 1 -- 172.21.15.69:0/109766910 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9b00045a0 con 0x7fe9c007ac60 2023-07-06T20:09:57.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.515+0000 7fe9c68b6700 1 -- 172.21.15.69:0/109766910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 msgr2=0x7fe9c0079160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:09:57.984 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.515+0000 7fe9c68b6700 1 --2- 172.21.15.69:0/109766910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 0x7fe9c0079160 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00098a0 tx=0x7fe9b0009bb0 comp rx=0 tx=0).stop 2023-07-06T20:09:57.985 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.515+0000 7fe9c68b6700 1 -- 172.21.15.69:0/109766910 shutdown_connections 2023-07-06T20:09:57.985 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.515+0000 7fe9c68b6700 1 --2- 172.21.15.69:0/109766910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 0x7fe9c0079160 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:09:57.985 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.515+0000 7fe9c68b6700 1 -- 172.21.15.69:0/109766910 >> 172.21.15.69:0/109766910 conn(0x7fe9c0100b90 msgr2=0x7fe9c0102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:09:57.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.515+0000 7fe9c68b6700 1 -- 172.21.15.69:0/109766910 shutdown_connections 2023-07-06T20:09:57.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.515+0000 7fe9c68b6700 1 -- 172.21.15.69:0/109766910 wait complete. 2023-07-06T20:09:57.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.516+0000 7fe9c68b6700 1 Processor -- start 2023-07-06T20:09:57.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.516+0000 7fe9c68b6700 1 -- start start 2023-07-06T20:09:57.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.516+0000 7fe9c68b6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 0x7fe9c019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:09:57.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.516+0000 7fe9c68b6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c019b240 con 0x7fe9c007ac60 2023-07-06T20:09:57.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.517+0000 7fe9bffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 0x7fe9c019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:09:57.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.517+0000 7fe9bffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 0x7fe9c019ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57002/0 (socket says 172.21.15.69:57002) 2023-07-06T20:09:57.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.517+0000 7fe9bffff700 1 -- 172.21.15.69:0/4168757308 learned_addr learned my addr 172.21.15.69:0/4168757308 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:09:57.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.517+0000 7fe9bffff700 1 -- 172.21.15.69:0/4168757308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b0009550 con 0x7fe9c007ac60 2023-07-06T20:09:57.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.517+0000 7fe9bffff700 1 --2- 172.21.15.69:0/4168757308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 0x7fe9c019ad00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe9c007a1d0 tx=0x7fe9b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:09:57.989 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.518+0000 7fe9bd7fa700 1 -- 172.21.15.69:0/4168757308 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9b0003da0 con 0x7fe9c007ac60 2023-07-06T20:09:57.989 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.518+0000 7fe9bd7fa700 1 -- 172.21.15.69:0/4168757308 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b0011630 con 0x7fe9c007ac60 2023-07-06T20:09:57.989 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.518+0000 7fe9bd7fa700 1 -- 172.21.15.69:0/4168757308 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9b0011910 con 0x7fe9c007ac60 2023-07-06T20:09:57.990 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.518+0000 7fe9c68b6700 1 -- 172.21.15.69:0/4168757308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c019b440 con 0x7fe9c007ac60 2023-07-06T20:09:57.990 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.518+0000 7fe9c68b6700 1 -- 172.21.15.69:0/4168757308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c019b860 con 0x7fe9c007ac60 2023-07-06T20:09:57.990 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.518+0000 7fe9bd7fa700 1 -- 172.21.15.69:0/4168757308 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe9b001c460 con 0x7fe9c007ac60 2023-07-06T20:09:57.990 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.518+0000 7fe9bd7fa700 1 -- 172.21.15.69:0/4168757308 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe9b001a790 con 0x7fe9c007ac60 2023-07-06T20:09:57.991 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.519+0000 7fe9c68b6700 1 -- 172.21.15.69:0/4168757308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c0194a90 con 0x7fe9c007ac60 2023-07-06T20:09:57.991 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.522+0000 7fe9bd7fa700 1 -- 172.21.15.69:0/4168757308 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fe9b001a950 con 0x7fe9c007ac60 2023-07-06T20:09:57.991 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.594+0000 7fe9c68b6700 1 -- 172.21.15.69:0/4168757308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fe9c0061250 con 0x7fe9c007ac60 2023-07-06T20:09:57.992 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.594+0000 7fe9bd7fa700 1 -- 172.21.15.69:0/4168757308 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+316 (secure 0 0 0) 0x7fe9b001aaf0 con 0x7fe9c007ac60 2023-07-06T20:09:57.992 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.595+0000 7fe9c68b6700 1 -- 172.21.15.69:0/4168757308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 msgr2=0x7fe9c019ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:09:57.992 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.595+0000 7fe9c68b6700 1 --2- 172.21.15.69:0/4168757308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 0x7fe9c019ad00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe9c007a1d0 tx=0x7fe9b0004750 comp rx=0 tx=0).stop 2023-07-06T20:09:57.993 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.595+0000 7fe9c68b6700 1 -- 172.21.15.69:0/4168757308 shutdown_connections 2023-07-06T20:09:57.993 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.595+0000 7fe9c68b6700 1 --2- 172.21.15.69:0/4168757308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9c007ac60 0x7fe9c019ad00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:09:57.993 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.595+0000 7fe9c68b6700 1 -- 172.21.15.69:0/4168757308 >> 172.21.15.69:0/4168757308 conn(0x7fe9c0100b90 msgr2=0x7fe9c0102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:09:57.993 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.595+0000 7fe9c68b6700 1 -- 172.21.15.69:0/4168757308 shutdown_connections 2023-07-06T20:09:57.994 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:57.595+0000 7fe9c68b6700 1 -- 172.21.15.69:0/4168757308 wait complete. 2023-07-06T20:09:57.994 INFO:teuthology.orchestra.run.smithi069.stdout:mon is available 2023-07-06T20:09:57.994 INFO:teuthology.orchestra.run.smithi069.stdout:Assimilating anything we can from ceph.conf... 2023-07-06T20:09:58.950 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:09:58.950 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [global] 2023-07-06T20:09:58.951 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout fsid = e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:09:58.951 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2023-07-06T20:09:58.951 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-06T20:09:58.951 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-06T20:09:58.951 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-06T20:09:58.951 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-06T20:09:58.951 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:09:58.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-06T20:09:58.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-06T20:09:58.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:09:58.952 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [osd] 2023-07-06T20:09:58.953 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-06T20:09:58.953 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-06T20:09:58.953 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-06T20:09:58.953 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-06T20:09:58.953 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-06T20:09:58.953 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.483+0000 7fafe8198700 1 Processor -- start 2023-07-06T20:09:58.953 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.483+0000 7fafe8198700 1 -- start start 2023-07-06T20:09:58.953 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.484+0000 7fafe8198700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 0x7fafe00791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:09:58.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.484+0000 7fafe8198700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe0079700 con 0x7fafe007acc0 2023-07-06T20:09:58.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.485+0000 7fafe5f34700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 0x7fafe00791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:09:58.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.485+0000 7fafe5f34700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 0x7fafe00791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57018/0 (socket says 172.21.15.69:57018) 2023-07-06T20:09:58.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.485+0000 7fafe5f34700 1 -- 172.21.15.69:0/3479740026 learned_addr learned my addr 172.21.15.69:0/3479740026 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:09:58.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.485+0000 7fafe5f34700 1 -- 172.21.15.69:0/3479740026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe0079840 con 0x7fafe007acc0 2023-07-06T20:09:58.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.485+0000 7fafe5f34700 1 --2- 172.21.15.69:0/3479740026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 0x7fafe00791c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0098a0 tx=0x7fafdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70e30628cdf1c85 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:09:58.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.485+0000 7fafe4f32700 1 -- 172.21.15.69:0/3479740026 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafdc004160 con 0x7fafe007acc0 2023-07-06T20:09:58.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.486+0000 7fafe4f32700 1 -- 172.21.15.69:0/3479740026 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc0042c0 con 0x7fafe007acc0 2023-07-06T20:09:58.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.486+0000 7fafe4f32700 1 -- 172.21.15.69:0/3479740026 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafdc0045a0 con 0x7fafe007acc0 2023-07-06T20:09:58.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.486+0000 7fafe8198700 1 -- 172.21.15.69:0/3479740026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 msgr2=0x7fafe00791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:09:58.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.486+0000 7fafe8198700 1 --2- 172.21.15.69:0/3479740026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 0x7fafe00791c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0098a0 tx=0x7fafdc009bb0 comp rx=0 tx=0).stop 2023-07-06T20:09:58.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.486+0000 7fafe8198700 1 -- 172.21.15.69:0/3479740026 shutdown_connections 2023-07-06T20:09:58.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.486+0000 7fafe8198700 1 --2- 172.21.15.69:0/3479740026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 0x7fafe00791c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:09:58.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.486+0000 7fafe8198700 1 -- 172.21.15.69:0/3479740026 >> 172.21.15.69:0/3479740026 conn(0x7fafe0100b90 msgr2=0x7fafe0102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:09:58.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.486+0000 7fafe8198700 1 -- 172.21.15.69:0/3479740026 shutdown_connections 2023-07-06T20:09:58.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.486+0000 7fafe8198700 1 -- 172.21.15.69:0/3479740026 wait complete. 2023-07-06T20:09:58.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.487+0000 7fafe8198700 1 Processor -- start 2023-07-06T20:09:58.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.487+0000 7fafe8198700 1 -- start start 2023-07-06T20:09:58.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.488+0000 7fafe8198700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 0x7fafe019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:09:58.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.488+0000 7fafe8198700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe019f660 con 0x7fafe007acc0 2023-07-06T20:09:58.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.488+0000 7fafe5f34700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 0x7fafe019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:09:58.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.488+0000 7fafe5f34700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 0x7fafe019f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57020/0 (socket says 172.21.15.69:57020) 2023-07-06T20:09:58.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.488+0000 7fafe5f34700 1 -- 172.21.15.69:0/3030095448 learned_addr learned my addr 172.21.15.69:0/3030095448 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:09:58.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.488+0000 7fafe5f34700 1 -- 172.21.15.69:0/3030095448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc009550 con 0x7fafe007acc0 2023-07-06T20:09:58.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.488+0000 7fafe5f34700 1 --2- 172.21.15.69:0/3030095448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 0x7fafe019f120 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fafdc004000 tx=0x7fafdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:09:58.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.489+0000 7fafd6ffd700 1 -- 172.21.15.69:0/3030095448 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafdc011630 con 0x7fafe007acc0 2023-07-06T20:09:58.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.489+0000 7fafd6ffd700 1 -- 172.21.15.69:0/3030095448 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc011790 con 0x7fafe007acc0 2023-07-06T20:09:58.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.489+0000 7fafe8198700 1 -- 172.21.15.69:0/3030095448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe019f860 con 0x7fafe007acc0 2023-07-06T20:09:58.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.489+0000 7fafd6ffd700 1 -- 172.21.15.69:0/3030095448 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafdc011a30 con 0x7fafe007acc0 2023-07-06T20:09:58.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.489+0000 7fafe8198700 1 -- 172.21.15.69:0/3030095448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe019fc80 con 0x7fafe007acc0 2023-07-06T20:09:58.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.489+0000 7fafd6ffd700 1 -- 172.21.15.69:0/3030095448 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fafdc022070 con 0x7fafe007acc0 2023-07-06T20:09:58.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.489+0000 7fafd6ffd700 1 -- 172.21.15.69:0/3030095448 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fafdc01c450 con 0x7fafe007acc0 2023-07-06T20:09:58.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.490+0000 7fafe8198700 1 -- 172.21.15.69:0/3030095448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe0199080 con 0x7fafe007acc0 2023-07-06T20:09:58.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.493+0000 7fafd6ffd700 1 -- 172.21.15.69:0/3030095448 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fafdc01b450 con 0x7fafe007acc0 2023-07-06T20:09:58.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.562+0000 7fafe8198700 1 -- 172.21.15.69:0/3030095448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fafe002cb60 con 0x7fafe007acc0 2023-07-06T20:09:58.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.569+0000 7fafd6ffd700 1 -- 172.21.15.69:0/3030095448 <== mon.0 v2:172.21.15.69:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fafdc01b5f0 con 0x7fafe007acc0 2023-07-06T20:09:58.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.569+0000 7fafd6ffd700 1 -- 172.21.15.69:0/3030095448 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fafdc0375f0 con 0x7fafe007acc0 2023-07-06T20:09:58.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.570+0000 7fafe8198700 1 -- 172.21.15.69:0/3030095448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 msgr2=0x7fafe019f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:09:58.959 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.570+0000 7fafe8198700 1 --2- 172.21.15.69:0/3030095448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 0x7fafe019f120 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fafdc004000 tx=0x7fafdc004750 comp rx=0 tx=0).stop 2023-07-06T20:09:58.959 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.570+0000 7fafe8198700 1 -- 172.21.15.69:0/3030095448 shutdown_connections 2023-07-06T20:09:58.959 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.570+0000 7fafe8198700 1 --2- 172.21.15.69:0/3030095448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafe007acc0 0x7fafe019f120 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:09:58.959 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.570+0000 7fafe8198700 1 -- 172.21.15.69:0/3030095448 >> 172.21.15.69:0/3030095448 conn(0x7fafe0100b90 msgr2=0x7fafe0102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:09:58.959 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.570+0000 7fafe8198700 1 -- 172.21.15.69:0/3030095448 shutdown_connections 2023-07-06T20:09:58.960 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:58.570+0000 7fafe8198700 1 -- 172.21.15.69:0/3030095448 wait complete. 2023-07-06T20:09:58.960 INFO:teuthology.orchestra.run.smithi069.stdout:Generating new minimal ceph.conf... 2023-07-06T20:09:58.984 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:58 smithi069 ceph-mon[125892]: from='client.? 172.21.15.69:0/4168757308' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-06T20:09:59.979 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:59 smithi069 ceph-mon[125892]: from='client.? 172.21.15.69:0/3030095448' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-06T20:09:59.979 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:09:59 smithi069 ceph-mon[125892]: from='client.? 172.21.15.69:0/3030095448' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-06T20:10:00.030 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.475+0000 7f8043771700 1 Processor -- start 2023-07-06T20:10:00.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.475+0000 7f8043771700 1 -- start start 2023-07-06T20:10:00.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.476+0000 7f8043771700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 0x7f803c079140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:00.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.476+0000 7f8043771700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c079680 con 0x7f803c07ac40 2023-07-06T20:10:00.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.476+0000 7f804150d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 0x7f803c079140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:00.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.476+0000 7f804150d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 0x7f803c079140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57022/0 (socket says 172.21.15.69:57022) 2023-07-06T20:10:00.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.476+0000 7f804150d700 1 -- 172.21.15.69:0/3883461901 learned_addr learned my addr 172.21.15.69:0/3883461901 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:00.031 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.477+0000 7f804150d700 1 -- 172.21.15.69:0/3883461901 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c0797c0 con 0x7f803c07ac40 2023-07-06T20:10:00.032 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.477+0000 7f804150d700 1 --2- 172.21.15.69:0/3883461901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 0x7f803c079140 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f80380098a0 tx=0x7f8038009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4855505010c377eb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:00.032 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.477+0000 7f8033fff700 1 -- 172.21.15.69:0/3883461901 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8038004160 con 0x7f803c07ac40 2023-07-06T20:10:00.032 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.477+0000 7f8033fff700 1 -- 172.21.15.69:0/3883461901 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f80380042c0 con 0x7f803c07ac40 2023-07-06T20:10:00.032 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.477+0000 7f8033fff700 1 -- 172.21.15.69:0/3883461901 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8038011a00 con 0x7f803c07ac40 2023-07-06T20:10:00.032 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.477+0000 7f8043771700 1 -- 172.21.15.69:0/3883461901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 msgr2=0x7f803c079140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:00.032 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.477+0000 7f8043771700 1 --2- 172.21.15.69:0/3883461901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 0x7f803c079140 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f80380098a0 tx=0x7f8038009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:00.032 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.478+0000 7f8043771700 1 -- 172.21.15.69:0/3883461901 shutdown_connections 2023-07-06T20:10:00.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.478+0000 7f8043771700 1 --2- 172.21.15.69:0/3883461901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 0x7f803c079140 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:00.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.478+0000 7f8043771700 1 -- 172.21.15.69:0/3883461901 >> 172.21.15.69:0/3883461901 conn(0x7f803c100b90 msgr2=0x7f803c102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:00.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.478+0000 7f8043771700 1 -- 172.21.15.69:0/3883461901 shutdown_connections 2023-07-06T20:10:00.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.478+0000 7f8043771700 1 -- 172.21.15.69:0/3883461901 wait complete. 2023-07-06T20:10:00.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.479+0000 7f8043771700 1 Processor -- start 2023-07-06T20:10:00.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.479+0000 7f8043771700 1 -- start start 2023-07-06T20:10:00.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.479+0000 7f8043771700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 0x7f803c19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:00.033 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.479+0000 7f8043771700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c19baa0 con 0x7f803c07ac40 2023-07-06T20:10:00.034 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.479+0000 7f804150d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 0x7f803c19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:00.034 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.479+0000 7f804150d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 0x7f803c19b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57030/0 (socket says 172.21.15.69:57030) 2023-07-06T20:10:00.034 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.479+0000 7f804150d700 1 -- 172.21.15.69:0/914893720 learned_addr learned my addr 172.21.15.69:0/914893720 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:00.034 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.480+0000 7f804150d700 1 -- 172.21.15.69:0/914893720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8038009550 con 0x7f803c07ac40 2023-07-06T20:10:00.034 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.480+0000 7f804150d700 1 --2- 172.21.15.69:0/914893720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 0x7f803c19b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8038009130 tx=0x7f8038011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:00.034 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.480+0000 7f80327fc700 1 -- 172.21.15.69:0/914893720 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8038003ab0 con 0x7f803c07ac40 2023-07-06T20:10:00.034 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.480+0000 7f80327fc700 1 -- 172.21.15.69:0/914893720 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8038003c10 con 0x7f803c07ac40 2023-07-06T20:10:00.034 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.480+0000 7f8043771700 1 -- 172.21.15.69:0/914893720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c19bca0 con 0x7f803c07ac40 2023-07-06T20:10:00.035 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.480+0000 7f80327fc700 1 -- 172.21.15.69:0/914893720 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80380247d0 con 0x7f803c07ac40 2023-07-06T20:10:00.035 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.480+0000 7f8043771700 1 -- 172.21.15.69:0/914893720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c19c0c0 con 0x7f803c07ac40 2023-07-06T20:10:00.035 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.481+0000 7f80327fc700 1 -- 172.21.15.69:0/914893720 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8038024d00 con 0x7f803c07ac40 2023-07-06T20:10:00.035 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.481+0000 7f80327fc700 1 -- 172.21.15.69:0/914893720 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8038019450 con 0x7f803c07ac40 2023-07-06T20:10:00.035 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.481+0000 7f8043771700 1 -- 172.21.15.69:0/914893720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803c195510 con 0x7f803c07ac40 2023-07-06T20:10:00.035 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.484+0000 7f80327fc700 1 -- 172.21.15.69:0/914893720 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f803801abb0 con 0x7f803c07ac40 2023-07-06T20:10:00.035 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.554+0000 7f8043771700 1 -- 172.21.15.69:0/914893720 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f803c02c8e0 con 0x7f803c07ac40 2023-07-06T20:10:00.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.554+0000 7f80327fc700 1 -- 172.21.15.69:0/914893720 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f803801ad50 con 0x7f803c07ac40 2023-07-06T20:10:00.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.555+0000 7f8043771700 1 -- 172.21.15.69:0/914893720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 msgr2=0x7f803c19b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:00.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.555+0000 7f8043771700 1 --2- 172.21.15.69:0/914893720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 0x7f803c19b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8038009130 tx=0x7f8038011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:00.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.556+0000 7f8043771700 1 -- 172.21.15.69:0/914893720 shutdown_connections 2023-07-06T20:10:00.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.556+0000 7f8043771700 1 --2- 172.21.15.69:0/914893720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f803c07ac40 0x7f803c19b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:00.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.556+0000 7f8043771700 1 -- 172.21.15.69:0/914893720 >> 172.21.15.69:0/914893720 conn(0x7f803c100b90 msgr2=0x7f803c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:00.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.556+0000 7f8043771700 1 -- 172.21.15.69:0/914893720 shutdown_connections 2023-07-06T20:10:00.036 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:09:59.556+0000 7f8043771700 1 -- 172.21.15.69:0/914893720 wait complete. 2023-07-06T20:10:00.037 INFO:teuthology.orchestra.run.smithi069.stdout:Restarting the monitor... 2023-07-06T20:10:00.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:00 smithi069 systemd[1]: Stopping Ceph mon.a for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:10:00.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:00 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-a[125868]: 2023-07-06T20:10:00.483+0000 7fa15d190700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:10:00.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:00 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-a[125868]: 2023-07-06T20:10:00.483+0000 7fa15d190700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-06T20:10:01.037 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:00 smithi069 bash[126236]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-a 2023-07-06T20:10:01.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:01 smithi069 systemd[1]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.a.service: Succeeded. 2023-07-06T20:10:01.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:01 smithi069 systemd[1]: Stopped Ceph mon.a for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:10:01.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:01 smithi069 systemd[1]: Starting Ceph mon.a for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:10:02.023 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:01 smithi069 podman[126359]: 2023-07-06T20:10:02.023 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 bash[126359]: 28dcfc1fcd487124de2c86f15000f6bf44635a4ae2ae10a18f159ea4d2dde94e 2023-07-06T20:10:02.023 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: set uid:gid to 167:167 (ceph:ceph) 2023-07-06T20:10:02.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev), process ceph-mon, pid 7 2023-07-06T20:10:02.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: pidfile_write: ignore empty --pid-file 2023-07-06T20:10:02.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: load: jerasure load: lrc 2023-07-06T20:10:02.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: RocksDB version: 7.9.2 2023-07-06T20:10:02.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Git sha 0 2023-07-06T20:10:02.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Compile date 2023-07-05 19:32:32 2023-07-06T20:10:02.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: DB SUMMARY 2023-07-06T20:10:02.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: DB Session ID: ZAVW95FKKJFVBT3GNVI6 2023-07-06T20:10:02.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: CURRENT file: CURRENT 2023-07-06T20:10:02.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: IDENTITY file: IDENTITY 2023-07-06T20:10:02.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-06T20:10:02.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-06T20:10:02.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98557 ; 2023-07-06T20:10:02.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.error_if_exists: 0 2023-07-06T20:10:02.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.create_if_missing: 0 2023-07-06T20:10:02.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.paranoid_checks: 1 2023-07-06T20:10:02.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-06T20:10:02.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-06T20:10:02.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-06T20:10:02.100 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.env: 0x562e90fa6720 2023-07-06T20:10:02.100 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.fs: PosixFileSystem 2023-07-06T20:10:02.100 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.info_log: 0x562e92299280 2023-07-06T20:10:02.100 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_file_opening_threads: 16 2023-07-06T20:10:02.100 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.statistics: (nil) 2023-07-06T20:10:02.100 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.use_fsync: 0 2023-07-06T20:10:02.100 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_log_file_size: 0 2023-07-06T20:10:02.101 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-06T20:10:02.101 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-06T20:10:02.101 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.keep_log_file_num: 1000 2023-07-06T20:10:02.101 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.recycle_log_file_num: 0 2023-07-06T20:10:02.101 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.allow_fallocate: 1 2023-07-06T20:10:02.101 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.allow_mmap_reads: 0 2023-07-06T20:10:02.101 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.allow_mmap_writes: 0 2023-07-06T20:10:02.102 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.use_direct_reads: 0 2023-07-06T20:10:02.102 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-06T20:10:02.102 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.create_missing_column_families: 0 2023-07-06T20:10:02.102 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.db_log_dir: 2023-07-06T20:10:02.102 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.wal_dir: 2023-07-06T20:10:02.102 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-06T20:10:02.102 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-06T20:10:02.103 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-06T20:10:02.103 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-06T20:10:02.103 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-06T20:10:02.103 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-06T20:10:02.103 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.advise_random_on_open: 1 2023-07-06T20:10:02.103 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.db_write_buffer_size: 0 2023-07-06T20:10:02.103 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.write_buffer_manager: 0x562e921ba3c0 2023-07-06T20:10:02.104 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-06T20:10:02.104 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-06T20:10:02.104 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-06T20:10:02.104 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.rate_limiter: (nil) 2023-07-06T20:10:02.104 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-06T20:10:02.104 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.wal_recovery_mode: 2 2023-07-06T20:10:02.104 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.enable_thread_tracking: 0 2023-07-06T20:10:02.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.enable_pipelined_write: 0 2023-07-06T20:10:02.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.unordered_write: 0 2023-07-06T20:10:02.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-06T20:10:02.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-06T20:10:02.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-06T20:10:02.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-06T20:10:02.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.row_cache: None 2023-07-06T20:10:02.106 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.wal_filter: None 2023-07-06T20:10:02.106 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-06T20:10:02.106 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.allow_ingest_behind: 0 2023-07-06T20:10:02.106 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.two_write_queues: 0 2023-07-06T20:10:02.106 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.manual_wal_flush: 0 2023-07-06T20:10:02.106 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.wal_compression: 0 2023-07-06T20:10:02.106 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.atomic_flush: 0 2023-07-06T20:10:02.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-06T20:10:02.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-06T20:10:02.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-06T20:10:02.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.log_readahead_size: 0 2023-07-06T20:10:02.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-06T20:10:02.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.best_efforts_recovery: 0 2023-07-06T20:10:02.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-06T20:10:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-06T20:10:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.allow_data_in_errors: 0 2023-07-06T20:10:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.db_host_id: __hostname__ 2023-07-06T20:10:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-06T20:10:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_background_jobs: 2 2023-07-06T20:10:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_background_compactions: -1 2023-07-06T20:10:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_subcompactions: 1 2023-07-06T20:10:02.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-06T20:10:02.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-06T20:10:02.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-06T20:10:02.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_total_wal_size: 0 2023-07-06T20:10:02.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-06T20:10:02.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-06T20:10:02.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-06T20:10:02.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-06T20:10:02.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_open_files: -1 2023-07-06T20:10:02.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bytes_per_sync: 0 2023-07-06T20:10:02.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-06T20:10:02.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-06T20:10:02.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_readahead_size: 0 2023-07-06T20:10:02.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_background_flushes: -1 2023-07-06T20:10:02.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Compression algorithms supported: 2023-07-06T20:10:02.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-06T20:10:02.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: kZSTD supported: 0 2023-07-06T20:10:02.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: kXpressCompression supported: 0 2023-07-06T20:10:02.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: kLZ4HCCompression supported: 1 2023-07-06T20:10:02.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: kZlibCompression supported: 1 2023-07-06T20:10:02.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: kSnappyCompression supported: 1 2023-07-06T20:10:02.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: kLZ4Compression supported: 1 2023-07-06T20:10:02.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: kBZip2Compression supported: 0 2023-07-06T20:10:02.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-06T20:10:02.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-06T20:10:02.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-06T20:10:02.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-06T20:10:02.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-06T20:10:02.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.merge_operator: 2023-07-06T20:10:02.113 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_filter: None 2023-07-06T20:10:02.113 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_filter_factory: None 2023-07-06T20:10:02.113 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.sst_partitioner_factory: None 2023-07-06T20:10:02.113 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-06T20:10:02.113 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-06T20:10:02.113 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562e92299380) 2023-07-06T20:10:02.113 INFO:journalctl@ceph.mon.a.smithi069.stdout: cache_index_and_filter_blocks: 1 2023-07-06T20:10:02.114 INFO:journalctl@ceph.mon.a.smithi069.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-06T20:10:02.114 INFO:journalctl@ceph.mon.a.smithi069.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-06T20:10:02.114 INFO:journalctl@ceph.mon.a.smithi069.stdout: pin_top_level_index_and_filter: 1 2023-07-06T20:10:02.114 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_type: 0 2023-07-06T20:10:02.114 INFO:journalctl@ceph.mon.a.smithi069.stdout: data_block_index_type: 0 2023-07-06T20:10:02.114 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_shortening: 1 2023-07-06T20:10:02.114 INFO:journalctl@ceph.mon.a.smithi069.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-06T20:10:02.115 INFO:journalctl@ceph.mon.a.smithi069.stdout: checksum: 4 2023-07-06T20:10:02.115 INFO:journalctl@ceph.mon.a.smithi069.stdout: no_block_cache: 0 2023-07-06T20:10:02.115 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache: 0x562e922591f0 2023-07-06T20:10:02.115 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_name: BinnedLRUCache 2023-07-06T20:10:02.115 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_options: 2023-07-06T20:10:02.115 INFO:journalctl@ceph.mon.a.smithi069.stdout: capacity : 536870912 2023-07-06T20:10:02.115 INFO:journalctl@ceph.mon.a.smithi069.stdout: num_shard_bits : 4 2023-07-06T20:10:02.116 INFO:journalctl@ceph.mon.a.smithi069.stdout: strict_capacity_limit : 0 2023-07-06T20:10:02.116 INFO:journalctl@ceph.mon.a.smithi069.stdout: high_pri_pool_ratio: 0.000 2023-07-06T20:10:02.116 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_compressed: (nil) 2023-07-06T20:10:02.116 INFO:journalctl@ceph.mon.a.smithi069.stdout: persistent_cache: (nil) 2023-07-06T20:10:02.116 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_size: 4096 2023-07-06T20:10:02.116 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_size_deviation: 10 2023-07-06T20:10:02.116 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_restart_interval: 16 2023-07-06T20:10:02.117 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_block_restart_interval: 1 2023-07-06T20:10:02.117 INFO:journalctl@ceph.mon.a.smithi069.stdout: metadata_block_size: 4096 2023-07-06T20:10:02.117 INFO:journalctl@ceph.mon.a.smithi069.stdout: partition_filters: 0 2023-07-06T20:10:02.117 INFO:journalctl@ceph.mon.a.smithi069.stdout: use_delta_encoding: 1 2023-07-06T20:10:02.117 INFO:journalctl@ceph.mon.a.smithi069.stdout: filter_policy: bloomfilter 2023-07-06T20:10:02.117 INFO:journalctl@ceph.mon.a.smithi069.stdout: whole_key_filtering: 1 2023-07-06T20:10:02.117 INFO:journalctl@ceph.mon.a.smithi069.stdout: verify_compression: 0 2023-07-06T20:10:02.117 INFO:journalctl@ceph.mon.a.smithi069.stdout: read_amp_bytes_per_bit: 0 2023-07-06T20:10:02.118 INFO:journalctl@ceph.mon.a.smithi069.stdout: format_version: 5 2023-07-06T20:10:02.118 INFO:journalctl@ceph.mon.a.smithi069.stdout: enable_index_compression: 1 2023-07-06T20:10:02.118 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_align: 0 2023-07-06T20:10:02.118 INFO:journalctl@ceph.mon.a.smithi069.stdout: max_auto_readahead_size: 262144 2023-07-06T20:10:02.118 INFO:journalctl@ceph.mon.a.smithi069.stdout: prepopulate_block_cache: 0 2023-07-06T20:10:02.118 INFO:journalctl@ceph.mon.a.smithi069.stdout: initial_auto_readahead_size: 8192 2023-07-06T20:10:02.118 INFO:journalctl@ceph.mon.a.smithi069.stdout: num_file_reads_for_auto_readahead: 2 2023-07-06T20:10:02.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.write_buffer_size: 33554432 2023-07-06T20:10:02.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_write_buffer_number: 2 2023-07-06T20:10:02.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compression: NoCompression 2023-07-06T20:10:02.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bottommost_compression: Disabled 2023-07-06T20:10:02.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.prefix_extractor: nullptr 2023-07-06T20:10:02.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-06T20:10:02.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.num_levels: 7 2023-07-06T20:10:02.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-06T20:10:02.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-06T20:10:02.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-06T20:10:02.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-06T20:10:02.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-06T20:10:02.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-06T20:10:02.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-06T20:10:02.121 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-06T20:10:02.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-06T20:10:02.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-06T20:10:02.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-06T20:10:02.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-06T20:10:02.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-06T20:10:02.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compression_opts.level: 32767 2023-07-06T20:10:02.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compression_opts.strategy: 0 2023-07-06T20:10:02.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-06T20:10:02.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-06T20:10:02.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-06T20:10:02.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-06T20:10:02.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compression_opts.enabled: false 2023-07-06T20:10:02.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-06T20:10:02.124 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-06T20:10:02.124 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-06T20:10:02.124 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-06T20:10:02.124 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.target_file_size_base: 67108864 2023-07-06T20:10:02.124 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-06T20:10:02.124 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-06T20:10:02.124 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-06T20:10:02.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-06T20:10:02.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-06T20:10:02.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-06T20:10:02.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-06T20:10:02.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-06T20:10:02.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-06T20:10:02.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-06T20:10:02.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-06T20:10:02.126 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-06T20:10:02.126 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-06T20:10:02.126 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-06T20:10:02.126 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.arena_block_size: 1048576 2023-07-06T20:10:02.126 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-06T20:10:02.126 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-06T20:10:02.126 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.disable_auto_compactions: 0 2023-07-06T20:10:02.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-06T20:10:02.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-06T20:10:02.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-06T20:10:02.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-06T20:10:02.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-06T20:10:02.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-06T20:10:02.127 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-06T20:10:02.128 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-06T20:10:02.128 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-06T20:10:02.128 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-06T20:10:02.128 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.table_properties_collectors: 2023-07-06T20:10:02.128 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.inplace_update_support: 0 2023-07-06T20:10:02.128 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-06T20:10:02.128 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-06T20:10:02.129 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-06T20:10:02.129 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-06T20:10:02.129 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.bloom_locality: 0 2023-07-06T20:10:02.129 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.max_successive_merges: 0 2023-07-06T20:10:02.129 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-06T20:10:02.129 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.paranoid_file_checks: 0 2023-07-06T20:10:02.129 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.force_consistency_checks: 1 2023-07-06T20:10:02.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.report_bg_io_stats: 0 2023-07-06T20:10:02.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.ttl: 2592000 2023-07-06T20:10:02.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-06T20:10:02.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-06T20:10:02.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-06T20:10:02.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.enable_blob_files: false 2023-07-06T20:10:02.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.min_blob_size: 0 2023-07-06T20:10:02.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.blob_file_size: 268435456 2023-07-06T20:10:02.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-06T20:10:02.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-06T20:10:02.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-06T20:10:02.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-06T20:10:02.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-06T20:10:02.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.blob_file_starting_level: 0 2023-07-06T20:10:02.132 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-06T20:10:02.132 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-06T20:10:02.132 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-06T20:10:02.132 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3920bbce-cdf0-46f3-a0a0-191d0852de8d 2023-07-06T20:10:02.132 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: EVENT_LOG_v1 {"time_micros": 1688674202030637, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-06T20:10:02.132 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-06T20:10:02.133 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: EVENT_LOG_v1 {"time_micros": 1688674202032483, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93726, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 317, "table_properties": {"data_size": 91717, "index_size": 291, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13853, "raw_average_key_size": 50, "raw_value_size": 84171, "raw_average_value_size": 304, "num_data_blocks": 11, "num_entries": 276, "num_filter_entries": 276, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1688674202, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3920bbce-cdf0-46f3-a0a0-191d0852de8d", "db_session_id": "ZAVW95FKKJFVBT3GNVI6", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-06T20:10:02.133 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: EVENT_LOG_v1 {"time_micros": 1688674202032613, "job": 1, "event": "recovery_finished"} 2023-07-06T20:10:02.133 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-06T20:10:02.133 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-06T20:10:02.133 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562e922f2000 2023-07-06T20:10:02.133 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: DB pointer 0x562e922de000 2023-07-06T20:10:02.133 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-06T20:10:02.134 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-06T20:10:02.134 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** DB Stats ** 2023-07-06T20:10:02.134 INFO:journalctl@ceph.mon.a.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-06T20:10:02.134 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-06T20:10:02.134 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-06T20:10:02.134 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-06T20:10:02.134 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-06T20:10:02.135 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-06T20:10:02.135 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-06T20:10:02.135 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2023-07-06T20:10:02.135 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** Compaction Stats [default] ** 2023-07-06T20:10:02.135 INFO:journalctl@ceph.mon.a.smithi069.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-06T20:10:02.135 INFO:journalctl@ceph.mon.a.smithi069.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-06T20:10:02.135 INFO:journalctl@ceph.mon.a.smithi069.stdout: L0 2/0 93.36 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-06T20:10:02.136 INFO:journalctl@ceph.mon.a.smithi069.stdout: Sum 2/0 93.36 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-06T20:10:02.136 INFO:journalctl@ceph.mon.a.smithi069.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-06T20:10:02.136 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2023-07-06T20:10:02.136 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** Compaction Stats [default] ** 2023-07-06T20:10:02.136 INFO:journalctl@ceph.mon.a.smithi069.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-06T20:10:02.136 INFO:journalctl@ceph.mon.a.smithi069.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-06T20:10:02.136 INFO:journalctl@ceph.mon.a.smithi069.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 72.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-06T20:10:02.137 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2023-07-06T20:10:02.137 INFO:journalctl@ceph.mon.a.smithi069.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-06T20:10:02.137 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2023-07-06T20:10:02.137 INFO:journalctl@ceph.mon.a.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-06T20:10:02.137 INFO:journalctl@ceph.mon.a.smithi069.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-06T20:10:02.137 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-06T20:10:02.137 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-06T20:10:02.138 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-06T20:10:02.138 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-06T20:10:02.138 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative compaction: 0.00 GB write, 14.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-06T20:10:02.138 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval compaction: 0.00 GB write, 14.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-06T20:10:02.138 INFO:journalctl@ceph.mon.a.smithi069.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-06T20:10:02.138 INFO:journalctl@ceph.mon.a.smithi069.stdout: Block cache BinnedLRUCache@0x562e922591f0#7 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-07-06T20:10:02.138 INFO:journalctl@ceph.mon.a.smithi069.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2023-07-06T20:10:02.139 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2023-07-06T20:10:02.139 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-06T20:10:02.139 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: starting mon.a rank 0 at public addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] at bind addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:10:02.139 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: mon.a@-1(???) e1 preinit fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:10:02.139 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: mon.a@-1(???).mds e1 new map 2023-07-06T20:10:02.139 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: mon.a@-1(???).mds e1 print_map 2023-07-06T20:10:02.139 INFO:journalctl@ceph.mon.a.smithi069.stdout: e1 2023-07-06T20:10:02.140 INFO:journalctl@ceph.mon.a.smithi069.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-06T20:10:02.140 INFO:journalctl@ceph.mon.a.smithi069.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-06T20:10:02.140 INFO:journalctl@ceph.mon.a.smithi069.stdout: legacy client fscid: -1 2023-07-06T20:10:02.140 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2023-07-06T20:10:02.140 INFO:journalctl@ceph.mon.a.smithi069.stdout: No filesystems configured 2023-07-06T20:10:02.140 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-06T20:10:02.140 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-06T20:10:02.141 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-06T20:10:02.141 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-06T20:10:02.141 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-06T20:10:02.141 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-06T20:10:02.141 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: monmap e1: 1 mons at {a=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0]} removed_ranks: {} 2023-07-06T20:10:02.141 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: fsmap 2023-07-06T20:10:02.141 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: osdmap e1: 0 total, 0 up, 0 in 2023-07-06T20:10:02.142 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 ceph-mon[126399]: mgrmap e1: no daemons active 2023-07-06T20:10:02.142 INFO:teuthology.orchestra.run.smithi069.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-06T20:10:02.372 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:02 smithi069 systemd[1]: Started Ceph mon.a for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:10:03.185 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.648+0000 7fdc03c28700 1 Processor -- start 2023-07-06T20:10:03.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.648+0000 7fdc03c28700 1 -- start start 2023-07-06T20:10:03.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.648+0000 7fdc03c28700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 0x7fdbfc079160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:03.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.648+0000 7fdc03c28700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc0796a0 con 0x7fdbfc07ac60 2023-07-06T20:10:03.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.649+0000 7fdc019c4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 0x7fdbfc079160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:03.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.649+0000 7fdc019c4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 0x7fdbfc079160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57032/0 (socket says 172.21.15.69:57032) 2023-07-06T20:10:03.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.649+0000 7fdc019c4700 1 -- 172.21.15.69:0/3913908536 learned_addr learned my addr 172.21.15.69:0/3913908536 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:03.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.649+0000 7fdc019c4700 1 -- 172.21.15.69:0/3913908536 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc0797e0 con 0x7fdbfc07ac60 2023-07-06T20:10:03.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.650+0000 7fdc019c4700 1 --2- 172.21.15.69:0/3913908536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 0x7fdbfc079160 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80098a0 tx=0x7fdbf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf51419482241d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:03.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.650+0000 7fdc009c2700 1 -- 172.21.15.69:0/3913908536 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbf8004160 con 0x7fdbfc07ac60 2023-07-06T20:10:03.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.650+0000 7fdc009c2700 1 -- 172.21.15.69:0/3913908536 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdbf80042c0 con 0x7fdbfc07ac60 2023-07-06T20:10:03.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.650+0000 7fdc009c2700 1 -- 172.21.15.69:0/3913908536 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbf8011a00 con 0x7fdbfc07ac60 2023-07-06T20:10:03.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.650+0000 7fdc03c28700 1 -- 172.21.15.69:0/3913908536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 msgr2=0x7fdbfc079160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:03.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.650+0000 7fdc03c28700 1 --2- 172.21.15.69:0/3913908536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 0x7fdbfc079160 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80098a0 tx=0x7fdbf8009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:03.188 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.650+0000 7fdc03c28700 1 -- 172.21.15.69:0/3913908536 shutdown_connections 2023-07-06T20:10:03.188 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.650+0000 7fdc03c28700 1 --2- 172.21.15.69:0/3913908536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 0x7fdbfc079160 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:03.188 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.650+0000 7fdc03c28700 1 -- 172.21.15.69:0/3913908536 >> 172.21.15.69:0/3913908536 conn(0x7fdbfc100b90 msgr2=0x7fdbfc102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:03.188 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.651+0000 7fdc03c28700 1 -- 172.21.15.69:0/3913908536 shutdown_connections 2023-07-06T20:10:03.188 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.651+0000 7fdc03c28700 1 -- 172.21.15.69:0/3913908536 wait complete. 2023-07-06T20:10:03.188 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.652+0000 7fdc03c28700 1 Processor -- start 2023-07-06T20:10:03.188 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.652+0000 7fdc03c28700 1 -- start start 2023-07-06T20:10:03.189 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.652+0000 7fdc03c28700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 0x7fdbfc19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:03.189 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.652+0000 7fdc03c28700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc19baa0 con 0x7fdbfc07ac60 2023-07-06T20:10:03.189 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.652+0000 7fdc019c4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 0x7fdbfc19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:03.189 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.652+0000 7fdc019c4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 0x7fdbfc19b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57044/0 (socket says 172.21.15.69:57044) 2023-07-06T20:10:03.189 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.652+0000 7fdc019c4700 1 -- 172.21.15.69:0/2880480447 learned_addr learned my addr 172.21.15.69:0/2880480447 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:03.189 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.653+0000 7fdc019c4700 1 -- 172.21.15.69:0/2880480447 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf8009550 con 0x7fdbfc07ac60 2023-07-06T20:10:03.189 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.653+0000 7fdc019c4700 1 --2- 172.21.15.69:0/2880480447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 0x7fdbfc19b560 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009130 tx=0x7fdbf8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:03.189 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.653+0000 7fdbf2ffd700 1 -- 172.21.15.69:0/2880480447 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbf8003ab0 con 0x7fdbfc07ac60 2023-07-06T20:10:03.190 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.653+0000 7fdbf2ffd700 1 -- 172.21.15.69:0/2880480447 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdbf8003c10 con 0x7fdbfc07ac60 2023-07-06T20:10:03.190 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.653+0000 7fdc03c28700 1 -- 172.21.15.69:0/2880480447 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc19bca0 con 0x7fdbfc07ac60 2023-07-06T20:10:03.190 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.653+0000 7fdbf2ffd700 1 -- 172.21.15.69:0/2880480447 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbf80247d0 con 0x7fdbfc07ac60 2023-07-06T20:10:03.190 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.653+0000 7fdc03c28700 1 -- 172.21.15.69:0/2880480447 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc19c0c0 con 0x7fdbfc07ac60 2023-07-06T20:10:03.190 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.653+0000 7fdbf2ffd700 1 -- 172.21.15.69:0/2880480447 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdbf8024d50 con 0x7fdbfc07ac60 2023-07-06T20:10:03.190 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.654+0000 7fdbf2ffd700 1 -- 172.21.15.69:0/2880480447 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdbf80193c0 con 0x7fdbfc07ac60 2023-07-06T20:10:03.190 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.654+0000 7fdc03c28700 1 -- 172.21.15.69:0/2880480447 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbfc195510 con 0x7fdbfc07ac60 2023-07-06T20:10:03.191 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.657+0000 7fdbf2ffd700 1 -- 172.21.15.69:0/2880480447 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fdbf801abb0 con 0x7fdbfc07ac60 2023-07-06T20:10:03.191 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.726+0000 7fdc03c28700 1 -- 172.21.15.69:0/2880480447 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fdbfc02c8e0 con 0x7fdbfc07ac60 2023-07-06T20:10:03.191 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.729+0000 7fdbf2ffd700 1 -- 172.21.15.69:0/2880480447 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdbf801ad50 con 0x7fdbfc07ac60 2023-07-06T20:10:03.191 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.729+0000 7fdc03c28700 1 -- 172.21.15.69:0/2880480447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 msgr2=0x7fdbfc19b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:03.191 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.729+0000 7fdc03c28700 1 --2- 172.21.15.69:0/2880480447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 0x7fdbfc19b560 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009130 tx=0x7fdbf8011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:03.191 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.730+0000 7fdc03c28700 1 -- 172.21.15.69:0/2880480447 shutdown_connections 2023-07-06T20:10:03.191 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.730+0000 7fdc03c28700 1 --2- 172.21.15.69:0/2880480447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbfc07ac60 0x7fdbfc19b560 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:03.192 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.730+0000 7fdc03c28700 1 -- 172.21.15.69:0/2880480447 >> 172.21.15.69:0/2880480447 conn(0x7fdbfc100b90 msgr2=0x7fdbfc1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:03.192 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.730+0000 7fdc03c28700 1 -- 172.21.15.69:0/2880480447 shutdown_connections 2023-07-06T20:10:03.192 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:02.730+0000 7fdc03c28700 1 -- 172.21.15.69:0/2880480447 wait complete. 2023-07-06T20:10:03.192 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-06T20:10:03.216 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-06T20:10:03.216 INFO:teuthology.orchestra.run.smithi069.stdout:Creating mgr... 2023-07-06T20:10:03.217 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying port 9283 ... 2023-07-06T20:10:03.217 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying port 8765 ... 2023-07-06T20:10:03.554 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.x 2023-07-06T20:10:03.555 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.x.service: Unit ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.x.service not loaded. 2023-07-06T20:10:03.762 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c.target.wants/ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@.service. 2023-07-06T20:10:04.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:03 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2880480447' entity='client.admin' 2023-07-06T20:10:04.010 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:03 smithi069 systemd[1]: Starting Ceph mgr.x for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:10:04.504 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:04 smithi069 podman[126633]: 2023-07-06T20:10:04.559 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-06T20:10:04.559 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2023-07-06T20:10:04.576 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-06T20:10:04.576 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2023-07-06T20:10:04.577 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2023-07-06T20:10:04.577 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to enable service . firewalld.service is not available 2023-07-06T20:10:04.594 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-06T20:10:04.595 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2023-07-06T20:10:04.610 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-06T20:10:04.611 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2023-07-06T20:10:04.611 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2023-07-06T20:10:04.611 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-06T20:10:04.611 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr to start... 2023-07-06T20:10:04.611 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr... 2023-07-06T20:10:04.759 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:04 smithi069 bash[126633]: 568a2d077ffd7520a7cfeb9d4eb5c62f47f366ab0510475305a9f26bd698cba7 2023-07-06T20:10:04.759 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:04 smithi069 systemd[1]: Started Ceph mgr.x for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:10:04.759 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:04 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:04.688+0000 7f5b66f21200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T20:10:05.052 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:04 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:04.794+0000 7f5b66f21200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T20:10:05.053 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:04 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:04.920+0000 7f5b66f21200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T20:10:05.053 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:05 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:05.050+0000 7f5b66f21200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T20:10:05.356 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:05 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2648958919' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-06T20:10:05.357 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:05 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:05.276+0000 7f5b66f21200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T20:10:05.594 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:10:05.596 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2023-07-06T20:10:05.597 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "e3e8ec0c-1c38-11ee-9b2f-001a4aab830c", 2023-07-06T20:10:05.597 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2023-07-06T20:10:05.597 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-06T20:10:05.597 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-06T20:10:05.597 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-06T20:10:05.597 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:05.598 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-06T20:10:05.598 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-06T20:10:05.598 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2023-07-06T20:10:05.598 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2023-07-06T20:10:05.598 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-06T20:10:05.598 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2023-07-06T20:10:05.598 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2023-07-06T20:10:05.598 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-07-06T20:10:05.599 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-06T20:10:05.599 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:05.599 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-06T20:10:05.599 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-06T20:10:05.599 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:05.599 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-06T20:10:05.599 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:05.599 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-06T20:10:05.600 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-06T20:10:05.600 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-06T20:10:05.605 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-06T20:10:05.605 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-06T20:10:05.605 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-06T20:10:05.605 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:05.606 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-06T20:10:05.606 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-06T20:10:05.606 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-06T20:10:05.606 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-06T20:10:05.606 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-06T20:10:05.606 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-06T20:10:05.606 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-06T20:10:05.607 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-06T20:10:05.607 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-06T20:10:05.607 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:05.607 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-06T20:10:05.607 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:05.607 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-06T20:10:05.607 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-06T20:10:05.607 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:05.608 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-06T20:10:05.608 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-06T20:10:05.608 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-06T20:10:05.608 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-06T20:10:05.608 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-06T20:10:05.608 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-06T20:10:05.608 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2023-07-06T20:10:05.608 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2023-07-06T20:10:05.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-06T20:10:05.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:05.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-06T20:10:05.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:05.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2023-07-06T20:09:56.507323+0000", 2023-07-06T20:10:05.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-06T20:10:05.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:05.609 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-06T20:10:05.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2023-07-06T20:10:05.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.163+0000 7fc6ac195700 1 Processor -- start 2023-07-06T20:10:05.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.163+0000 7fc6ac195700 1 -- start start 2023-07-06T20:10:05.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.164+0000 7fc6ac195700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 0x7fc6a4105950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:05.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.164+0000 7fc6ac195700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a4105e90 con 0x7fc6a4105540 2023-07-06T20:10:05.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.164+0000 7fc6a9f31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 0x7fc6a4105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:05.610 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.164+0000 7fc6a9f31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 0x7fc6a4105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60906/0 (socket says 172.21.15.69:60906) 2023-07-06T20:10:05.611 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.164+0000 7fc6a9f31700 1 -- 172.21.15.69:0/1050524139 learned_addr learned my addr 172.21.15.69:0/1050524139 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:05.611 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.165+0000 7fc6a9f31700 1 -- 172.21.15.69:0/1050524139 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a4105fd0 con 0x7fc6a4105540 2023-07-06T20:10:05.611 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.165+0000 7fc6a9f31700 1 --2- 172.21.15.69:0/1050524139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 0x7fc6a4105950 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00098a0 tx=0x7fc6a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d761b6514294c20 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:05.611 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.165+0000 7fc6a8f2f700 1 -- 172.21.15.69:0/1050524139 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6a0004160 con 0x7fc6a4105540 2023-07-06T20:10:05.611 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.165+0000 7fc6a8f2f700 1 -- 172.21.15.69:0/1050524139 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6a00042c0 con 0x7fc6a4105540 2023-07-06T20:10:05.611 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.165+0000 7fc6a8f2f700 1 -- 172.21.15.69:0/1050524139 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6a0011a00 con 0x7fc6a4105540 2023-07-06T20:10:05.611 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.166+0000 7fc6ac195700 1 -- 172.21.15.69:0/1050524139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 msgr2=0x7fc6a4105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:05.611 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.166+0000 7fc6ac195700 1 --2- 172.21.15.69:0/1050524139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 0x7fc6a4105950 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00098a0 tx=0x7fc6a0009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:05.612 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.166+0000 7fc6ac195700 1 -- 172.21.15.69:0/1050524139 shutdown_connections 2023-07-06T20:10:05.612 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.166+0000 7fc6ac195700 1 --2- 172.21.15.69:0/1050524139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 0x7fc6a4105950 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:05.612 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.166+0000 7fc6ac195700 1 -- 172.21.15.69:0/1050524139 >> 172.21.15.69:0/1050524139 conn(0x7fc6a4100b90 msgr2=0x7fc6a4102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:05.612 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.166+0000 7fc6ac195700 1 -- 172.21.15.69:0/1050524139 shutdown_connections 2023-07-06T20:10:05.612 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.166+0000 7fc6ac195700 1 -- 172.21.15.69:0/1050524139 wait complete. 2023-07-06T20:10:05.612 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.167+0000 7fc6ac195700 1 Processor -- start 2023-07-06T20:10:05.612 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.167+0000 7fc6ac195700 1 -- start start 2023-07-06T20:10:05.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.168+0000 7fc6ac195700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 0x7fc6a419b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:05.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.168+0000 7fc6ac195700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a419baa0 con 0x7fc6a4105540 2023-07-06T20:10:05.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.168+0000 7fc6a9f31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 0x7fc6a419b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:05.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.168+0000 7fc6a9f31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 0x7fc6a419b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60920/0 (socket says 172.21.15.69:60920) 2023-07-06T20:10:05.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.168+0000 7fc6a9f31700 1 -- 172.21.15.69:0/2648958919 learned_addr learned my addr 172.21.15.69:0/2648958919 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:05.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.168+0000 7fc6a9f31700 1 -- 172.21.15.69:0/2648958919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a0009550 con 0x7fc6a4105540 2023-07-06T20:10:05.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.168+0000 7fc6a9f31700 1 --2- 172.21.15.69:0/2648958919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 0x7fc6a419b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009fb0 tx=0x7fc6a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:05.613 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.169+0000 7fc69affd700 1 -- 172.21.15.69:0/2648958919 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6a0003a40 con 0x7fc6a4105540 2023-07-06T20:10:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.169+0000 7fc69affd700 1 -- 172.21.15.69:0/2648958919 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6a0003ba0 con 0x7fc6a4105540 2023-07-06T20:10:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.169+0000 7fc6ac195700 1 -- 172.21.15.69:0/2648958919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a419bca0 con 0x7fc6a4105540 2023-07-06T20:10:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.169+0000 7fc69affd700 1 -- 172.21.15.69:0/2648958919 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6a001a750 con 0x7fc6a4105540 2023-07-06T20:10:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.169+0000 7fc6ac195700 1 -- 172.21.15.69:0/2648958919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a419c0c0 con 0x7fc6a4105540 2023-07-06T20:10:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.169+0000 7fc69affd700 1 -- 172.21.15.69:0/2648958919 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc6a001acd0 con 0x7fc6a4105540 2023-07-06T20:10:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.169+0000 7fc69affd700 1 -- 172.21.15.69:0/2648958919 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc6a00193c0 con 0x7fc6a4105540 2023-07-06T20:10:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.170+0000 7fc6ac195700 1 -- 172.21.15.69:0/2648958919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a4061c70 con 0x7fc6a4105540 2023-07-06T20:10:05.614 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.173+0000 7fc69affd700 1 -- 172.21.15.69:0/2648958919 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fc6a0024b60 con 0x7fc6a4105540 2023-07-06T20:10:05.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.247+0000 7fc6ac195700 1 -- 172.21.15.69:0/2648958919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc6a4107400 con 0x7fc6a4105540 2023-07-06T20:10:05.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.247+0000 7fc69affd700 1 -- 172.21.15.69:0/2648958919 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc6a0024d00 con 0x7fc6a4105540 2023-07-06T20:10:05.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.248+0000 7fc6ac195700 1 -- 172.21.15.69:0/2648958919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 msgr2=0x7fc6a419b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:05.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.248+0000 7fc6ac195700 1 --2- 172.21.15.69:0/2648958919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 0x7fc6a419b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009fb0 tx=0x7fc6a0011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:05.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.248+0000 7fc6ac195700 1 -- 172.21.15.69:0/2648958919 shutdown_connections 2023-07-06T20:10:05.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.248+0000 7fc6ac195700 1 --2- 172.21.15.69:0/2648958919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6a4105540 0x7fc6a419b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:05.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.248+0000 7fc6ac195700 1 -- 172.21.15.69:0/2648958919 >> 172.21.15.69:0/2648958919 conn(0x7fc6a4100b90 msgr2=0x7fc6a4102290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:05.615 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.248+0000 7fc6ac195700 1 -- 172.21.15.69:0/2648958919 shutdown_connections 2023-07-06T20:10:05.616 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:05.248+0000 7fc6ac195700 1 -- 172.21.15.69:0/2648958919 wait complete. 2023-07-06T20:10:05.616 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (1/15)... 2023-07-06T20:10:05.968 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:05 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:05.693+0000 7f5b66f21200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T20:10:05.969 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:05 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:05.807+0000 7f5b66f21200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T20:10:05.969 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:05 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:05.966+0000 7f5b66f21200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T20:10:06.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:06 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:06.088+0000 7f5b66f21200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T20:10:07.008 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:06 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:06.622+0000 7f5b66f21200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T20:10:07.759 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:07 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:07.393+0000 7f5b66f21200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T20:10:07.759 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:07 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:07.600+0000 7f5b66f21200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T20:10:08.887 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:08 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:08.546+0000 7f5b66f21200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:10:08.888 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:08 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:08.667+0000 7f5b66f21200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T20:10:09.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:08 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:08.885+0000 7f5b66f21200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:10:09.554 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:09 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2512463739' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-06T20:10:09.554 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:09 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:09.260+0000 7f5b66f21200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T20:10:09.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:10:09.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2023-07-06T20:10:09.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "e3e8ec0c-1c38-11ee-9b2f-001a4aab830c", 2023-07-06T20:10:09.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2023-07-06T20:10:09.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-06T20:10:09.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-06T20:10:09.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-06T20:10:09.689 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:09.690 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-06T20:10:09.690 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-06T20:10:09.690 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2023-07-06T20:10:09.690 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2023-07-06T20:10:09.690 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-06T20:10:09.690 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2023-07-06T20:10:09.691 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2023-07-06T20:10:09.691 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2023-07-06T20:10:09.691 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-06T20:10:09.691 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:09.691 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-06T20:10:09.691 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-06T20:10:09.691 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:09.692 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-06T20:10:09.692 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:09.692 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-06T20:10:09.692 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-06T20:10:09.692 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-06T20:10:09.697 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-06T20:10:09.698 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-06T20:10:09.698 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-06T20:10:09.698 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:09.698 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-06T20:10:09.698 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-06T20:10:09.698 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-06T20:10:09.699 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-06T20:10:09.699 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-06T20:10:09.699 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-06T20:10:09.699 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-06T20:10:09.699 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-06T20:10:09.699 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-06T20:10:09.699 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:09.699 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-06T20:10:09.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:09.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-06T20:10:09.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-06T20:10:09.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:09.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-06T20:10:09.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-06T20:10:09.700 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-06T20:10:09.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-06T20:10:09.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-06T20:10:09.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-06T20:10:09.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2023-07-06T20:10:09.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2023-07-06T20:10:09.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-06T20:10:09.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:09.701 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-06T20:10:09.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:09.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2023-07-06T20:09:56.507323+0000", 2023-07-06T20:10:09.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-06T20:10:09.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:09.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-06T20:10:09.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2023-07-06T20:10:09.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.219+0000 7f402c2c5700 1 Processor -- start 2023-07-06T20:10:09.702 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.220+0000 7f402c2c5700 1 -- start start 2023-07-06T20:10:09.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.220+0000 7f402c2c5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 0x7f4024079160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:09.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.220+0000 7f402c2c5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40240796a0 con 0x7f402407ac60 2023-07-06T20:10:09.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.221+0000 7f402a061700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 0x7f4024079160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:09.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.221+0000 7f402a061700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 0x7f4024079160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60934/0 (socket says 172.21.15.69:60934) 2023-07-06T20:10:09.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.221+0000 7f402a061700 1 -- 172.21.15.69:0/1514055600 learned_addr learned my addr 172.21.15.69:0/1514055600 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:09.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.221+0000 7f402a061700 1 -- 172.21.15.69:0/1514055600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40240797e0 con 0x7f402407ac60 2023-07-06T20:10:09.703 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.221+0000 7f402a061700 1 --2- 172.21.15.69:0/1514055600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 0x7f4024079160 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40200098a0 tx=0x7f4020009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a429db9edf98b6b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:09.704 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.222+0000 7f402905f700 1 -- 172.21.15.69:0/1514055600 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4020004160 con 0x7f402407ac60 2023-07-06T20:10:09.704 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.222+0000 7f402905f700 1 -- 172.21.15.69:0/1514055600 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f40200042c0 con 0x7f402407ac60 2023-07-06T20:10:09.704 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.222+0000 7f402905f700 1 -- 172.21.15.69:0/1514055600 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4020011a00 con 0x7f402407ac60 2023-07-06T20:10:09.704 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.222+0000 7f402c2c5700 1 -- 172.21.15.69:0/1514055600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 msgr2=0x7f4024079160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:09.704 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.222+0000 7f402c2c5700 1 --2- 172.21.15.69:0/1514055600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 0x7f4024079160 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40200098a0 tx=0x7f4020009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:09.704 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.222+0000 7f402c2c5700 1 -- 172.21.15.69:0/1514055600 shutdown_connections 2023-07-06T20:10:09.704 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.222+0000 7f402c2c5700 1 --2- 172.21.15.69:0/1514055600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 0x7f4024079160 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:09.704 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.222+0000 7f402c2c5700 1 -- 172.21.15.69:0/1514055600 >> 172.21.15.69:0/1514055600 conn(0x7f4024100b90 msgr2=0x7f4024102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:09.705 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.222+0000 7f402c2c5700 1 -- 172.21.15.69:0/1514055600 shutdown_connections 2023-07-06T20:10:09.705 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.223+0000 7f402c2c5700 1 -- 172.21.15.69:0/1514055600 wait complete. 2023-07-06T20:10:09.705 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.224+0000 7f402c2c5700 1 Processor -- start 2023-07-06T20:10:09.705 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.224+0000 7f402c2c5700 1 -- start start 2023-07-06T20:10:09.705 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.224+0000 7f402c2c5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 0x7f402419b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:09.705 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.224+0000 7f402c2c5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402419baa0 con 0x7f402407ac60 2023-07-06T20:10:09.705 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.224+0000 7f402a061700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 0x7f402419b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:09.705 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.224+0000 7f402a061700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 0x7f402419b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60942/0 (socket says 172.21.15.69:60942) 2023-07-06T20:10:09.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.224+0000 7f402a061700 1 -- 172.21.15.69:0/2512463739 learned_addr learned my addr 172.21.15.69:0/2512463739 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:09.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.225+0000 7f402a061700 1 -- 172.21.15.69:0/2512463739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4020009550 con 0x7f402407ac60 2023-07-06T20:10:09.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.225+0000 7f402a061700 1 --2- 172.21.15.69:0/2512463739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 0x7f402419b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4020009130 tx=0x7f4020011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:09.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.225+0000 7f401b7fe700 1 -- 172.21.15.69:0/2512463739 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4020003ab0 con 0x7f402407ac60 2023-07-06T20:10:09.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.225+0000 7f401b7fe700 1 -- 172.21.15.69:0/2512463739 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4020003c10 con 0x7f402407ac60 2023-07-06T20:10:09.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.225+0000 7f402c2c5700 1 -- 172.21.15.69:0/2512463739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402419bca0 con 0x7f402407ac60 2023-07-06T20:10:09.706 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.225+0000 7f401b7fe700 1 -- 172.21.15.69:0/2512463739 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40200247d0 con 0x7f402407ac60 2023-07-06T20:10:09.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.225+0000 7f402c2c5700 1 -- 172.21.15.69:0/2512463739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402419c0c0 con 0x7f402407ac60 2023-07-06T20:10:09.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.226+0000 7f401b7fe700 1 -- 172.21.15.69:0/2512463739 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4020024d00 con 0x7f402407ac60 2023-07-06T20:10:09.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.226+0000 7f401b7fe700 1 -- 172.21.15.69:0/2512463739 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4020019450 con 0x7f402407ac60 2023-07-06T20:10:09.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.227+0000 7f402c2c5700 1 -- 172.21.15.69:0/2512463739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4024195510 con 0x7f402407ac60 2023-07-06T20:10:09.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.229+0000 7f401b7fe700 1 -- 172.21.15.69:0/2512463739 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f402001abb0 con 0x7f402407ac60 2023-07-06T20:10:09.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.304+0000 7f402c2c5700 1 -- 172.21.15.69:0/2512463739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f402402c8e0 con 0x7f402407ac60 2023-07-06T20:10:09.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.304+0000 7f401b7fe700 1 -- 172.21.15.69:0/2512463739 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f402001ad50 con 0x7f402407ac60 2023-07-06T20:10:09.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.305+0000 7f402c2c5700 1 -- 172.21.15.69:0/2512463739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 msgr2=0x7f402419b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:09.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.305+0000 7f402c2c5700 1 --2- 172.21.15.69:0/2512463739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 0x7f402419b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4020009130 tx=0x7f4020011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:09.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.305+0000 7f402c2c5700 1 -- 172.21.15.69:0/2512463739 shutdown_connections 2023-07-06T20:10:09.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.305+0000 7f402c2c5700 1 --2- 172.21.15.69:0/2512463739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f402407ac60 0x7f402419b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:09.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.305+0000 7f402c2c5700 1 -- 172.21.15.69:0/2512463739 >> 172.21.15.69:0/2512463739 conn(0x7f4024100b90 msgr2=0x7f40241022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:09.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.305+0000 7f402c2c5700 1 -- 172.21.15.69:0/2512463739 shutdown_connections 2023-07-06T20:10:09.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:09.305+0000 7f402c2c5700 1 -- 172.21.15.69:0/2512463739 wait complete. 2023-07-06T20:10:09.708 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (2/15)... 2023-07-06T20:10:10.009 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:09 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:09.552+0000 7f5b66f21200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T20:10:10.009 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:09 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:09.751+0000 7f5b66f21200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T20:10:10.508 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:10 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:10.087+0000 7f5b66f21200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T20:10:11.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:10 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:10.992+0000 7f5b66f21200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T20:10:11.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:11 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:11.162+0000 7f5b66f21200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T20:10:11.758 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:11 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:11.399+0000 7f5b66f21200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T20:10:11.759 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:11 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:11.542+0000 7f5b66f21200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T20:10:12.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:11 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:11.815+0000 7f5b66f21200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T20:10:12.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:10:12.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2023-07-06T20:10:12.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "e3e8ec0c-1c38-11ee-9b2f-001a4aab830c", 2023-07-06T20:10:12.773 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2023-07-06T20:10:12.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-06T20:10:12.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-06T20:10:12.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-06T20:10:12.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:12.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-06T20:10:12.774 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-06T20:10:12.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2023-07-06T20:10:12.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2023-07-06T20:10:12.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-06T20:10:12.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2023-07-06T20:10:12.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2023-07-06T20:10:12.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2023-07-06T20:10:12.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-06T20:10:12.775 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:12.776 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-06T20:10:12.776 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-06T20:10:12.776 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:12.776 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-06T20:10:12.776 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:12.776 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-06T20:10:12.777 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-06T20:10:12.777 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-06T20:10:12.783 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-06T20:10:12.783 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-06T20:10:12.784 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-06T20:10:12.784 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:12.784 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-06T20:10:12.785 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-06T20:10:12.785 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-06T20:10:12.785 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-06T20:10:12.785 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-06T20:10:12.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-06T20:10:12.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-06T20:10:12.786 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-06T20:10:12.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-06T20:10:12.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:12.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-06T20:10:12.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:12.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-06T20:10:12.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-06T20:10:12.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:12.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-06T20:10:12.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-06T20:10:12.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-06T20:10:12.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-06T20:10:12.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-06T20:10:12.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-06T20:10:12.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2023-07-06T20:10:12.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2023-07-06T20:10:12.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-06T20:10:12.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:12.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-06T20:10:12.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:12.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2023-07-06T20:09:56.507323+0000", 2023-07-06T20:10:12.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-06T20:10:12.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:12.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-06T20:10:12.792 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2023-07-06T20:10:12.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.220+0000 7f58344c2700 1 Processor -- start 2023-07-06T20:10:12.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.220+0000 7f58344c2700 1 -- start start 2023-07-06T20:10:12.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.221+0000 7f58344c2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 0x7f582c105950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:12.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.221+0000 7f58344c2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c105e90 con 0x7f582c105540 2023-07-06T20:10:12.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.221+0000 7f583225e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 0x7f582c105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:12.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.221+0000 7f583225e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 0x7f582c105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60952/0 (socket says 172.21.15.69:60952) 2023-07-06T20:10:12.793 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.221+0000 7f583225e700 1 -- 172.21.15.69:0/2363485615 learned_addr learned my addr 172.21.15.69:0/2363485615 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:12.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.221+0000 7f583225e700 1 -- 172.21.15.69:0/2363485615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c105fd0 con 0x7f582c105540 2023-07-06T20:10:12.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.221+0000 7f583225e700 1 --2- 172.21.15.69:0/2363485615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 0x7f582c105950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f58280098a0 tx=0x7f5828009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5843e42bc163d9db server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:12.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.222+0000 7f583125c700 1 -- 172.21.15.69:0/2363485615 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5828004160 con 0x7f582c105540 2023-07-06T20:10:12.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.222+0000 7f583125c700 1 -- 172.21.15.69:0/2363485615 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f58280042c0 con 0x7f582c105540 2023-07-06T20:10:12.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.222+0000 7f583125c700 1 -- 172.21.15.69:0/2363485615 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5828011a00 con 0x7f582c105540 2023-07-06T20:10:12.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.222+0000 7f58344c2700 1 -- 172.21.15.69:0/2363485615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 msgr2=0x7f582c105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:12.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.222+0000 7f58344c2700 1 --2- 172.21.15.69:0/2363485615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 0x7f582c105950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f58280098a0 tx=0x7f5828009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:12.794 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.222+0000 7f58344c2700 1 -- 172.21.15.69:0/2363485615 shutdown_connections 2023-07-06T20:10:12.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.222+0000 7f58344c2700 1 --2- 172.21.15.69:0/2363485615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 0x7f582c105950 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:12.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.222+0000 7f58344c2700 1 -- 172.21.15.69:0/2363485615 >> 172.21.15.69:0/2363485615 conn(0x7f582c100b90 msgr2=0x7f582c102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:12.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.223+0000 7f58344c2700 1 -- 172.21.15.69:0/2363485615 shutdown_connections 2023-07-06T20:10:12.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.223+0000 7f58344c2700 1 -- 172.21.15.69:0/2363485615 wait complete. 2023-07-06T20:10:12.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.223+0000 7f58344c2700 1 Processor -- start 2023-07-06T20:10:12.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.224+0000 7f58344c2700 1 -- start start 2023-07-06T20:10:12.795 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.224+0000 7f58344c2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 0x7f582c19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:12.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.224+0000 7f58344c2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c19baa0 con 0x7f582c105540 2023-07-06T20:10:12.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.224+0000 7f583225e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 0x7f582c19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:12.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.224+0000 7f583225e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 0x7f582c19b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60956/0 (socket says 172.21.15.69:60956) 2023-07-06T20:10:12.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.224+0000 7f583225e700 1 -- 172.21.15.69:0/2228188487 learned_addr learned my addr 172.21.15.69:0/2228188487 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:12.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.224+0000 7f583225e700 1 -- 172.21.15.69:0/2228188487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5828009550 con 0x7f582c105540 2023-07-06T20:10:12.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.225+0000 7f583225e700 1 --2- 172.21.15.69:0/2228188487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 0x7f582c19b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5828009130 tx=0x7f5828011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:12.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.225+0000 7f58237fe700 1 -- 172.21.15.69:0/2228188487 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5828003b70 con 0x7f582c105540 2023-07-06T20:10:12.796 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.225+0000 7f58344c2700 1 -- 172.21.15.69:0/2228188487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c19bca0 con 0x7f582c105540 2023-07-06T20:10:12.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.225+0000 7f58237fe700 1 -- 172.21.15.69:0/2228188487 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5828003cd0 con 0x7f582c105540 2023-07-06T20:10:12.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.225+0000 7f58237fe700 1 -- 172.21.15.69:0/2228188487 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58280197d0 con 0x7f582c105540 2023-07-06T20:10:12.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.225+0000 7f58344c2700 1 -- 172.21.15.69:0/2228188487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c19c0c0 con 0x7f582c105540 2023-07-06T20:10:12.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.225+0000 7f58237fe700 1 -- 172.21.15.69:0/2228188487 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5828019a10 con 0x7f582c105540 2023-07-06T20:10:12.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.226+0000 7f58237fe700 1 -- 172.21.15.69:0/2228188487 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f582800b720 con 0x7f582c105540 2023-07-06T20:10:12.797 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.226+0000 7f58344c2700 1 -- 172.21.15.69:0/2228188487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c061c70 con 0x7f582c105540 2023-07-06T20:10:12.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.229+0000 7f58237fe700 1 -- 172.21.15.69:0/2228188487 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f582802a060 con 0x7f582c105540 2023-07-06T20:10:12.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.303+0000 7f58344c2700 1 -- 172.21.15.69:0/2228188487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f582c19f090 con 0x7f582c105540 2023-07-06T20:10:12.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.303+0000 7f58237fe700 1 -- 172.21.15.69:0/2228188487 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f5828019e00 con 0x7f582c105540 2023-07-06T20:10:12.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.305+0000 7f58344c2700 1 -- 172.21.15.69:0/2228188487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 msgr2=0x7f582c19b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:12.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.305+0000 7f58344c2700 1 --2- 172.21.15.69:0/2228188487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 0x7f582c19b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5828009130 tx=0x7f5828011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:12.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.305+0000 7f58344c2700 1 -- 172.21.15.69:0/2228188487 shutdown_connections 2023-07-06T20:10:12.798 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.305+0000 7f58344c2700 1 --2- 172.21.15.69:0/2228188487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f582c105540 0x7f582c19b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:12.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.305+0000 7f58344c2700 1 -- 172.21.15.69:0/2228188487 >> 172.21.15.69:0/2228188487 conn(0x7f582c100b90 msgr2=0x7f582c102290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:12.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.305+0000 7f58344c2700 1 -- 172.21.15.69:0/2228188487 shutdown_connections 2023-07-06T20:10:12.799 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:12.305+0000 7f58344c2700 1 -- 172.21.15.69:0/2228188487 wait complete. 2023-07-06T20:10:12.799 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (3/15)... 2023-07-06T20:10:13.009 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:12 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:12.915+0000 7f5b66f21200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T20:10:13.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2228188487' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-06T20:10:13.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: Activating manager daemon x 2023-07-06T20:10:13.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: mgrmap e2: x(active, starting, since 0.00420959s) 2023-07-06T20:10:13.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: from='mgr.14100 172.21.15.69:0/1238620345' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-06T20:10:13.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: from='mgr.14100 172.21.15.69:0/1238620345' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-06T20:10:13.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: from='mgr.14100 172.21.15.69:0/1238620345' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-06T20:10:13.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: from='mgr.14100 172.21.15.69:0/1238620345' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:10:13.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: from='mgr.14100 172.21.15.69:0/1238620345' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-06T20:10:13.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: Manager daemon x is now available 2023-07-06T20:10:13.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: from='mgr.14100 172.21.15.69:0/1238620345' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-06T20:10:13.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: from='mgr.14100 172.21.15.69:0/1238620345' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-06T20:10:13.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: from='mgr.14100 172.21.15.69:0/1238620345' entity='mgr.x' 2023-07-06T20:10:13.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: from='mgr.14100 172.21.15.69:0/1238620345' entity='mgr.x' 2023-07-06T20:10:13.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:13 smithi069 ceph-mon[126399]: from='mgr.14100 172.21.15.69:0/1238620345' entity='mgr.x' 2023-07-06T20:10:15.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:14 smithi069 ceph-mon[126399]: mgrmap e3: x(active, since 1.00886s) 2023-07-06T20:10:16.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:15 smithi069 ceph-mon[126399]: mgrmap e4: x(active, since 2s) 2023-07-06T20:10:20.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:20 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1309748521' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-06T20:10:20.873 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:10:20.875 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2023-07-06T20:10:20.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "e3e8ec0c-1c38-11ee-9b2f-001a4aab830c", 2023-07-06T20:10:20.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2023-07-06T20:10:20.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-06T20:10:20.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-06T20:10:20.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-06T20:10:20.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:20.876 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-06T20:10:20.877 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-06T20:10:20.877 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2023-07-06T20:10:20.877 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2023-07-06T20:10:20.877 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-06T20:10:20.877 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2023-07-06T20:10:20.877 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2023-07-06T20:10:20.877 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2023-07-06T20:10:20.877 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-06T20:10:20.878 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:20.878 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-06T20:10:20.878 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-06T20:10:20.878 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:20.878 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-06T20:10:20.878 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:20.878 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-06T20:10:20.879 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-06T20:10:20.884 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-06T20:10:20.885 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-06T20:10:20.885 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-06T20:10:20.885 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-06T20:10:20.885 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:20.885 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-06T20:10:20.885 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-06T20:10:20.885 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-06T20:10:20.886 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-06T20:10:20.886 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-06T20:10:20.886 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-06T20:10:20.886 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-06T20:10:20.886 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-06T20:10:20.886 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-06T20:10:20.886 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:20.886 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-06T20:10:20.887 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:20.887 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-06T20:10:20.887 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-06T20:10:20.887 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:20.887 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-06T20:10:20.887 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-06T20:10:20.887 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-06T20:10:20.888 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-06T20:10:20.888 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-06T20:10:20.888 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-06T20:10:20.888 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2023-07-06T20:10:20.888 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2023-07-06T20:10:20.888 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-06T20:10:20.888 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:20.888 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-06T20:10:20.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-06T20:10:20.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2023-07-06T20:09:56.507323+0000", 2023-07-06T20:10:20.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-06T20:10:20.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2023-07-06T20:10:20.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-06T20:10:20.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2023-07-06T20:10:20.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.159+0000 7f3f28af2700 1 Processor -- start 2023-07-06T20:10:20.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.159+0000 7f3f28af2700 1 -- start start 2023-07-06T20:10:20.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.159+0000 7f3f28af2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 0x7f3f24079160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:20.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.159+0000 7f3f28af2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f240796a0 con 0x7f3f2407ac60 2023-07-06T20:10:20.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.160+0000 7f3f2259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 0x7f3f24079160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:20.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.160+0000 7f3f2259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 0x7f3f24079160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56276/0 (socket says 172.21.15.69:56276) 2023-07-06T20:10:20.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.160+0000 7f3f2259c700 1 -- 172.21.15.69:0/3114061750 learned_addr learned my addr 172.21.15.69:0/3114061750 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:20.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.160+0000 7f3f2259c700 1 -- 172.21.15.69:0/3114061750 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240797e0 con 0x7f3f2407ac60 2023-07-06T20:10:20.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.160+0000 7f3f2259c700 1 --2- 172.21.15.69:0/3114061750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 0x7f3f24079160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f140098a0 tx=0x7f3f14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e89c5dbd41533c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:20.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.161+0000 7f3f2159a700 1 -- 172.21.15.69:0/3114061750 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f14004160 con 0x7f3f2407ac60 2023-07-06T20:10:20.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.161+0000 7f3f2159a700 1 -- 172.21.15.69:0/3114061750 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f140042c0 con 0x7f3f2407ac60 2023-07-06T20:10:20.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.161+0000 7f3f2159a700 1 -- 172.21.15.69:0/3114061750 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f14011a00 con 0x7f3f2407ac60 2023-07-06T20:10:20.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.161+0000 7f3f28af2700 1 -- 172.21.15.69:0/3114061750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 msgr2=0x7f3f24079160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:20.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.161+0000 7f3f28af2700 1 --2- 172.21.15.69:0/3114061750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 0x7f3f24079160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f140098a0 tx=0x7f3f14009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:20.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.161+0000 7f3f28af2700 1 -- 172.21.15.69:0/3114061750 shutdown_connections 2023-07-06T20:10:20.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.161+0000 7f3f28af2700 1 --2- 172.21.15.69:0/3114061750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 0x7f3f24079160 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:20.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.161+0000 7f3f28af2700 1 -- 172.21.15.69:0/3114061750 >> 172.21.15.69:0/3114061750 conn(0x7f3f24100b90 msgr2=0x7f3f24102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:20.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.161+0000 7f3f28af2700 1 -- 172.21.15.69:0/3114061750 shutdown_connections 2023-07-06T20:10:20.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.161+0000 7f3f28af2700 1 -- 172.21.15.69:0/3114061750 wait complete. 2023-07-06T20:10:20.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.162+0000 7f3f28af2700 1 Processor -- start 2023-07-06T20:10:20.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.163+0000 7f3f28af2700 1 -- start start 2023-07-06T20:10:20.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.163+0000 7f3f28af2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 0x7f3f2419b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:20.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.163+0000 7f3f28af2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2419baa0 con 0x7f3f2407ac60 2023-07-06T20:10:20.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.163+0000 7f3f2259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 0x7f3f2419b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:20.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.163+0000 7f3f2259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 0x7f3f2419b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56290/0 (socket says 172.21.15.69:56290) 2023-07-06T20:10:20.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.163+0000 7f3f2259c700 1 -- 172.21.15.69:0/1309748521 learned_addr learned my addr 172.21.15.69:0/1309748521 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:20.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.164+0000 7f3f2259c700 1 -- 172.21.15.69:0/1309748521 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f14009550 con 0x7f3f2407ac60 2023-07-06T20:10:20.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.164+0000 7f3f2259c700 1 --2- 172.21.15.69:0/1309748521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 0x7f3f2419b560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009130 tx=0x7f3f14011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:20.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.164+0000 7f3f137fe700 1 -- 172.21.15.69:0/1309748521 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f14003ab0 con 0x7f3f2407ac60 2023-07-06T20:10:20.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.164+0000 7f3f137fe700 1 -- 172.21.15.69:0/1309748521 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f14003c10 con 0x7f3f2407ac60 2023-07-06T20:10:20.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.164+0000 7f3f137fe700 1 -- 172.21.15.69:0/1309748521 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f140247d0 con 0x7f3f2407ac60 2023-07-06T20:10:20.894 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.164+0000 7f3f28af2700 1 -- 172.21.15.69:0/1309748521 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2419bca0 con 0x7f3f2407ac60 2023-07-06T20:10:20.894 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.164+0000 7f3f28af2700 1 -- 172.21.15.69:0/1309748521 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2419c0c0 con 0x7f3f2407ac60 2023-07-06T20:10:20.894 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.165+0000 7f3f137fe700 1 -- 172.21.15.69:0/1309748521 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f3f1401abb0 con 0x7f3f2407ac60 2023-07-06T20:10:20.894 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.165+0000 7f3f28af2700 1 -- 172.21.15.69:0/1309748521 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f24195510 con 0x7f3f2407ac60 2023-07-06T20:10:20.894 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.166+0000 7f3f137fe700 1 --2- 172.21.15.69:0/1309748521 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f3f0c038710 0x7f3f0c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:20.894 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.166+0000 7f3f137fe700 1 -- 172.21.15.69:0/1309748521 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3f1404c3e0 con 0x7f3f2407ac60 2023-07-06T20:10:20.894 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.166+0000 7f3f21d9b700 1 --2- 172.21.15.69:0/1309748521 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f3f0c038710 0x7f3f0c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:20.894 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.167+0000 7f3f21d9b700 1 --2- 172.21.15.69:0/1309748521 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f3f0c038710 0x7f3f0c03abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3f18006fd0 tx=0x7f3f18006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:20.895 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.171+0000 7f3f137fe700 1 -- 172.21.15.69:0/1309748521 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3f14018710 con 0x7f3f2407ac60 2023-07-06T20:10:20.895 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.439+0000 7f3f28af2700 1 -- 172.21.15.69:0/1309748521 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3f24061c70 con 0x7f3f2407ac60 2023-07-06T20:10:20.895 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.440+0000 7f3f137fe700 1 -- 172.21.15.69:0/1309748521 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f3f14018350 con 0x7f3f2407ac60 2023-07-06T20:10:20.895 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.442+0000 7f3f28af2700 1 -- 172.21.15.69:0/1309748521 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f3f0c038710 msgr2=0x7f3f0c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:20.895 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.442+0000 7f3f28af2700 1 --2- 172.21.15.69:0/1309748521 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f3f0c038710 0x7f3f0c03abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3f18006fd0 tx=0x7f3f18006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:20.895 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.442+0000 7f3f28af2700 1 -- 172.21.15.69:0/1309748521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 msgr2=0x7f3f2419b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:20.895 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.442+0000 7f3f28af2700 1 --2- 172.21.15.69:0/1309748521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 0x7f3f2419b560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009130 tx=0x7f3f14011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:20.896 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.442+0000 7f3f28af2700 1 -- 172.21.15.69:0/1309748521 shutdown_connections 2023-07-06T20:10:20.896 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.442+0000 7f3f28af2700 1 --2- 172.21.15.69:0/1309748521 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f3f0c038710 0x7f3f0c03abc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:20.896 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.442+0000 7f3f28af2700 1 --2- 172.21.15.69:0/1309748521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2407ac60 0x7f3f2419b560 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:20.896 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.442+0000 7f3f28af2700 1 -- 172.21.15.69:0/1309748521 >> 172.21.15.69:0/1309748521 conn(0x7f3f24100b90 msgr2=0x7f3f241022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:20.896 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.443+0000 7f3f28af2700 1 -- 172.21.15.69:0/1309748521 shutdown_connections 2023-07-06T20:10:20.896 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:20.443+0000 7f3f28af2700 1 -- 172.21.15.69:0/1309748521 wait complete. 2023-07-06T20:10:20.896 INFO:teuthology.orchestra.run.smithi069.stdout:mgr is available 2023-07-06T20:10:22.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:21 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3336348937' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-06T20:10:22.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:10:22.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [global] 2023-07-06T20:10:22.058 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout fsid = e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:10:22.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-06T20:10:22.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-06T20:10:22.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-06T20:10:22.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-06T20:10:22.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:10:22.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-06T20:10:22.059 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-06T20:10:22.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2023-07-06T20:10:22.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [osd] 2023-07-06T20:10:22.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-06T20:10:22.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-06T20:10:22.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-06T20:10:22.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-06T20:10:22.060 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-06T20:10:22.061 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.406+0000 7f236cbb4700 1 Processor -- start 2023-07-06T20:10:22.061 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.406+0000 7f236cbb4700 1 -- start start 2023-07-06T20:10:22.061 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.407+0000 7f236cbb4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 0x7f2368105950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:22.061 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.407+0000 7f236cbb4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2368105e90 con 0x7f2368105540 2023-07-06T20:10:22.061 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.407+0000 7f236659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 0x7f2368105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:22.061 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.407+0000 7f236659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 0x7f2368105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56300/0 (socket says 172.21.15.69:56300) 2023-07-06T20:10:22.061 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.407+0000 7f236659c700 1 -- 172.21.15.69:0/1873543304 learned_addr learned my addr 172.21.15.69:0/1873543304 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:22.061 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.408+0000 7f236659c700 1 -- 172.21.15.69:0/1873543304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368105fd0 con 0x7f2368105540 2023-07-06T20:10:22.062 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.408+0000 7f236659c700 1 --2- 172.21.15.69:0/1873543304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 0x7f2368105950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f23580098a0 tx=0x7f2358009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d7bf3fad3ed5c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:22.062 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.408+0000 7f236559a700 1 -- 172.21.15.69:0/1873543304 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2358004160 con 0x7f2368105540 2023-07-06T20:10:22.062 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.408+0000 7f236559a700 1 -- 172.21.15.69:0/1873543304 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f23580042c0 con 0x7f2368105540 2023-07-06T20:10:22.062 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.408+0000 7f236559a700 1 -- 172.21.15.69:0/1873543304 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2358011a00 con 0x7f2368105540 2023-07-06T20:10:22.062 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.409+0000 7f236cbb4700 1 -- 172.21.15.69:0/1873543304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 msgr2=0x7f2368105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:22.062 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.409+0000 7f236cbb4700 1 --2- 172.21.15.69:0/1873543304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 0x7f2368105950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f23580098a0 tx=0x7f2358009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:22.062 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.409+0000 7f236cbb4700 1 -- 172.21.15.69:0/1873543304 shutdown_connections 2023-07-06T20:10:22.063 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.409+0000 7f236cbb4700 1 --2- 172.21.15.69:0/1873543304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 0x7f2368105950 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:22.063 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.409+0000 7f236cbb4700 1 -- 172.21.15.69:0/1873543304 >> 172.21.15.69:0/1873543304 conn(0x7f2368100b90 msgr2=0x7f2368102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:22.063 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.409+0000 7f236cbb4700 1 -- 172.21.15.69:0/1873543304 shutdown_connections 2023-07-06T20:10:22.063 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.409+0000 7f236cbb4700 1 -- 172.21.15.69:0/1873543304 wait complete. 2023-07-06T20:10:22.063 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.410+0000 7f236cbb4700 1 Processor -- start 2023-07-06T20:10:22.063 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.410+0000 7f236cbb4700 1 -- start start 2023-07-06T20:10:22.063 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.410+0000 7f236cbb4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 0x7f236807aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:22.063 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.410+0000 7f236cbb4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236807b010 con 0x7f2368105540 2023-07-06T20:10:22.064 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.410+0000 7f236659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 0x7f236807aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:22.064 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.411+0000 7f236659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 0x7f236807aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56314/0 (socket says 172.21.15.69:56314) 2023-07-06T20:10:22.064 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.411+0000 7f236659c700 1 -- 172.21.15.69:0/3336348937 learned_addr learned my addr 172.21.15.69:0/3336348937 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:22.064 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.411+0000 7f236659c700 1 -- 172.21.15.69:0/3336348937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2358009550 con 0x7f2368105540 2023-07-06T20:10:22.064 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.411+0000 7f236659c700 1 --2- 172.21.15.69:0/3336348937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 0x7f236807aad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2358009fb0 tx=0x7f2358011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:22.064 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.411+0000 7f23577fe700 1 -- 172.21.15.69:0/3336348937 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2358003a40 con 0x7f2368105540 2023-07-06T20:10:22.064 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.411+0000 7f236cbb4700 1 -- 172.21.15.69:0/3336348937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2368079120 con 0x7f2368105540 2023-07-06T20:10:22.065 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.411+0000 7f23577fe700 1 -- 172.21.15.69:0/3336348937 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2358003ba0 con 0x7f2368105540 2023-07-06T20:10:22.065 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.411+0000 7f23577fe700 1 -- 172.21.15.69:0/3336348937 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f235801a750 con 0x7f2368105540 2023-07-06T20:10:22.065 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.412+0000 7f236cbb4700 1 -- 172.21.15.69:0/3336348937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2368079540 con 0x7f2368105540 2023-07-06T20:10:22.065 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.412+0000 7f23577fe700 1 -- 172.21.15.69:0/3336348937 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f2358024b60 con 0x7f2368105540 2023-07-06T20:10:22.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.412+0000 7f236cbb4700 1 -- 172.21.15.69:0/3336348937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2368061c70 con 0x7f2368105540 2023-07-06T20:10:22.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.413+0000 7f23577fe700 1 --2- 172.21.15.69:0/3336348937 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f2350038670 0x7f235003ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:22.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.413+0000 7f23577fe700 1 -- 172.21.15.69:0/3336348937 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f235804c370 con 0x7f2368105540 2023-07-06T20:10:22.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.413+0000 7f2365d9b700 1 --2- 172.21.15.69:0/3336348937 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f2350038670 0x7f235003ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:22.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.414+0000 7f2365d9b700 1 --2- 172.21.15.69:0/3336348937 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f2350038670 0x7f235003ab20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f235c006fd0 tx=0x7f235c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:22.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.418+0000 7f23577fe700 1 -- 172.21.15.69:0/3336348937 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2358018350 con 0x7f2368105540 2023-07-06T20:10:22.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.610+0000 7f236cbb4700 1 -- 172.21.15.69:0/3336348937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f2368107400 con 0x7f2368105540 2023-07-06T20:10:22.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.611+0000 7f23577fe700 1 -- 172.21.15.69:0/3336348937 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f2358018350 con 0x7f2368105540 2023-07-06T20:10:22.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.614+0000 7f236cbb4700 1 -- 172.21.15.69:0/3336348937 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f2350038670 msgr2=0x7f235003ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:22.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.614+0000 7f236cbb4700 1 --2- 172.21.15.69:0/3336348937 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f2350038670 0x7f235003ab20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f235c006fd0 tx=0x7f235c006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:22.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.614+0000 7f236cbb4700 1 -- 172.21.15.69:0/3336348937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 msgr2=0x7f236807aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:22.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.614+0000 7f236cbb4700 1 --2- 172.21.15.69:0/3336348937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 0x7f236807aad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2358009fb0 tx=0x7f2358011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:22.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.614+0000 7f236cbb4700 1 -- 172.21.15.69:0/3336348937 shutdown_connections 2023-07-06T20:10:22.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.614+0000 7f236cbb4700 1 --2- 172.21.15.69:0/3336348937 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f2350038670 0x7f235003ab20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:22.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.614+0000 7f236cbb4700 1 --2- 172.21.15.69:0/3336348937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2368105540 0x7f236807aad0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:22.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.614+0000 7f236cbb4700 1 -- 172.21.15.69:0/3336348937 >> 172.21.15.69:0/3336348937 conn(0x7f2368100b90 msgr2=0x7f2368102290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:22.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.614+0000 7f236cbb4700 1 -- 172.21.15.69:0/3336348937 shutdown_connections 2023-07-06T20:10:22.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:21.614+0000 7f236cbb4700 1 -- 172.21.15.69:0/3336348937 wait complete. 2023-07-06T20:10:22.074 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling cephadm module... 2023-07-06T20:10:23.311 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:23 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/338472984' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-06T20:10:23.311 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:23 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: ignoring --setuser ceph since I am not root 2023-07-06T20:10:23.312 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:23 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: ignoring --setgroup ceph since I am not root 2023-07-06T20:10:23.443 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.567+0000 7f6854607700 1 Processor -- start 2023-07-06T20:10:23.443 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.567+0000 7f6854607700 1 -- start start 2023-07-06T20:10:23.443 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.568+0000 7f6854607700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 0x7f684c0791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:23.443 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.568+0000 7f6854607700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0796e0 con 0x7f684c07aca0 2023-07-06T20:10:23.443 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.568+0000 7f68523a3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 0x7f684c0791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:23.443 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.568+0000 7f68523a3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 0x7f684c0791a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56318/0 (socket says 172.21.15.69:56318) 2023-07-06T20:10:23.444 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.568+0000 7f68523a3700 1 -- 172.21.15.69:0/1049263844 learned_addr learned my addr 172.21.15.69:0/1049263844 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:23.444 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.568+0000 7f68523a3700 1 -- 172.21.15.69:0/1049263844 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c079820 con 0x7f684c07aca0 2023-07-06T20:10:23.444 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.569+0000 7f68523a3700 1 --2- 172.21.15.69:0/1049263844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 0x7f684c0791a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f68480098a0 tx=0x7f6848009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa51423df35b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:23.444 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.569+0000 7f68513a1700 1 -- 172.21.15.69:0/1049263844 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6848004160 con 0x7f684c07aca0 2023-07-06T20:10:23.444 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.569+0000 7f68513a1700 1 -- 172.21.15.69:0/1049263844 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f68480042c0 con 0x7f684c07aca0 2023-07-06T20:10:23.444 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.569+0000 7f68513a1700 1 -- 172.21.15.69:0/1049263844 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6848011a00 con 0x7f684c07aca0 2023-07-06T20:10:23.444 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.569+0000 7f6854607700 1 -- 172.21.15.69:0/1049263844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 msgr2=0x7f684c0791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:23.445 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.569+0000 7f6854607700 1 --2- 172.21.15.69:0/1049263844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 0x7f684c0791a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f68480098a0 tx=0x7f6848009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:23.445 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.570+0000 7f6854607700 1 -- 172.21.15.69:0/1049263844 shutdown_connections 2023-07-06T20:10:23.445 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.570+0000 7f6854607700 1 --2- 172.21.15.69:0/1049263844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 0x7f684c0791a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:23.445 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.570+0000 7f6854607700 1 -- 172.21.15.69:0/1049263844 >> 172.21.15.69:0/1049263844 conn(0x7f684c100b90 msgr2=0x7f684c102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:23.445 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.570+0000 7f6854607700 1 -- 172.21.15.69:0/1049263844 shutdown_connections 2023-07-06T20:10:23.445 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.570+0000 7f6854607700 1 -- 172.21.15.69:0/1049263844 wait complete. 2023-07-06T20:10:23.445 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.571+0000 7f6854607700 1 Processor -- start 2023-07-06T20:10:23.445 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.571+0000 7f6854607700 1 -- start start 2023-07-06T20:10:23.446 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.571+0000 7f6854607700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 0x7f684c19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:23.446 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.571+0000 7f6854607700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c19bb00 con 0x7f684c07aca0 2023-07-06T20:10:23.446 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.572+0000 7f68523a3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 0x7f684c19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:23.446 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.572+0000 7f68523a3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 0x7f684c19b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56332/0 (socket says 172.21.15.69:56332) 2023-07-06T20:10:23.446 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.572+0000 7f68523a3700 1 -- 172.21.15.69:0/338472984 learned_addr learned my addr 172.21.15.69:0/338472984 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:23.446 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.572+0000 7f68523a3700 1 -- 172.21.15.69:0/338472984 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848009550 con 0x7f684c07aca0 2023-07-06T20:10:23.446 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.572+0000 7f68523a3700 1 --2- 172.21.15.69:0/338472984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 0x7f684c19b5c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6848009130 tx=0x7f6848011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:23.447 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.572+0000 7f68437fe700 1 -- 172.21.15.69:0/338472984 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6848003ab0 con 0x7f684c07aca0 2023-07-06T20:10:23.447 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.572+0000 7f68437fe700 1 -- 172.21.15.69:0/338472984 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6848003c10 con 0x7f684c07aca0 2023-07-06T20:10:23.447 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.572+0000 7f6854607700 1 -- 172.21.15.69:0/338472984 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c19bd00 con 0x7f684c07aca0 2023-07-06T20:10:23.447 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.572+0000 7f68437fe700 1 -- 172.21.15.69:0/338472984 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68480247d0 con 0x7f684c07aca0 2023-07-06T20:10:23.447 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.573+0000 7f6854607700 1 -- 172.21.15.69:0/338472984 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c19c120 con 0x7f684c07aca0 2023-07-06T20:10:23.447 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.574+0000 7f6854607700 1 -- 172.21.15.69:0/338472984 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c195520 con 0x7f684c07aca0 2023-07-06T20:10:23.447 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.574+0000 7f68437fe700 1 -- 172.21.15.69:0/338472984 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f684801abb0 con 0x7f684c07aca0 2023-07-06T20:10:23.447 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.574+0000 7f68437fe700 1 --2- 172.21.15.69:0/338472984 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f6838038710 0x7f683803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:23.448 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.574+0000 7f68437fe700 1 -- 172.21.15.69:0/338472984 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f684804c3e0 con 0x7f684c07aca0 2023-07-06T20:10:23.448 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.574+0000 7f6851ba2700 1 --2- 172.21.15.69:0/338472984 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f6838038710 0x7f683803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:23.448 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.575+0000 7f6851ba2700 1 --2- 172.21.15.69:0/338472984 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f6838038710 0x7f683803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f683c006fd0 tx=0x7f683c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:23.448 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.579+0000 7f68437fe700 1 -- 172.21.15.69:0/338472984 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6848018710 con 0x7f684c07aca0 2023-07-06T20:10:23.448 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:22.825+0000 7f6854607700 1 -- 172.21.15.69:0/338472984 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f684c061c70 con 0x7f684c07aca0 2023-07-06T20:10:23.448 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.048+0000 7f68437fe700 1 -- 172.21.15.69:0/338472984 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f684802d770 con 0x7f684c07aca0 2023-07-06T20:10:23.448 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.050+0000 7f68437fe700 1 -- 172.21.15.69:0/338472984 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f684802ad60 con 0x7f684c07aca0 2023-07-06T20:10:23.449 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.053+0000 7f6854607700 1 -- 172.21.15.69:0/338472984 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f6838038710 msgr2=0x7f683803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:23.449 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.053+0000 7f6854607700 1 --2- 172.21.15.69:0/338472984 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f6838038710 0x7f683803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f683c006fd0 tx=0x7f683c006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:23.449 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.053+0000 7f6854607700 1 -- 172.21.15.69:0/338472984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 msgr2=0x7f684c19b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:23.449 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.053+0000 7f6854607700 1 --2- 172.21.15.69:0/338472984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 0x7f684c19b5c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6848009130 tx=0x7f6848011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:23.449 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.053+0000 7f6854607700 1 -- 172.21.15.69:0/338472984 shutdown_connections 2023-07-06T20:10:23.449 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.053+0000 7f6854607700 1 --2- 172.21.15.69:0/338472984 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7f6838038710 0x7f683803abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:23.449 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.053+0000 7f6854607700 1 --2- 172.21.15.69:0/338472984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684c07aca0 0x7f684c19b5c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:23.449 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.053+0000 7f6854607700 1 -- 172.21.15.69:0/338472984 >> 172.21.15.69:0/338472984 conn(0x7f684c100b90 msgr2=0x7f684c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:23.450 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.053+0000 7f6854607700 1 -- 172.21.15.69:0/338472984 shutdown_connections 2023-07-06T20:10:23.450 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.053+0000 7f6854607700 1 -- 172.21.15.69:0/338472984 wait complete. 2023-07-06T20:10:23.664 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:23 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:23.309+0000 7ffba3680200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T20:10:23.664 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:23 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:23.415+0000 7ffba3680200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T20:10:23.664 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:23 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:23.542+0000 7ffba3680200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T20:10:23.664 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:23 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:23.662+0000 7ffba3680200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T20:10:24.008 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:23 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:23.893+0000 7ffba3680200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T20:10:24.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:24 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/338472984' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-06T20:10:24.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:24 smithi069 ceph-mon[126399]: mgrmap e5: x(active, since 10s) 2023-07-06T20:10:24.593 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:24 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:24.301+0000 7ffba3680200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T20:10:24.593 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:24 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:24.428+0000 7ffba3680200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T20:10:24.594 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:24 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:24.591+0000 7ffba3680200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T20:10:24.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2023-07-06T20:10:24.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-06T20:10:24.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-06T20:10:24.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-06T20:10:24.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-06T20:10:24.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2023-07-06T20:10:24.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.984+0000 7efd73d00700 1 Processor -- start 2023-07-06T20:10:24.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.984+0000 7efd73d00700 1 -- start start 2023-07-06T20:10:24.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.984+0000 7efd73d00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 0x7efd6c079160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:24.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.984+0000 7efd73d00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c0796a0 con 0x7efd6c07ac60 2023-07-06T20:10:24.727 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.985+0000 7efd71a9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 0x7efd6c079160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:24.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.985+0000 7efd71a9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 0x7efd6c079160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47398/0 (socket says 172.21.15.69:47398) 2023-07-06T20:10:24.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.985+0000 7efd71a9c700 1 -- 172.21.15.69:0/2381976973 learned_addr learned my addr 172.21.15.69:0/2381976973 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:24.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.985+0000 7efd71a9c700 1 -- 172.21.15.69:0/2381976973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c0797e0 con 0x7efd6c07ac60 2023-07-06T20:10:24.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.985+0000 7efd71a9c700 1 --2- 172.21.15.69:0/2381976973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 0x7efd6c079160 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd680098a0 tx=0x7efd68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffcf625d1154d82 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:24.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.986+0000 7efd70a9a700 1 -- 172.21.15.69:0/2381976973 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd68004160 con 0x7efd6c07ac60 2023-07-06T20:10:24.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.986+0000 7efd70a9a700 1 -- 172.21.15.69:0/2381976973 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efd680042c0 con 0x7efd6c07ac60 2023-07-06T20:10:24.728 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.986+0000 7efd70a9a700 1 -- 172.21.15.69:0/2381976973 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd68011a00 con 0x7efd6c07ac60 2023-07-06T20:10:24.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.986+0000 7efd73d00700 1 -- 172.21.15.69:0/2381976973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 msgr2=0x7efd6c079160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:24.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.986+0000 7efd73d00700 1 --2- 172.21.15.69:0/2381976973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 0x7efd6c079160 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd680098a0 tx=0x7efd68009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:24.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.986+0000 7efd73d00700 1 -- 172.21.15.69:0/2381976973 shutdown_connections 2023-07-06T20:10:24.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.986+0000 7efd73d00700 1 --2- 172.21.15.69:0/2381976973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 0x7efd6c079160 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:24.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.986+0000 7efd73d00700 1 -- 172.21.15.69:0/2381976973 >> 172.21.15.69:0/2381976973 conn(0x7efd6c100b90 msgr2=0x7efd6c102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:24.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.986+0000 7efd73d00700 1 -- 172.21.15.69:0/2381976973 shutdown_connections 2023-07-06T20:10:24.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.986+0000 7efd73d00700 1 -- 172.21.15.69:0/2381976973 wait complete. 2023-07-06T20:10:24.729 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.987+0000 7efd73d00700 1 Processor -- start 2023-07-06T20:10:24.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.988+0000 7efd73d00700 1 -- start start 2023-07-06T20:10:24.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.988+0000 7efd73d00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 0x7efd6c19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:24.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.988+0000 7efd73d00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c19baa0 con 0x7efd6c07ac60 2023-07-06T20:10:24.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.988+0000 7efd71a9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 0x7efd6c19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:24.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.988+0000 7efd71a9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 0x7efd6c19b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47410/0 (socket says 172.21.15.69:47410) 2023-07-06T20:10:24.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.988+0000 7efd71a9c700 1 -- 172.21.15.69:0/3411839864 learned_addr learned my addr 172.21.15.69:0/3411839864 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:24.730 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.988+0000 7efd71a9c700 1 -- 172.21.15.69:0/3411839864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68009550 con 0x7efd6c07ac60 2023-07-06T20:10:24.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.989+0000 7efd71a9c700 1 --2- 172.21.15.69:0/3411839864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 0x7efd6c19b560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd68009130 tx=0x7efd68011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:24.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.989+0000 7efd62ffd700 1 -- 172.21.15.69:0/3411839864 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd68003ab0 con 0x7efd6c07ac60 2023-07-06T20:10:24.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.989+0000 7efd62ffd700 1 -- 172.21.15.69:0/3411839864 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efd68003c10 con 0x7efd6c07ac60 2023-07-06T20:10:24.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.989+0000 7efd73d00700 1 -- 172.21.15.69:0/3411839864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c19bca0 con 0x7efd6c07ac60 2023-07-06T20:10:24.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.989+0000 7efd62ffd700 1 -- 172.21.15.69:0/3411839864 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd680247d0 con 0x7efd6c07ac60 2023-07-06T20:10:24.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.989+0000 7efd73d00700 1 -- 172.21.15.69:0/3411839864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c19c0c0 con 0x7efd6c07ac60 2023-07-06T20:10:24.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.990+0000 7efd73d00700 1 -- 172.21.15.69:0/3411839864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6c195510 con 0x7efd6c07ac60 2023-07-06T20:10:24.731 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.990+0000 7efd62ffd700 1 -- 172.21.15.69:0/3411839864 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7efd6801a430 con 0x7efd6c07ac60 2023-07-06T20:10:24.732 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.991+0000 7efd62ffd700 1 --2- 172.21.15.69:0/3411839864 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7efd58038760 0x7efd5803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:24.732 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.991+0000 7efd62ffd700 1 -- 172.21.15.69:0/3411839864 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efd6804c450 con 0x7efd6c07ac60 2023-07-06T20:10:24.732 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.991+0000 7efd7129b700 1 -- 172.21.15.69:0/3411839864 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7efd58038760 msgr2=0x7efd5803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1829009377 2023-07-06T20:10:24.732 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.991+0000 7efd7129b700 1 --2- 172.21.15.69:0/3411839864 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7efd58038760 0x7efd5803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T20:10:24.732 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:23.996+0000 7efd62ffd700 1 -- 172.21.15.69:0/3411839864 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7efd68018b40 con 0x7efd6c07ac60 2023-07-06T20:10:24.732 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.192+0000 7efd7129b700 1 -- 172.21.15.69:0/3411839864 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7efd58038760 msgr2=0x7efd5803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1829009377 2023-07-06T20:10:24.732 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.192+0000 7efd7129b700 1 --2- 172.21.15.69:0/3411839864 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7efd58038760 0x7efd5803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-06T20:10:24.733 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.267+0000 7efd73d00700 1 -- 172.21.15.69:0/3411839864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7efd6c061c70 con 0x7efd6c07ac60 2023-07-06T20:10:24.733 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.268+0000 7efd62ffd700 1 -- 172.21.15.69:0/3411839864 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7efd68018b40 con 0x7efd6c07ac60 2023-07-06T20:10:24.733 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.270+0000 7efd73d00700 1 -- 172.21.15.69:0/3411839864 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7efd58038760 msgr2=0x7efd5803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T20:10:24.733 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.270+0000 7efd73d00700 1 --2- 172.21.15.69:0/3411839864 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7efd58038760 0x7efd5803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:24.733 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.270+0000 7efd73d00700 1 -- 172.21.15.69:0/3411839864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 msgr2=0x7efd6c19b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:24.733 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.270+0000 7efd73d00700 1 --2- 172.21.15.69:0/3411839864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 0x7efd6c19b560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd68009130 tx=0x7efd68011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:24.733 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.270+0000 7efd73d00700 1 -- 172.21.15.69:0/3411839864 shutdown_connections 2023-07-06T20:10:24.733 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.270+0000 7efd73d00700 1 --2- 172.21.15.69:0/3411839864 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7efd58038760 0x7efd5803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:24.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.270+0000 7efd73d00700 1 --2- 172.21.15.69:0/3411839864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd6c07ac60 0x7efd6c19b560 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:24.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.270+0000 7efd73d00700 1 -- 172.21.15.69:0/3411839864 >> 172.21.15.69:0/3411839864 conn(0x7efd6c100b90 msgr2=0x7efd6c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:24.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.270+0000 7efd73d00700 1 -- 172.21.15.69:0/3411839864 shutdown_connections 2023-07-06T20:10:24.734 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:24.270+0000 7efd73d00700 1 -- 172.21.15.69:0/3411839864 wait complete. 2023-07-06T20:10:24.734 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for the mgr to restart... 2023-07-06T20:10:24.734 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr epoch 5... 2023-07-06T20:10:24.992 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:24 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:24.714+0000 7ffba3680200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T20:10:25.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:25 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3411839864' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-06T20:10:25.758 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:25 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:25.273+0000 7ffba3680200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T20:10:26.258 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:25 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:25.995+0000 7ffba3680200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T20:10:26.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:26 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:26.201+0000 7ffba3680200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T20:10:27.465 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:27 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:27.123+0000 7ffba3680200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:10:27.466 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:27 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:27.241+0000 7ffba3680200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T20:10:27.758 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:27 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:27.463+0000 7ffba3680200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:10:28.092 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:27 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:27.818+0000 7ffba3680200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T20:10:28.508 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:28 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:28.090+0000 7ffba3680200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T20:10:28.509 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:28 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:28.279+0000 7ffba3680200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T20:10:29.008 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:28 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:28.617+0000 7ffba3680200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T20:10:29.940 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:29 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:29.525+0000 7ffba3680200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T20:10:29.940 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:29 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:29.696+0000 7ffba3680200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T20:10:30.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:29 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:29.937+0000 7ffba3680200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T20:10:30.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:30 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:30.083+0000 7ffba3680200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T20:10:30.758 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:30 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:30.355+0000 7ffba3680200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T20:10:31.758 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:31.402+0000 7ffba3680200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T20:10:31.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: Active manager daemon x restarted 2023-07-06T20:10:31.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: Activating manager daemon x 2023-07-06T20:10:31.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: osdmap e2: 0 total, 0 up, 0 in 2023-07-06T20:10:31.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: mgrmap e6: x(active, starting, since 0.00371819s) 2023-07-06T20:10:31.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:10:31.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-06T20:10:31.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-06T20:10:31.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-06T20:10:31.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-06T20:10:31.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: Manager daemon x is now available 2023-07-06T20:10:31.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:31.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:31.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:10:31.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:10:31.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:31 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:10:32.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:32 smithi069 ceph-mon[126399]: Found migration_current of "None". Setting to last migration. 2023-07-06T20:10:32.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:32 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-06T20:10:32.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:32 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-06T20:10:32.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:32 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:32.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:32 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:32.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:32 smithi069 ceph-mon[126399]: [06/Jul/2023:20:10:31] ENGINE Bus STARTING 2023-07-06T20:10:32.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:32 smithi069 ceph-mon[126399]: [06/Jul/2023:20:10:31] ENGINE Serving on http://172.21.15.69:8765 2023-07-06T20:10:32.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:32 smithi069 ceph-mon[126399]: [06/Jul/2023:20:10:32] ENGINE Serving on https://172.21.15.69:7150 2023-07-06T20:10:32.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:32 smithi069 ceph-mon[126399]: [06/Jul/2023:20:10:32] ENGINE Bus STARTED 2023-07-06T20:10:32.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:32 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:10:32.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:32 smithi069 ceph-mon[126399]: mgrmap e7: x(active, since 1.00714s) 2023-07-06T20:10:32.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2023-07-06T20:10:32.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-06T20:10:32.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-06T20:10:32.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2023-07-06T20:10:32.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.259+0000 7ffb95c9a700 1 Processor -- start 2023-07-06T20:10:32.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.260+0000 7ffb95c9a700 1 -- start start 2023-07-06T20:10:32.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.260+0000 7ffb95c9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 0x7ffb90079160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:32.825 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.260+0000 7ffb95c9a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb900796a0 con 0x7ffb9007ac60 2023-07-06T20:10:32.825 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.260+0000 7ffb8f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 0x7ffb90079160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:32.825 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.260+0000 7ffb8f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 0x7ffb90079160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47418/0 (socket says 172.21.15.69:47418) 2023-07-06T20:10:32.825 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.260+0000 7ffb8f7fe700 1 -- 172.21.15.69:0/4232347056 learned_addr learned my addr 172.21.15.69:0/4232347056 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:32.825 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.261+0000 7ffb8f7fe700 1 -- 172.21.15.69:0/4232347056 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb900797e0 con 0x7ffb9007ac60 2023-07-06T20:10:32.825 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.261+0000 7ffb8f7fe700 1 --2- 172.21.15.69:0/4232347056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 0x7ffb90079160 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb800098a0 tx=0x7ffb80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c29cab626e1581 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:32.825 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.261+0000 7ffb8e7fc700 1 -- 172.21.15.69:0/4232347056 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb80004160 con 0x7ffb9007ac60 2023-07-06T20:10:32.826 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.261+0000 7ffb8e7fc700 1 -- 172.21.15.69:0/4232347056 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb800042c0 con 0x7ffb9007ac60 2023-07-06T20:10:32.826 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.261+0000 7ffb8e7fc700 1 -- 172.21.15.69:0/4232347056 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb80011a00 con 0x7ffb9007ac60 2023-07-06T20:10:32.826 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.262+0000 7ffb95c9a700 1 -- 172.21.15.69:0/4232347056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 msgr2=0x7ffb90079160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:32.826 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.262+0000 7ffb95c9a700 1 --2- 172.21.15.69:0/4232347056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 0x7ffb90079160 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb800098a0 tx=0x7ffb80009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:32.826 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.262+0000 7ffb95c9a700 1 -- 172.21.15.69:0/4232347056 shutdown_connections 2023-07-06T20:10:32.826 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.262+0000 7ffb95c9a700 1 --2- 172.21.15.69:0/4232347056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 0x7ffb90079160 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:32.826 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.262+0000 7ffb95c9a700 1 -- 172.21.15.69:0/4232347056 >> 172.21.15.69:0/4232347056 conn(0x7ffb90100b90 msgr2=0x7ffb90102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:32.827 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.262+0000 7ffb95c9a700 1 -- 172.21.15.69:0/4232347056 shutdown_connections 2023-07-06T20:10:32.827 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.262+0000 7ffb95c9a700 1 -- 172.21.15.69:0/4232347056 wait complete. 2023-07-06T20:10:32.827 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.264+0000 7ffb95c9a700 1 Processor -- start 2023-07-06T20:10:32.827 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.264+0000 7ffb95c9a700 1 -- start start 2023-07-06T20:10:32.827 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.264+0000 7ffb95c9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 0x7ffb9019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:32.827 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.264+0000 7ffb95c9a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9019baa0 con 0x7ffb9007ac60 2023-07-06T20:10:32.827 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.265+0000 7ffb8f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 0x7ffb9019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:32.827 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.265+0000 7ffb8f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 0x7ffb9019b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47426/0 (socket says 172.21.15.69:47426) 2023-07-06T20:10:32.828 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.265+0000 7ffb8f7fe700 1 -- 172.21.15.69:0/1338450057 learned_addr learned my addr 172.21.15.69:0/1338450057 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:32.828 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.265+0000 7ffb8f7fe700 1 -- 172.21.15.69:0/1338450057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb80009550 con 0x7ffb9007ac60 2023-07-06T20:10:32.828 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.265+0000 7ffb8f7fe700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 0x7ffb9019b560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009130 tx=0x7ffb80011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:32.828 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.265+0000 7ffb8cff9700 1 -- 172.21.15.69:0/1338450057 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb80003ab0 con 0x7ffb9007ac60 2023-07-06T20:10:32.828 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.265+0000 7ffb8cff9700 1 -- 172.21.15.69:0/1338450057 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb80003c10 con 0x7ffb9007ac60 2023-07-06T20:10:32.828 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.265+0000 7ffb8cff9700 1 -- 172.21.15.69:0/1338450057 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb800247d0 con 0x7ffb9007ac60 2023-07-06T20:10:32.828 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.266+0000 7ffb95c9a700 1 -- 172.21.15.69:0/1338450057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9019bca0 con 0x7ffb9007ac60 2023-07-06T20:10:32.829 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.266+0000 7ffb95c9a700 1 -- 172.21.15.69:0/1338450057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9019c0c0 con 0x7ffb9007ac60 2023-07-06T20:10:32.829 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.267+0000 7ffb8cff9700 1 -- 172.21.15.69:0/1338450057 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ffb8001a430 con 0x7ffb9007ac60 2023-07-06T20:10:32.829 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.267+0000 7ffb8cff9700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 0x7ffb7803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:32.829 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.267+0000 7ffb8cff9700 1 -- 172.21.15.69:0/1338450057 --> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb7803b320 con 0x7ffb78038760 2023-07-06T20:10:32.829 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.267+0000 7ffb8cff9700 1 -- 172.21.15.69:0/1338450057 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffb8004c4a0 con 0x7ffb9007ac60 2023-07-06T20:10:32.829 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.267+0000 7ffb8effd700 1 -- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 msgr2=0x7ffb7803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1829009377 2023-07-06T20:10:32.829 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.267+0000 7ffb8effd700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 0x7ffb7803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T20:10:32.830 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.468+0000 7ffb8effd700 1 -- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 msgr2=0x7ffb7803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1829009377 2023-07-06T20:10:32.830 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.468+0000 7ffb8effd700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 0x7ffb7803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-06T20:10:32.830 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.869+0000 7ffb8effd700 1 -- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 msgr2=0x7ffb7803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1829009377 2023-07-06T20:10:32.830 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:25.869+0000 7ffb8effd700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 0x7ffb7803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-06T20:10:32.830 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:26.670+0000 7ffb8effd700 1 -- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 msgr2=0x7ffb7803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1829009377 2023-07-06T20:10:32.830 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:26.670+0000 7ffb8effd700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 0x7ffb7803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-06T20:10:32.830 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:28.271+0000 7ffb8effd700 1 -- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 msgr2=0x7ffb7803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1829009377 2023-07-06T20:10:32.830 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:28.271+0000 7ffb8effd700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 0x7ffb7803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-06T20:10:32.831 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:31.406+0000 7ffb8cff9700 1 -- 172.21.15.69:0/1338450057 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7ffb8001abb0 con 0x7ffb9007ac60 2023-07-06T20:10:32.831 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:31.406+0000 7ffb8cff9700 1 -- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 msgr2=0x7ffb7803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T20:10:32.831 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:31.406+0000 7ffb8cff9700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 0x7ffb7803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:32.831 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.408+0000 7ffb8cff9700 1 -- 172.21.15.69:0/1338450057 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7ffb8004e2b0 con 0x7ffb9007ac60 2023-07-06T20:10:32.831 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.409+0000 7ffb8cff9700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7ffb7803bfe0 0x7ffb7803e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:32.831 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.409+0000 7ffb8cff9700 1 -- 172.21.15.69:0/1338450057 --> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb7803b320 con 0x7ffb7803bfe0 2023-07-06T20:10:32.831 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.409+0000 7ffb8effd700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7ffb7803bfe0 0x7ffb7803e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:32.831 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.410+0000 7ffb8effd700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7ffb7803bfe0 0x7ffb7803e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffb84003a10 tx=0x7ffb840092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:32.832 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.411+0000 7ffb8cff9700 1 -- 172.21.15.69:0/1338450057 <== mgr.14120 v2:172.21.15.69:6800/9699389 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7ffb7803b320 con 0x7ffb7803bfe0 2023-07-06T20:10:32.832 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.418+0000 7ffb95c9a700 1 -- 172.21.15.69:0/1338450057 --> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ffb9019c8a0 con 0x7ffb7803bfe0 2023-07-06T20:10:32.832 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.419+0000 7ffb8cff9700 1 -- 172.21.15.69:0/1338450057 <== mgr.14120 v2:172.21.15.69:6800/9699389 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ffb9019c8a0 con 0x7ffb7803bfe0 2023-07-06T20:10:32.832 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.419+0000 7ffb95c9a700 1 -- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7ffb7803bfe0 msgr2=0x7ffb7803e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:32.832 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.419+0000 7ffb95c9a700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7ffb7803bfe0 0x7ffb7803e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffb84003a10 tx=0x7ffb840092b0 comp rx=0 tx=0).stop 2023-07-06T20:10:32.832 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.419+0000 7ffb95c9a700 1 -- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 msgr2=0x7ffb9019b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:32.832 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.419+0000 7ffb95c9a700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 0x7ffb9019b560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009130 tx=0x7ffb80011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:32.833 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.419+0000 7ffb95c9a700 1 -- 172.21.15.69:0/1338450057 shutdown_connections 2023-07-06T20:10:32.833 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.419+0000 7ffb95c9a700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9007ac60 0x7ffb9019b560 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:32.833 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.419+0000 7ffb95c9a700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7ffb7803bfe0 0x7ffb7803e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:32.833 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.419+0000 7ffb95c9a700 1 --2- 172.21.15.69:0/1338450057 >> [v2:172.21.15.69:6800/1829009377,v1:172.21.15.69:6801/1829009377] conn(0x7ffb78038760 0x7ffb7803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:32.833 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.419+0000 7ffb95c9a700 1 -- 172.21.15.69:0/1338450057 >> 172.21.15.69:0/1338450057 conn(0x7ffb90100b90 msgr2=0x7ffb901022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:32.833 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.419+0000 7ffb95c9a700 1 -- 172.21.15.69:0/1338450057 shutdown_connections 2023-07-06T20:10:32.833 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:32.419+0000 7ffb95c9a700 1 -- 172.21.15.69:0/1338450057 wait complete. 2023-07-06T20:10:32.834 INFO:teuthology.orchestra.run.smithi069.stdout:mgr epoch 5 is available 2023-07-06T20:10:32.834 INFO:teuthology.orchestra.run.smithi069.stdout:Setting orchestrator backend to cephadm... 2023-07-06T20:10:33.722 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:33 smithi069 ceph-mon[126399]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-06T20:10:33.722 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:33 smithi069 ceph-mon[126399]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-06T20:10:34.003 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.339+0000 7fc8706e9700 1 Processor -- start 2023-07-06T20:10:34.003 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.339+0000 7fc8706e9700 1 -- start start 2023-07-06T20:10:34.003 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.340+0000 7fc8706e9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 0x7fc8680791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:34.003 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.340+0000 7fc8706e9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc868079700 con 0x7fc86807acc0 2023-07-06T20:10:34.003 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.340+0000 7fc86e485700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 0x7fc8680791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:34.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.340+0000 7fc86e485700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 0x7fc8680791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58668/0 (socket says 172.21.15.69:58668) 2023-07-06T20:10:34.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.340+0000 7fc86e485700 1 -- 172.21.15.69:0/3787966166 learned_addr learned my addr 172.21.15.69:0/3787966166 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:34.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.340+0000 7fc86e485700 1 -- 172.21.15.69:0/3787966166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868079840 con 0x7fc86807acc0 2023-07-06T20:10:34.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.341+0000 7fc86e485700 1 --2- 172.21.15.69:0/3787966166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 0x7fc8680791c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc8640098a0 tx=0x7fc864009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb34e3fc8df7dfe3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:34.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.341+0000 7fc86d483700 1 -- 172.21.15.69:0/3787966166 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc864004160 con 0x7fc86807acc0 2023-07-06T20:10:34.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.341+0000 7fc86d483700 1 -- 172.21.15.69:0/3787966166 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8640042c0 con 0x7fc86807acc0 2023-07-06T20:10:34.004 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.341+0000 7fc86d483700 1 -- 172.21.15.69:0/3787966166 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc864011a00 con 0x7fc86807acc0 2023-07-06T20:10:34.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.341+0000 7fc8706e9700 1 -- 172.21.15.69:0/3787966166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 msgr2=0x7fc8680791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:34.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.341+0000 7fc8706e9700 1 --2- 172.21.15.69:0/3787966166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 0x7fc8680791c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc8640098a0 tx=0x7fc864009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:34.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.342+0000 7fc8706e9700 1 -- 172.21.15.69:0/3787966166 shutdown_connections 2023-07-06T20:10:34.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.342+0000 7fc8706e9700 1 --2- 172.21.15.69:0/3787966166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 0x7fc8680791c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:34.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.342+0000 7fc8706e9700 1 -- 172.21.15.69:0/3787966166 >> 172.21.15.69:0/3787966166 conn(0x7fc868100b90 msgr2=0x7fc868102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:34.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.342+0000 7fc8706e9700 1 -- 172.21.15.69:0/3787966166 shutdown_connections 2023-07-06T20:10:34.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.342+0000 7fc8706e9700 1 -- 172.21.15.69:0/3787966166 wait complete. 2023-07-06T20:10:34.005 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.343+0000 7fc8706e9700 1 Processor -- start 2023-07-06T20:10:34.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.343+0000 7fc8706e9700 1 -- start start 2023-07-06T20:10:34.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.344+0000 7fc8706e9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 0x7fc86819b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:34.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.344+0000 7fc8706e9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86819baa0 con 0x7fc86807acc0 2023-07-06T20:10:34.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.344+0000 7fc86e485700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 0x7fc86819b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:34.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.344+0000 7fc86e485700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 0x7fc86819b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58676/0 (socket says 172.21.15.69:58676) 2023-07-06T20:10:34.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.344+0000 7fc86e485700 1 -- 172.21.15.69:0/4273996877 learned_addr learned my addr 172.21.15.69:0/4273996877 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:34.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.344+0000 7fc86e485700 1 -- 172.21.15.69:0/4273996877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc864009550 con 0x7fc86807acc0 2023-07-06T20:10:34.006 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.345+0000 7fc86e485700 1 --2- 172.21.15.69:0/4273996877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 0x7fc86819b560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc864009130 tx=0x7fc864011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:34.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.345+0000 7fc85f7fe700 1 -- 172.21.15.69:0/4273996877 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc864003ab0 con 0x7fc86807acc0 2023-07-06T20:10:34.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.345+0000 7fc85f7fe700 1 -- 172.21.15.69:0/4273996877 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc864003c10 con 0x7fc86807acc0 2023-07-06T20:10:34.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.345+0000 7fc8706e9700 1 -- 172.21.15.69:0/4273996877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86819bca0 con 0x7fc86807acc0 2023-07-06T20:10:34.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.345+0000 7fc85f7fe700 1 -- 172.21.15.69:0/4273996877 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8640247d0 con 0x7fc86807acc0 2023-07-06T20:10:34.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.345+0000 7fc8706e9700 1 -- 172.21.15.69:0/4273996877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86819c0c0 con 0x7fc86807acc0 2023-07-06T20:10:34.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.346+0000 7fc85f7fe700 1 -- 172.21.15.69:0/4273996877 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7fc86401abb0 con 0x7fc86807acc0 2023-07-06T20:10:34.007 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.346+0000 7fc8706e9700 1 -- 172.21.15.69:0/4273996877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc868195510 con 0x7fc86807acc0 2023-07-06T20:10:34.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.346+0000 7fc85f7fe700 1 --2- 172.21.15.69:0/4273996877 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fc854038630 0x7fc85403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:34.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.347+0000 7fc85f7fe700 1 -- 172.21.15.69:0/4273996877 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc86404c650 con 0x7fc86807acc0 2023-07-06T20:10:34.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.347+0000 7fc86dc84700 1 --2- 172.21.15.69:0/4273996877 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fc854038630 0x7fc85403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:34.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.347+0000 7fc86dc84700 1 --2- 172.21.15.69:0/4273996877 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fc854038630 0x7fc85403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc858006fd0 tx=0x7fc858006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:34.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.352+0000 7fc85f7fe700 1 -- 172.21.15.69:0/4273996877 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc864018530 con 0x7fc86807acc0 2023-07-06T20:10:34.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.462+0000 7fc85f7fe700 1 -- 172.21.15.69:0/4273996877 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fc86402d770 con 0x7fc86807acc0 2023-07-06T20:10:34.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.562+0000 7fc8706e9700 1 -- 172.21.15.69:0/4273996877 --> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fc868060a80 con 0x7fc854038630 2023-07-06T20:10:34.008 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.569+0000 7fc85f7fe700 1 -- 172.21.15.69:0/4273996877 <== mgr.14120 v2:172.21.15.69:6800/9699389 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc868060a80 con 0x7fc854038630 2023-07-06T20:10:34.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.571+0000 7fc8706e9700 1 -- 172.21.15.69:0/4273996877 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fc854038630 msgr2=0x7fc85403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:34.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.571+0000 7fc8706e9700 1 --2- 172.21.15.69:0/4273996877 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fc854038630 0x7fc85403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc858006fd0 tx=0x7fc858006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:34.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.571+0000 7fc8706e9700 1 -- 172.21.15.69:0/4273996877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 msgr2=0x7fc86819b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:34.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.571+0000 7fc8706e9700 1 --2- 172.21.15.69:0/4273996877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 0x7fc86819b560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc864009130 tx=0x7fc864011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:34.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.571+0000 7fc8706e9700 1 -- 172.21.15.69:0/4273996877 shutdown_connections 2023-07-06T20:10:34.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.571+0000 7fc8706e9700 1 --2- 172.21.15.69:0/4273996877 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fc854038630 0x7fc85403aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:34.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.571+0000 7fc8706e9700 1 --2- 172.21.15.69:0/4273996877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86807acc0 0x7fc86819b560 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:34.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.571+0000 7fc8706e9700 1 -- 172.21.15.69:0/4273996877 >> 172.21.15.69:0/4273996877 conn(0x7fc868100b90 msgr2=0x7fc8681022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:34.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.572+0000 7fc8706e9700 1 -- 172.21.15.69:0/4273996877 shutdown_connections 2023-07-06T20:10:34.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:33.572+0000 7fc8706e9700 1 -- 172.21.15.69:0/4273996877 wait complete. 2023-07-06T20:10:34.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:34 smithi069 ceph-mon[126399]: mgrmap e8: x(active, since 2s) 2023-07-06T20:10:34.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:34 smithi069 ceph-mon[126399]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:10:34.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:34 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:34.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:34 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:10:35.178 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-06T20:10:35.178 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.539+0000 7f0d0e7d0700 1 Processor -- start 2023-07-06T20:10:35.179 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.540+0000 7f0d0e7d0700 1 -- start start 2023-07-06T20:10:35.179 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.540+0000 7f0d0e7d0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 0x7f0d08108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:35.179 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.540+0000 7f0d0e7d0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08073d80 con 0x7f0d08105fb0 2023-07-06T20:10:35.179 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.540+0000 7f0d07fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 0x7f0d08108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:35.179 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.540+0000 7f0d07fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 0x7f0d08108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58682/0 (socket says 172.21.15.69:58682) 2023-07-06T20:10:35.179 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.540+0000 7f0d07fff700 1 -- 172.21.15.69:0/75621562 learned_addr learned my addr 172.21.15.69:0/75621562 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:35.180 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.541+0000 7f0d07fff700 1 -- 172.21.15.69:0/75621562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08073ec0 con 0x7f0d08105fb0 2023-07-06T20:10:35.180 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.541+0000 7f0d07fff700 1 --2- 172.21.15.69:0/75621562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 0x7f0d08108390 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80098a0 tx=0x7f0cf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bf3896773d2214 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:35.180 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.541+0000 7f0d06ffd700 1 -- 172.21.15.69:0/75621562 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cf8004160 con 0x7f0d08105fb0 2023-07-06T20:10:35.180 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.541+0000 7f0d06ffd700 1 -- 172.21.15.69:0/75621562 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0cf80042c0 con 0x7f0d08105fb0 2023-07-06T20:10:35.180 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.541+0000 7f0d06ffd700 1 -- 172.21.15.69:0/75621562 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cf8011a00 con 0x7f0d08105fb0 2023-07-06T20:10:35.180 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.542+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/75621562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 msgr2=0x7f0d08108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:35.180 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.542+0000 7f0d0e7d0700 1 --2- 172.21.15.69:0/75621562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 0x7f0d08108390 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80098a0 tx=0x7f0cf8009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:35.180 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.542+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/75621562 shutdown_connections 2023-07-06T20:10:35.181 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.542+0000 7f0d0e7d0700 1 --2- 172.21.15.69:0/75621562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 0x7f0d08108390 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:35.181 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.542+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/75621562 >> 172.21.15.69:0/75621562 conn(0x7f0d080ff9b0 msgr2=0x7f0d08101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:35.181 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.542+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/75621562 shutdown_connections 2023-07-06T20:10:35.183 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.542+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/75621562 wait complete. 2023-07-06T20:10:35.184 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.543+0000 7f0d0e7d0700 1 Processor -- start 2023-07-06T20:10:35.184 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.544+0000 7f0d0e7d0700 1 -- start start 2023-07-06T20:10:35.184 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.544+0000 7f0d0e7d0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 0x7f0d0819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:35.184 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.544+0000 7f0d0e7d0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0819b900 con 0x7f0d08105fb0 2023-07-06T20:10:35.184 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.544+0000 7f0d07fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 0x7f0d0819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:35.184 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.544+0000 7f0d07fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 0x7f0d0819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58694/0 (socket says 172.21.15.69:58694) 2023-07-06T20:10:35.184 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.544+0000 7f0d07fff700 1 -- 172.21.15.69:0/2454070958 learned_addr learned my addr 172.21.15.69:0/2454070958 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:35.185 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.544+0000 7f0d07fff700 1 -- 172.21.15.69:0/2454070958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8009550 con 0x7f0d08105fb0 2023-07-06T20:10:35.185 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.545+0000 7f0d07fff700 1 --2- 172.21.15.69:0/2454070958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 0x7f0d0819b3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8006af0 tx=0x7f0cf8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:35.185 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.545+0000 7f0d057fa700 1 -- 172.21.15.69:0/2454070958 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cf8003a40 con 0x7f0d08105fb0 2023-07-06T20:10:35.185 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.545+0000 7f0d057fa700 1 -- 172.21.15.69:0/2454070958 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0cf8003ba0 con 0x7f0d08105fb0 2023-07-06T20:10:35.185 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.545+0000 7f0d057fa700 1 -- 172.21.15.69:0/2454070958 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cf801a700 con 0x7f0d08105fb0 2023-07-06T20:10:35.185 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.545+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/2454070958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0819bb00 con 0x7f0d08105fb0 2023-07-06T20:10:35.185 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.545+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/2454070958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0819bf20 con 0x7f0d08105fb0 2023-07-06T20:10:35.185 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.546+0000 7f0d057fa700 1 -- 172.21.15.69:0/2454070958 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f0cf8024ad0 con 0x7f0d08105fb0 2023-07-06T20:10:35.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.546+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/2454070958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d08061c70 con 0x7f0d08105fb0 2023-07-06T20:10:35.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.547+0000 7f0d057fa700 1 --2- 172.21.15.69:0/2454070958 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f0cf0038760 0x7f0cf003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:35.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.547+0000 7f0d057fa700 1 -- 172.21.15.69:0/2454070958 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0cf804c6d0 con 0x7f0d08105fb0 2023-07-06T20:10:35.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.547+0000 7f0d077fe700 1 --2- 172.21.15.69:0/2454070958 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f0cf0038760 0x7f0cf003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:35.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.548+0000 7f0d077fe700 1 --2- 172.21.15.69:0/2454070958 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f0cf0038760 0x7f0cf003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc006fd0 tx=0x7f0cfc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:35.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.552+0000 7f0d057fa700 1 -- 172.21.15.69:0/2454070958 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0cf8024430 con 0x7f0d08105fb0 2023-07-06T20:10:35.186 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.746+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/2454070958 --> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d08101cf0 con 0x7f0cf0038760 2023-07-06T20:10:35.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.747+0000 7f0d057fa700 1 -- 172.21.15.69:0/2454070958 <== mgr.14120 v2:172.21.15.69:6800/9699389 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f0d08101cf0 con 0x7f0cf0038760 2023-07-06T20:10:35.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.749+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/2454070958 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f0cf0038760 msgr2=0x7f0cf003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:35.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.749+0000 7f0d0e7d0700 1 --2- 172.21.15.69:0/2454070958 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f0cf0038760 0x7f0cf003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc006fd0 tx=0x7f0cfc006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:35.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.749+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/2454070958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 msgr2=0x7f0d0819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:35.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.749+0000 7f0d0e7d0700 1 --2- 172.21.15.69:0/2454070958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 0x7f0d0819b3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8006af0 tx=0x7f0cf8011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:35.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.749+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/2454070958 shutdown_connections 2023-07-06T20:10:35.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.749+0000 7f0d0e7d0700 1 --2- 172.21.15.69:0/2454070958 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f0cf0038760 0x7f0cf003ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:35.187 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.749+0000 7f0d0e7d0700 1 --2- 172.21.15.69:0/2454070958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d08105fb0 0x7f0d0819b3c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:35.188 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.749+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/2454070958 >> 172.21.15.69:0/2454070958 conn(0x7f0d080ff9b0 msgr2=0x7f0d08101090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:35.188 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.750+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/2454070958 shutdown_connections 2023-07-06T20:10:35.188 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:34.750+0000 7f0d0e7d0700 1 -- 172.21.15.69:0/2454070958 wait complete. 2023-07-06T20:10:35.188 INFO:teuthology.orchestra.run.smithi069.stdout:Generating ssh key... 2023-07-06T20:10:35.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:35 smithi069 ceph-mon[126399]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:10:36.465 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: Generating public/private rsa key pair. 2023-07-06T20:10:36.465 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: Your identification has been saved in /tmp/tmpcdyl1oai/key. 2023-07-06T20:10:36.465 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: Your public key has been saved in /tmp/tmpcdyl1oai/key.pub. 2023-07-06T20:10:36.465 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: The key fingerprint is: 2023-07-06T20:10:36.465 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: SHA256:Z/IVrACfMs+IeBGeJQlCPiHd1CCYmztmtuRQRmPJX5I ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:10:36.466 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: The key's randomart image is: 2023-07-06T20:10:36.466 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: +---[RSA 3072]----+ 2023-07-06T20:10:36.466 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: |+Bo===o | 2023-07-06T20:10:36.466 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: |=.OoE=oo . . | 2023-07-06T20:10:36.466 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: | B o+oo + o | 2023-07-06T20:10:36.466 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: |o +..o * . . . | 2023-07-06T20:10:36.466 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: | +. o . S + . | 2023-07-06T20:10:36.467 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: |+= . = . | 2023-07-06T20:10:36.467 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: |B.. . | 2023-07-06T20:10:36.467 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: | o | 2023-07-06T20:10:36.467 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: | | 2023-07-06T20:10:36.467 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: +----[SHA256]-----+ 2023-07-06T20:10:36.664 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.695+0000 7f05de6b8700 1 Processor -- start 2023-07-06T20:10:36.664 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.695+0000 7f05de6b8700 1 -- start start 2023-07-06T20:10:36.665 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.696+0000 7f05de6b8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 0x7f05d8105950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:36.665 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.696+0000 7f05de6b8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d8105e90 con 0x7f05d8105540 2023-07-06T20:10:36.665 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.696+0000 7f05d7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 0x7f05d8105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:36.666 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.696+0000 7f05d7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 0x7f05d8105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58700/0 (socket says 172.21.15.69:58700) 2023-07-06T20:10:36.666 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.696+0000 7f05d7fff700 1 -- 172.21.15.69:0/2664465893 learned_addr learned my addr 172.21.15.69:0/2664465893 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:36.666 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.697+0000 7f05d7fff700 1 -- 172.21.15.69:0/2664465893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d8105fd0 con 0x7f05d8105540 2023-07-06T20:10:36.666 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.697+0000 7f05d7fff700 1 --2- 172.21.15.69:0/2664465893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 0x7f05d8105950 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05c80098a0 tx=0x7f05c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c2d8d69d4b7ddc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:36.667 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.697+0000 7f05d6ffd700 1 -- 172.21.15.69:0/2664465893 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05c8004160 con 0x7f05d8105540 2023-07-06T20:10:36.667 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.697+0000 7f05d6ffd700 1 -- 172.21.15.69:0/2664465893 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f05c80042c0 con 0x7f05d8105540 2023-07-06T20:10:36.667 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.697+0000 7f05d6ffd700 1 -- 172.21.15.69:0/2664465893 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05c8011a00 con 0x7f05d8105540 2023-07-06T20:10:36.668 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.698+0000 7f05de6b8700 1 -- 172.21.15.69:0/2664465893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 msgr2=0x7f05d8105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:36.668 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.698+0000 7f05de6b8700 1 --2- 172.21.15.69:0/2664465893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 0x7f05d8105950 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05c80098a0 tx=0x7f05c8009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:36.668 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.698+0000 7f05de6b8700 1 -- 172.21.15.69:0/2664465893 shutdown_connections 2023-07-06T20:10:36.668 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.698+0000 7f05de6b8700 1 --2- 172.21.15.69:0/2664465893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 0x7f05d8105950 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:36.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.698+0000 7f05de6b8700 1 -- 172.21.15.69:0/2664465893 >> 172.21.15.69:0/2664465893 conn(0x7f05d8100b90 msgr2=0x7f05d8102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:36.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.698+0000 7f05de6b8700 1 -- 172.21.15.69:0/2664465893 shutdown_connections 2023-07-06T20:10:36.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.698+0000 7f05de6b8700 1 -- 172.21.15.69:0/2664465893 wait complete. 2023-07-06T20:10:36.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.699+0000 7f05de6b8700 1 Processor -- start 2023-07-06T20:10:36.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.699+0000 7f05de6b8700 1 -- start start 2023-07-06T20:10:36.670 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.699+0000 7f05de6b8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 0x7f05d8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:36.670 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.699+0000 7f05de6b8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d8199890 con 0x7f05d8105540 2023-07-06T20:10:36.670 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.700+0000 7f05d7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 0x7f05d8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:36.671 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.700+0000 7f05d7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 0x7f05d8199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58712/0 (socket says 172.21.15.69:58712) 2023-07-06T20:10:36.671 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.700+0000 7f05d7fff700 1 -- 172.21.15.69:0/528889541 learned_addr learned my addr 172.21.15.69:0/528889541 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:36.671 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.700+0000 7f05d7fff700 1 -- 172.21.15.69:0/528889541 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c8009550 con 0x7f05d8105540 2023-07-06T20:10:36.671 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.700+0000 7f05d7fff700 1 --2- 172.21.15.69:0/528889541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 0x7f05d8199350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009130 tx=0x7f05c8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:36.671 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.700+0000 7f05d57fa700 1 -- 172.21.15.69:0/528889541 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05c8003b70 con 0x7f05d8105540 2023-07-06T20:10:36.672 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.700+0000 7f05d57fa700 1 -- 172.21.15.69:0/528889541 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f05c8003cd0 con 0x7f05d8105540 2023-07-06T20:10:36.672 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.700+0000 7f05d57fa700 1 -- 172.21.15.69:0/528889541 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05c80197d0 con 0x7f05d8105540 2023-07-06T20:10:36.672 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.700+0000 7f05de6b8700 1 -- 172.21.15.69:0/528889541 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d8199a90 con 0x7f05d8105540 2023-07-06T20:10:36.672 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.701+0000 7f05de6b8700 1 -- 172.21.15.69:0/528889541 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d8199eb0 con 0x7f05d8105540 2023-07-06T20:10:36.672 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.702+0000 7f05d57fa700 1 -- 172.21.15.69:0/528889541 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f05c801abb0 con 0x7f05d8105540 2023-07-06T20:10:36.672 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.702+0000 7f05de6b8700 1 -- 172.21.15.69:0/528889541 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d8061c70 con 0x7f05d8105540 2023-07-06T20:10:36.673 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.702+0000 7f05d57fa700 1 --2- 172.21.15.69:0/528889541 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f05c0038760 0x7f05c003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:36.673 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.702+0000 7f05d57fa700 1 -- 172.21.15.69:0/528889541 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f05c804c950 con 0x7f05d8105540 2023-07-06T20:10:36.673 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.702+0000 7f05d77fe700 1 --2- 172.21.15.69:0/528889541 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f05c0038760 0x7f05c003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:36.673 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.703+0000 7f05d77fe700 1 --2- 172.21.15.69:0/528889541 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f05c0038760 0x7f05c003ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05cc006fd0 tx=0x7f05cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:36.673 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.707+0000 7f05d57fa700 1 -- 172.21.15.69:0/528889541 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f05c801a430 con 0x7f05d8105540 2023-07-06T20:10:36.673 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:35.901+0000 7f05de6b8700 1 -- 172.21.15.69:0/528889541 --> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f05d8102ef0 con 0x7f05c0038760 2023-07-06T20:10:36.673 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:36.221+0000 7f05d57fa700 1 -- 172.21.15.69:0/528889541 <== mgr.14120 v2:172.21.15.69:6800/9699389 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f05d8102ef0 con 0x7f05c0038760 2023-07-06T20:10:36.674 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:36.223+0000 7f05de6b8700 1 -- 172.21.15.69:0/528889541 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f05c0038760 msgr2=0x7f05c003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:36.674 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:36.223+0000 7f05de6b8700 1 --2- 172.21.15.69:0/528889541 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f05c0038760 0x7f05c003ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05cc006fd0 tx=0x7f05cc006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:36.674 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:36.223+0000 7f05de6b8700 1 -- 172.21.15.69:0/528889541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 msgr2=0x7f05d8199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:36.674 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:36.223+0000 7f05de6b8700 1 --2- 172.21.15.69:0/528889541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 0x7f05d8199350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009130 tx=0x7f05c8011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:36.674 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:36.224+0000 7f05de6b8700 1 -- 172.21.15.69:0/528889541 shutdown_connections 2023-07-06T20:10:36.674 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:36.224+0000 7f05de6b8700 1 --2- 172.21.15.69:0/528889541 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f05c0038760 0x7f05c003ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:36.674 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:36.224+0000 7f05de6b8700 1 --2- 172.21.15.69:0/528889541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05d8105540 0x7f05d8199350 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:36.674 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:36.224+0000 7f05de6b8700 1 -- 172.21.15.69:0/528889541 >> 172.21.15.69:0/528889541 conn(0x7f05d8100b90 msgr2=0x7f05d8102290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:36.675 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:36.224+0000 7f05de6b8700 1 -- 172.21.15.69:0/528889541 shutdown_connections 2023-07-06T20:10:36.675 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:36.224+0000 7f05de6b8700 1 -- 172.21.15.69:0/528889541 wait complete. 2023-07-06T20:10:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-mon[126399]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:10:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-mon[126399]: Generating ssh key... 2023-07-06T20:10:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:36 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:37.840 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC2OFHfi5oDewL7brB8IleCloiQOROvtAcE2bJbhzupcDovM/HZH2wyacyaF8AyW920eFAuEb9PHzNZct+ZB+/S4ZPO0AaFrLH7aKekt8sdfch0VJ2nVegkMXRkBVeth3cohXRCIdQfewnlPVo2b6alb5WJCdGerUkj7vOcZCTVswEgOe6puJtSevechEizVTqp1xEM8Q+4Jps8uVVeuZawteRzXHM1hOiaZNdpifRCvI1wuP3mSXSxLNuUbaRDNdKQqux/+QjA0f26fCbn3mggrjMnnN/ikffYQnpC4eCEruHGVq5NA7+2XTEVIsFkl+1OjDOCANI2AfQbmAATCJRKHd+H1T6LTb+1yTrZHuEM6tg9drzyMy3gFl+tBSJesZk1a7EOhCLaQw/Oz6TbAZamTA9Gsv0wcHLtL5dZreCf0a7uts6xMr1QzXtpN6ba4j2sdsCeJ2w7//OFuWt6ZVbHO7oOr8JMoZqbrKuESOJK1iR28n+GAfpClGxn+nKE9c= ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:10:37.840 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.162+0000 7f2c1c492700 1 Processor -- start 2023-07-06T20:10:37.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.162+0000 7f2c1c492700 1 -- start start 2023-07-06T20:10:37.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.163+0000 7f2c1c492700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 0x7f2c14105950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:37.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.163+0000 7f2c1c492700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c14105e90 con 0x7f2c14105540 2023-07-06T20:10:37.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.163+0000 7f2c1a22e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 0x7f2c14105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:37.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.163+0000 7f2c1a22e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 0x7f2c14105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58718/0 (socket says 172.21.15.69:58718) 2023-07-06T20:10:37.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.163+0000 7f2c1a22e700 1 -- 172.21.15.69:0/1062762560 learned_addr learned my addr 172.21.15.69:0/1062762560 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:37.841 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.163+0000 7f2c1a22e700 1 -- 172.21.15.69:0/1062762560 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14105fd0 con 0x7f2c14105540 2023-07-06T20:10:37.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.164+0000 7f2c1a22e700 1 --2- 172.21.15.69:0/1062762560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 0x7f2c14105950 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c100098a0 tx=0x7f2c10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b165894c3ca3782 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:37.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.164+0000 7f2c1922c700 1 -- 172.21.15.69:0/1062762560 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c10004160 con 0x7f2c14105540 2023-07-06T20:10:37.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.164+0000 7f2c1922c700 1 -- 172.21.15.69:0/1062762560 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c100042c0 con 0x7f2c14105540 2023-07-06T20:10:37.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.164+0000 7f2c1922c700 1 -- 172.21.15.69:0/1062762560 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c10011a00 con 0x7f2c14105540 2023-07-06T20:10:37.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.164+0000 7f2c1c492700 1 -- 172.21.15.69:0/1062762560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 msgr2=0x7f2c14105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:37.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.164+0000 7f2c1c492700 1 --2- 172.21.15.69:0/1062762560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 0x7f2c14105950 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c100098a0 tx=0x7f2c10009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:37.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.164+0000 7f2c1c492700 1 -- 172.21.15.69:0/1062762560 shutdown_connections 2023-07-06T20:10:37.842 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.164+0000 7f2c1c492700 1 --2- 172.21.15.69:0/1062762560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 0x7f2c14105950 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:37.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.164+0000 7f2c1c492700 1 -- 172.21.15.69:0/1062762560 >> 172.21.15.69:0/1062762560 conn(0x7f2c14100b90 msgr2=0x7f2c14102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:37.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.164+0000 7f2c1c492700 1 -- 172.21.15.69:0/1062762560 shutdown_connections 2023-07-06T20:10:37.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.165+0000 7f2c1c492700 1 -- 172.21.15.69:0/1062762560 wait complete. 2023-07-06T20:10:37.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.165+0000 7f2c1c492700 1 Processor -- start 2023-07-06T20:10:37.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.166+0000 7f2c1c492700 1 -- start start 2023-07-06T20:10:37.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.166+0000 7f2c1c492700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 0x7f2c1419b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:37.843 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.166+0000 7f2c1c492700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1419baa0 con 0x7f2c14105540 2023-07-06T20:10:37.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.166+0000 7f2c1a22e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 0x7f2c1419b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:37.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.166+0000 7f2c1a22e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 0x7f2c1419b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58722/0 (socket says 172.21.15.69:58722) 2023-07-06T20:10:37.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.166+0000 7f2c1a22e700 1 -- 172.21.15.69:0/3582508574 learned_addr learned my addr 172.21.15.69:0/3582508574 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:37.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.167+0000 7f2c1a22e700 1 -- 172.21.15.69:0/3582508574 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c10009550 con 0x7f2c14105540 2023-07-06T20:10:37.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.167+0000 7f2c1a22e700 1 --2- 172.21.15.69:0/3582508574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 0x7f2c1419b560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009130 tx=0x7f2c10011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:37.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.167+0000 7f2c0b7fe700 1 -- 172.21.15.69:0/3582508574 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c10003b70 con 0x7f2c14105540 2023-07-06T20:10:37.844 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.167+0000 7f2c0b7fe700 1 -- 172.21.15.69:0/3582508574 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c10003cd0 con 0x7f2c14105540 2023-07-06T20:10:37.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.167+0000 7f2c1c492700 1 -- 172.21.15.69:0/3582508574 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1419bca0 con 0x7f2c14105540 2023-07-06T20:10:37.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.167+0000 7f2c0b7fe700 1 -- 172.21.15.69:0/3582508574 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c100197d0 con 0x7f2c14105540 2023-07-06T20:10:37.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.167+0000 7f2c1c492700 1 -- 172.21.15.69:0/3582508574 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1419c0c0 con 0x7f2c14105540 2023-07-06T20:10:37.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.168+0000 7f2c1c492700 1 -- 172.21.15.69:0/3582508574 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c14061c70 con 0x7f2c14105540 2023-07-06T20:10:37.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.168+0000 7f2c0b7fe700 1 -- 172.21.15.69:0/3582508574 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f2c1001abb0 con 0x7f2c14105540 2023-07-06T20:10:37.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.169+0000 7f2c0b7fe700 1 --2- 172.21.15.69:0/3582508574 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f2c00038760 0x7f2c0003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:37.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.169+0000 7f2c0b7fe700 1 -- 172.21.15.69:0/3582508574 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2c1004c9a0 con 0x7f2c14105540 2023-07-06T20:10:37.845 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.169+0000 7f2c19a2d700 1 --2- 172.21.15.69:0/3582508574 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f2c00038760 0x7f2c0003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:37.846 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.170+0000 7f2c19a2d700 1 --2- 172.21.15.69:0/3582508574 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f2c00038760 0x7f2c0003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c04006fd0 tx=0x7f2c04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:37.846 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.174+0000 7f2c0b7fe700 1 -- 172.21.15.69:0/3582508574 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2c1001a430 con 0x7f2c14105540 2023-07-06T20:10:37.846 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.370+0000 7f2c1c492700 1 -- 172.21.15.69:0/3582508574 --> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c14102ef0 con 0x7f2c00038760 2023-07-06T20:10:37.846 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.371+0000 7f2c0b7fe700 1 -- 172.21.15.69:0/3582508574 <== mgr.14120 v2:172.21.15.69:6800/9699389 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f2c14102ef0 con 0x7f2c00038760 2023-07-06T20:10:37.846 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.373+0000 7f2c1c492700 1 -- 172.21.15.69:0/3582508574 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f2c00038760 msgr2=0x7f2c0003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:37.846 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.373+0000 7f2c1c492700 1 --2- 172.21.15.69:0/3582508574 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f2c00038760 0x7f2c0003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c04006fd0 tx=0x7f2c04006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:37.846 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.373+0000 7f2c1c492700 1 -- 172.21.15.69:0/3582508574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 msgr2=0x7f2c1419b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:37.847 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.373+0000 7f2c1c492700 1 --2- 172.21.15.69:0/3582508574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 0x7f2c1419b560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009130 tx=0x7f2c10011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:37.847 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.373+0000 7f2c1c492700 1 -- 172.21.15.69:0/3582508574 shutdown_connections 2023-07-06T20:10:37.847 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.373+0000 7f2c1c492700 1 --2- 172.21.15.69:0/3582508574 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f2c00038760 0x7f2c0003ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:37.847 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.373+0000 7f2c1c492700 1 --2- 172.21.15.69:0/3582508574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c14105540 0x7f2c1419b560 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:37.847 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.373+0000 7f2c1c492700 1 -- 172.21.15.69:0/3582508574 >> 172.21.15.69:0/3582508574 conn(0x7f2c14100b90 msgr2=0x7f2c14102290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:37.847 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.374+0000 7f2c1c492700 1 -- 172.21.15.69:0/3582508574 shutdown_connections 2023-07-06T20:10:37.847 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:37.374+0000 7f2c1c492700 1 -- 172.21.15.69:0/3582508574 wait complete. 2023-07-06T20:10:37.847 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-06T20:10:37.848 INFO:teuthology.orchestra.run.smithi069.stdout:Adding key to root@localhost authorized_keys... 2023-07-06T20:10:37.848 INFO:teuthology.orchestra.run.smithi069.stdout:Adding host smithi069... 2023-07-06T20:10:38.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:38 smithi069 ceph-mon[126399]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:10:39.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:39 smithi069 ceph-mon[126399]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "addr": "172.21.15.69", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:10:40.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:40 smithi069 ceph-mon[126399]: Deploying cephadm binary to smithi069 2023-07-06T20:10:42.129 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Added host 'smithi069' with addr '172.21.15.69' 2023-07-06T20:10:42.129 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.357+0000 7f7cbc49e700 1 Processor -- start 2023-07-06T20:10:42.129 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.357+0000 7f7cbc49e700 1 -- start start 2023-07-06T20:10:42.130 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.357+0000 7f7cbc49e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 0x7f7cb4105950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:42.130 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.357+0000 7f7cbc49e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4105e90 con 0x7f7cb4105540 2023-07-06T20:10:42.130 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.357+0000 7f7cba23a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 0x7f7cb4105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:42.130 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.358+0000 7f7cba23a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 0x7f7cb4105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58724/0 (socket says 172.21.15.69:58724) 2023-07-06T20:10:42.130 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.358+0000 7f7cba23a700 1 -- 172.21.15.69:0/1474918668 learned_addr learned my addr 172.21.15.69:0/1474918668 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:42.130 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.358+0000 7f7cba23a700 1 -- 172.21.15.69:0/1474918668 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb4105fd0 con 0x7f7cb4105540 2023-07-06T20:10:42.130 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.358+0000 7f7cba23a700 1 --2- 172.21.15.69:0/1474918668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 0x7f7cb4105950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00098a0 tx=0x7f7cb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2f17d21ed0bab server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:42.130 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.358+0000 7f7cb9238700 1 -- 172.21.15.69:0/1474918668 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cb0004160 con 0x7f7cb4105540 2023-07-06T20:10:42.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.358+0000 7f7cb9238700 1 -- 172.21.15.69:0/1474918668 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7cb00042c0 con 0x7f7cb4105540 2023-07-06T20:10:42.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.358+0000 7f7cb9238700 1 -- 172.21.15.69:0/1474918668 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cb0011a00 con 0x7f7cb4105540 2023-07-06T20:10:42.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.359+0000 7f7cbc49e700 1 -- 172.21.15.69:0/1474918668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 msgr2=0x7f7cb4105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:42.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.359+0000 7f7cbc49e700 1 --2- 172.21.15.69:0/1474918668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 0x7f7cb4105950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00098a0 tx=0x7f7cb0009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:42.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.359+0000 7f7cbc49e700 1 -- 172.21.15.69:0/1474918668 shutdown_connections 2023-07-06T20:10:42.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.359+0000 7f7cbc49e700 1 --2- 172.21.15.69:0/1474918668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 0x7f7cb4105950 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:42.131 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.359+0000 7f7cbc49e700 1 -- 172.21.15.69:0/1474918668 >> 172.21.15.69:0/1474918668 conn(0x7f7cb4100b90 msgr2=0x7f7cb4102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:42.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.359+0000 7f7cbc49e700 1 -- 172.21.15.69:0/1474918668 shutdown_connections 2023-07-06T20:10:42.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.359+0000 7f7cbc49e700 1 -- 172.21.15.69:0/1474918668 wait complete. 2023-07-06T20:10:42.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.360+0000 7f7cbc49e700 1 Processor -- start 2023-07-06T20:10:42.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.360+0000 7f7cbc49e700 1 -- start start 2023-07-06T20:10:42.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.360+0000 7f7cbc49e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 0x7f7cb419b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:42.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.360+0000 7f7cbc49e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb419baa0 con 0x7f7cb4105540 2023-07-06T20:10:42.132 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.361+0000 7f7cba23a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 0x7f7cb419b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:42.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.361+0000 7f7cba23a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 0x7f7cb419b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58728/0 (socket says 172.21.15.69:58728) 2023-07-06T20:10:42.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.361+0000 7f7cba23a700 1 -- 172.21.15.69:0/2220392233 learned_addr learned my addr 172.21.15.69:0/2220392233 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:42.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.361+0000 7f7cba23a700 1 -- 172.21.15.69:0/2220392233 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb0009550 con 0x7f7cb4105540 2023-07-06T20:10:42.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.361+0000 7f7cba23a700 1 --2- 172.21.15.69:0/2220392233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 0x7f7cb419b560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009130 tx=0x7f7cb0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:42.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.361+0000 7f7cab7fe700 1 -- 172.21.15.69:0/2220392233 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cb0003b70 con 0x7f7cb4105540 2023-07-06T20:10:42.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.361+0000 7f7cab7fe700 1 -- 172.21.15.69:0/2220392233 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7cb0003cd0 con 0x7f7cb4105540 2023-07-06T20:10:42.133 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.361+0000 7f7cab7fe700 1 -- 172.21.15.69:0/2220392233 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cb00197d0 con 0x7f7cb4105540 2023-07-06T20:10:42.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.362+0000 7f7cbc49e700 1 -- 172.21.15.69:0/2220392233 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb419bca0 con 0x7f7cb4105540 2023-07-06T20:10:42.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.362+0000 7f7cbc49e700 1 -- 172.21.15.69:0/2220392233 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb419c0c0 con 0x7f7cb4105540 2023-07-06T20:10:42.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.363+0000 7f7cab7fe700 1 -- 172.21.15.69:0/2220392233 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f7cb001abb0 con 0x7f7cb4105540 2023-07-06T20:10:42.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.363+0000 7f7cbc49e700 1 -- 172.21.15.69:0/2220392233 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb4061c70 con 0x7f7cb4105540 2023-07-06T20:10:42.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.363+0000 7f7cab7fe700 1 --2- 172.21.15.69:0/2220392233 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f7ca0038760 0x7f7ca003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:42.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.363+0000 7f7cab7fe700 1 -- 172.21.15.69:0/2220392233 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7cb004c990 con 0x7f7cb4105540 2023-07-06T20:10:42.134 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.363+0000 7f7cb9a39700 1 --2- 172.21.15.69:0/2220392233 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f7ca0038760 0x7f7ca003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:42.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.364+0000 7f7cb9a39700 1 --2- 172.21.15.69:0/2220392233 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f7ca0038760 0x7f7ca003ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4006fd0 tx=0x7f7ca4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:42.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.368+0000 7f7cab7fe700 1 -- 172.21.15.69:0/2220392233 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7cb001a430 con 0x7f7cb4105540 2023-07-06T20:10:42.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:38.564+0000 7f7cbc49e700 1 -- 172.21.15.69:0/2220392233 --> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi069", "addr": "172.21.15.69", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cb4102ef0 con 0x7f7ca0038760 2023-07-06T20:10:42.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:41.678+0000 7f7cab7fe700 1 -- 172.21.15.69:0/2220392233 <== mgr.14120 v2:172.21.15.69:6800/9699389 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f7cb4102ef0 con 0x7f7ca0038760 2023-07-06T20:10:42.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:41.681+0000 7f7cbc49e700 1 -- 172.21.15.69:0/2220392233 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f7ca0038760 msgr2=0x7f7ca003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:42.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:41.681+0000 7f7cbc49e700 1 --2- 172.21.15.69:0/2220392233 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f7ca0038760 0x7f7ca003ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4006fd0 tx=0x7f7ca4006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:42.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:41.681+0000 7f7cbc49e700 1 -- 172.21.15.69:0/2220392233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 msgr2=0x7f7cb419b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:42.135 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:41.681+0000 7f7cbc49e700 1 --2- 172.21.15.69:0/2220392233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 0x7f7cb419b560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009130 tx=0x7f7cb0011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:42.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:41.681+0000 7f7cbc49e700 1 -- 172.21.15.69:0/2220392233 shutdown_connections 2023-07-06T20:10:42.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:41.681+0000 7f7cbc49e700 1 --2- 172.21.15.69:0/2220392233 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f7ca0038760 0x7f7ca003ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:42.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:41.681+0000 7f7cbc49e700 1 --2- 172.21.15.69:0/2220392233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb4105540 0x7f7cb419b560 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:42.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:41.681+0000 7f7cbc49e700 1 -- 172.21.15.69:0/2220392233 >> 172.21.15.69:0/2220392233 conn(0x7f7cb4100b90 msgr2=0x7f7cb4102290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:42.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:41.681+0000 7f7cbc49e700 1 -- 172.21.15.69:0/2220392233 shutdown_connections 2023-07-06T20:10:42.136 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:41.681+0000 7f7cbc49e700 1 -- 172.21.15.69:0/2220392233 wait complete. 2023-07-06T20:10:42.136 INFO:teuthology.orchestra.run.smithi069.stdout:Deploying unmanaged mon service... 2023-07-06T20:10:43.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:42 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:43.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:42 smithi069 ceph-mon[126399]: Added host smithi069 2023-07-06T20:10:43.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:42 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:10:43.410 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-06T20:10:43.410 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.639+0000 7f371a5ec700 1 Processor -- start 2023-07-06T20:10:43.410 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.639+0000 7f371a5ec700 1 -- start start 2023-07-06T20:10:43.411 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.639+0000 7f371a5ec700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 0x7f3714105950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:43.411 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.639+0000 7f371a5ec700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3714105e90 con 0x7f3714105540 2023-07-06T20:10:43.411 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.640+0000 7f3713fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 0x7f3714105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:43.411 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.640+0000 7f3713fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 0x7f3714105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58736/0 (socket says 172.21.15.69:58736) 2023-07-06T20:10:43.411 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.640+0000 7f3713fff700 1 -- 172.21.15.69:0/2567724062 learned_addr learned my addr 172.21.15.69:0/2567724062 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:43.411 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.640+0000 7f3713fff700 1 -- 172.21.15.69:0/2567724062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3714105fd0 con 0x7f3714105540 2023-07-06T20:10:43.411 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.641+0000 7f3713fff700 1 --2- 172.21.15.69:0/2567724062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 0x7f3714105950 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f37040098a0 tx=0x7f3704009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e402b5ab4f0577 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:43.411 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.641+0000 7f3712ffd700 1 -- 172.21.15.69:0/2567724062 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3704004160 con 0x7f3714105540 2023-07-06T20:10:43.412 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.641+0000 7f3712ffd700 1 -- 172.21.15.69:0/2567724062 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f37040042c0 con 0x7f3714105540 2023-07-06T20:10:43.412 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.641+0000 7f3712ffd700 1 -- 172.21.15.69:0/2567724062 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3704011a00 con 0x7f3714105540 2023-07-06T20:10:43.412 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.641+0000 7f371a5ec700 1 -- 172.21.15.69:0/2567724062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 msgr2=0x7f3714105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:43.412 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.641+0000 7f371a5ec700 1 --2- 172.21.15.69:0/2567724062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 0x7f3714105950 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f37040098a0 tx=0x7f3704009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:43.412 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.642+0000 7f371a5ec700 1 -- 172.21.15.69:0/2567724062 shutdown_connections 2023-07-06T20:10:43.412 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.642+0000 7f371a5ec700 1 --2- 172.21.15.69:0/2567724062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 0x7f3714105950 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:43.413 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.642+0000 7f371a5ec700 1 -- 172.21.15.69:0/2567724062 >> 172.21.15.69:0/2567724062 conn(0x7f3714100b90 msgr2=0x7f3714102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:43.413 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.642+0000 7f371a5ec700 1 -- 172.21.15.69:0/2567724062 shutdown_connections 2023-07-06T20:10:43.413 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.642+0000 7f371a5ec700 1 -- 172.21.15.69:0/2567724062 wait complete. 2023-07-06T20:10:43.413 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.643+0000 7f371a5ec700 1 Processor -- start 2023-07-06T20:10:43.413 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.643+0000 7f371a5ec700 1 -- start start 2023-07-06T20:10:43.413 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.643+0000 7f371a5ec700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 0x7f3714199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:43.413 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.643+0000 7f371a5ec700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3714199890 con 0x7f3714105540 2023-07-06T20:10:43.413 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.643+0000 7f3713fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 0x7f3714199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:43.414 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.643+0000 7f3713fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 0x7f3714199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58742/0 (socket says 172.21.15.69:58742) 2023-07-06T20:10:43.414 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.643+0000 7f3713fff700 1 -- 172.21.15.69:0/1854976904 learned_addr learned my addr 172.21.15.69:0/1854976904 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:43.414 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.644+0000 7f3713fff700 1 -- 172.21.15.69:0/1854976904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3704009550 con 0x7f3714105540 2023-07-06T20:10:43.414 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.644+0000 7f3713fff700 1 --2- 172.21.15.69:0/1854976904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 0x7f3714199350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3704009fb0 tx=0x7f3704011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:43.414 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.644+0000 7f37117fa700 1 -- 172.21.15.69:0/1854976904 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3704003a40 con 0x7f3714105540 2023-07-06T20:10:43.414 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.644+0000 7f371a5ec700 1 -- 172.21.15.69:0/1854976904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3714199a90 con 0x7f3714105540 2023-07-06T20:10:43.415 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.644+0000 7f37117fa700 1 -- 172.21.15.69:0/1854976904 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3704003ba0 con 0x7f3714105540 2023-07-06T20:10:43.415 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.644+0000 7f37117fa700 1 -- 172.21.15.69:0/1854976904 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f370401a750 con 0x7f3714105540 2023-07-06T20:10:43.415 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.644+0000 7f371a5ec700 1 -- 172.21.15.69:0/1854976904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3714199eb0 con 0x7f3714105540 2023-07-06T20:10:43.415 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.645+0000 7f37117fa700 1 -- 172.21.15.69:0/1854976904 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f3704024b60 con 0x7f3714105540 2023-07-06T20:10:43.415 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.645+0000 7f371a5ec700 1 -- 172.21.15.69:0/1854976904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3714061c70 con 0x7f3714105540 2023-07-06T20:10:43.415 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.646+0000 7f37117fa700 1 --2- 172.21.15.69:0/1854976904 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f36fc038710 0x7f36fc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:43.415 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.646+0000 7f37117fa700 1 -- 172.21.15.69:0/1854976904 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f370404c6f0 con 0x7f3714105540 2023-07-06T20:10:43.415 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.646+0000 7f37137fe700 1 --2- 172.21.15.69:0/1854976904 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f36fc038710 0x7f36fc03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:43.416 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.647+0000 7f37137fe700 1 --2- 172.21.15.69:0/1854976904 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f36fc038710 0x7f36fc03abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3708006fd0 tx=0x7f3708006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:43.416 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.651+0000 7f37117fa700 1 -- 172.21.15.69:0/1854976904 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3704024430 con 0x7f3714105540 2023-07-06T20:10:43.416 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.869+0000 7f371a5ec700 1 -- 172.21.15.69:0/1854976904 --> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3714102ef0 con 0x7f36fc038710 2023-07-06T20:10:43.416 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.877+0000 7f37117fa700 1 -- 172.21.15.69:0/1854976904 <== mgr.14120 v2:172.21.15.69:6800/9699389 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3714102ef0 con 0x7f36fc038710 2023-07-06T20:10:43.416 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.879+0000 7f371a5ec700 1 -- 172.21.15.69:0/1854976904 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f36fc038710 msgr2=0x7f36fc03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:43.416 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.879+0000 7f371a5ec700 1 --2- 172.21.15.69:0/1854976904 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f36fc038710 0x7f36fc03abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3708006fd0 tx=0x7f3708006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:43.416 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.879+0000 7f371a5ec700 1 -- 172.21.15.69:0/1854976904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 msgr2=0x7f3714199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:43.417 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.879+0000 7f371a5ec700 1 --2- 172.21.15.69:0/1854976904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 0x7f3714199350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3704009fb0 tx=0x7f3704011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:43.417 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.879+0000 7f371a5ec700 1 -- 172.21.15.69:0/1854976904 shutdown_connections 2023-07-06T20:10:43.417 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.879+0000 7f371a5ec700 1 --2- 172.21.15.69:0/1854976904 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f36fc038710 0x7f36fc03abc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:43.417 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.879+0000 7f371a5ec700 1 --2- 172.21.15.69:0/1854976904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3714105540 0x7f3714199350 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:43.417 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.879+0000 7f371a5ec700 1 -- 172.21.15.69:0/1854976904 >> 172.21.15.69:0/1854976904 conn(0x7f3714100b90 msgr2=0x7f3714102290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:43.417 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.880+0000 7f371a5ec700 1 -- 172.21.15.69:0/1854976904 shutdown_connections 2023-07-06T20:10:43.418 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:42.880+0000 7f371a5ec700 1 -- 172.21.15.69:0/1854976904 wait complete. 2023-07-06T20:10:43.418 INFO:teuthology.orchestra.run.smithi069.stdout:Deploying unmanaged mgr service... 2023-07-06T20:10:44.168 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:43 smithi069 ceph-mon[126399]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:10:44.168 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:43 smithi069 ceph-mon[126399]: Saving service mon spec with placement count:5 2023-07-06T20:10:44.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:43 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:44.686 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-06T20:10:44.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.975+0000 7f8d0c066700 1 Processor -- start 2023-07-06T20:10:44.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.975+0000 7f8d0c066700 1 -- start start 2023-07-06T20:10:44.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.976+0000 7f8d0c066700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 0x7f8d04108430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:44.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.976+0000 7f8d0c066700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d04079d10 con 0x7f8d04106050 2023-07-06T20:10:44.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.976+0000 7f8d09e02700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 0x7f8d04108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:44.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.976+0000 7f8d09e02700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 0x7f8d04108430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56554/0 (socket says 172.21.15.69:56554) 2023-07-06T20:10:44.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.976+0000 7f8d09e02700 1 -- 172.21.15.69:0/3796229648 learned_addr learned my addr 172.21.15.69:0/3796229648 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:44.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.976+0000 7f8d09e02700 1 -- 172.21.15.69:0/3796229648 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d04079e50 con 0x7f8d04106050 2023-07-06T20:10:44.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.976+0000 7f8d09e02700 1 --2- 172.21.15.69:0/3796229648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 0x7f8d04108430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8d000098a0 tx=0x7f8d00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684593b13eb1f75d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:44.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.977+0000 7f8d08e00700 1 -- 172.21.15.69:0/3796229648 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d00004160 con 0x7f8d04106050 2023-07-06T20:10:44.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.977+0000 7f8d08e00700 1 -- 172.21.15.69:0/3796229648 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d000042c0 con 0x7f8d04106050 2023-07-06T20:10:44.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.977+0000 7f8d08e00700 1 -- 172.21.15.69:0/3796229648 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d00011a00 con 0x7f8d04106050 2023-07-06T20:10:44.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.977+0000 7f8d0c066700 1 -- 172.21.15.69:0/3796229648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 msgr2=0x7f8d04108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:44.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.977+0000 7f8d0c066700 1 --2- 172.21.15.69:0/3796229648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 0x7f8d04108430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8d000098a0 tx=0x7f8d00009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:44.691 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.977+0000 7f8d0c066700 1 -- 172.21.15.69:0/3796229648 shutdown_connections 2023-07-06T20:10:44.691 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.977+0000 7f8d0c066700 1 --2- 172.21.15.69:0/3796229648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 0x7f8d04108430 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:44.692 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.977+0000 7f8d0c066700 1 -- 172.21.15.69:0/3796229648 >> 172.21.15.69:0/3796229648 conn(0x7f8d040ffa90 msgr2=0x7f8d04101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:44.692 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.977+0000 7f8d0c066700 1 -- 172.21.15.69:0/3796229648 shutdown_connections 2023-07-06T20:10:44.692 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.978+0000 7f8d0c066700 1 -- 172.21.15.69:0/3796229648 wait complete. 2023-07-06T20:10:44.692 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.978+0000 7f8d0c066700 1 Processor -- start 2023-07-06T20:10:44.692 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.979+0000 7f8d0c066700 1 -- start start 2023-07-06T20:10:44.692 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.979+0000 7f8d0c066700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 0x7f8d0419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:44.692 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.979+0000 7f8d0c066700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0419ba60 con 0x7f8d04106050 2023-07-06T20:10:44.693 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.979+0000 7f8d09e02700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 0x7f8d0419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:44.693 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.979+0000 7f8d09e02700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 0x7f8d0419b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56556/0 (socket says 172.21.15.69:56556) 2023-07-06T20:10:44.693 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.979+0000 7f8d09e02700 1 -- 172.21.15.69:0/2129700413 learned_addr learned my addr 172.21.15.69:0/2129700413 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:44.693 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.979+0000 7f8d09e02700 1 -- 172.21.15.69:0/2129700413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d00009550 con 0x7f8d04106050 2023-07-06T20:10:44.693 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.980+0000 7f8d09e02700 1 --2- 172.21.15.69:0/2129700413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 0x7f8d0419b520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009130 tx=0x7f8d00011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:44.693 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.980+0000 7f8cfaffd700 1 -- 172.21.15.69:0/2129700413 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d00003ab0 con 0x7f8d04106050 2023-07-06T20:10:44.693 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.980+0000 7f8cfaffd700 1 -- 172.21.15.69:0/2129700413 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d00003c10 con 0x7f8d04106050 2023-07-06T20:10:44.694 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.980+0000 7f8d0c066700 1 -- 172.21.15.69:0/2129700413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0419bc60 con 0x7f8d04106050 2023-07-06T20:10:44.694 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.980+0000 7f8cfaffd700 1 -- 172.21.15.69:0/2129700413 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d000247d0 con 0x7f8d04106050 2023-07-06T20:10:44.694 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.980+0000 7f8d0c066700 1 -- 172.21.15.69:0/2129700413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0419c080 con 0x7f8d04106050 2023-07-06T20:10:44.694 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.981+0000 7f8cfaffd700 1 -- 172.21.15.69:0/2129700413 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8d0001abb0 con 0x7f8d04106050 2023-07-06T20:10:44.694 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.981+0000 7f8d0c066700 1 -- 172.21.15.69:0/2129700413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d04061c70 con 0x7f8d04106050 2023-07-06T20:10:44.694 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.982+0000 7f8cfaffd700 1 --2- 172.21.15.69:0/2129700413 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f8cf0038710 0x7f8cf003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:44.694 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.982+0000 7f8cfaffd700 1 -- 172.21.15.69:0/2129700413 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8d0004c8c0 con 0x7f8d04106050 2023-07-06T20:10:44.694 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.982+0000 7f8d09601700 1 --2- 172.21.15.69:0/2129700413 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f8cf0038710 0x7f8cf003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:44.695 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.982+0000 7f8d09601700 1 --2- 172.21.15.69:0/2129700413 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f8cf0038710 0x7f8cf003abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4006fd0 tx=0x7f8cf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:44.695 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:43.986+0000 7f8cfaffd700 1 -- 172.21.15.69:0/2129700413 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8d00018ba0 con 0x7f8d04106050 2023-07-06T20:10:44.695 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:44.199+0000 7f8d0c066700 1 -- 172.21.15.69:0/2129700413 --> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8d04101e00 con 0x7f8cf0038710 2023-07-06T20:10:44.695 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:44.206+0000 7f8cfaffd700 1 -- 172.21.15.69:0/2129700413 <== mgr.14120 v2:172.21.15.69:6800/9699389 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8d04101e00 con 0x7f8cf0038710 2023-07-06T20:10:44.696 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:44.208+0000 7f8d0c066700 1 -- 172.21.15.69:0/2129700413 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f8cf0038710 msgr2=0x7f8cf003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:44.697 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:44.208+0000 7f8d0c066700 1 --2- 172.21.15.69:0/2129700413 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f8cf0038710 0x7f8cf003abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4006fd0 tx=0x7f8cf4006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:44.697 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:44.208+0000 7f8d0c066700 1 -- 172.21.15.69:0/2129700413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 msgr2=0x7f8d0419b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:44.697 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:44.208+0000 7f8d0c066700 1 --2- 172.21.15.69:0/2129700413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 0x7f8d0419b520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009130 tx=0x7f8d00011ad0 comp rx=0 tx=0).stop 2023-07-06T20:10:44.698 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:44.208+0000 7f8d0c066700 1 -- 172.21.15.69:0/2129700413 shutdown_connections 2023-07-06T20:10:44.698 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:44.208+0000 7f8d0c066700 1 --2- 172.21.15.69:0/2129700413 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f8cf0038710 0x7f8cf003abc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:44.698 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:44.208+0000 7f8d0c066700 1 --2- 172.21.15.69:0/2129700413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d04106050 0x7f8d0419b520 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:44.698 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:44.208+0000 7f8d0c066700 1 -- 172.21.15.69:0/2129700413 >> 172.21.15.69:0/2129700413 conn(0x7f8d040ffa90 msgr2=0x7f8d041011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:44.698 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:44.208+0000 7f8d0c066700 1 -- 172.21.15.69:0/2129700413 shutdown_connections 2023-07-06T20:10:44.698 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:44.209+0000 7f8d0c066700 1 -- 172.21.15.69:0/2129700413 wait complete. 2023-07-06T20:10:45.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:45 smithi069 ceph-mon[126399]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:10:45.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:45 smithi069 ceph-mon[126399]: Saving service mgr spec with placement count:2 2023-07-06T20:10:45.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:45 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:45.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:45 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:45.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:45 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:46.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.258+0000 7f9b403a8700 1 Processor -- start 2023-07-06T20:10:46.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.259+0000 7f9b403a8700 1 -- start start 2023-07-06T20:10:46.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.259+0000 7f9b403a8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 0x7f9b380791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:46.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.259+0000 7f9b403a8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b380796e0 con 0x7f9b3807aca0 2023-07-06T20:10:46.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.259+0000 7f9b3e144700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 0x7f9b380791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:46.076 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.259+0000 7f9b3e144700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 0x7f9b380791a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56570/0 (socket says 172.21.15.69:56570) 2023-07-06T20:10:46.076 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.259+0000 7f9b3e144700 1 -- 172.21.15.69:0/2862462192 learned_addr learned my addr 172.21.15.69:0/2862462192 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:46.076 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.260+0000 7f9b3e144700 1 -- 172.21.15.69:0/2862462192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b38079820 con 0x7f9b3807aca0 2023-07-06T20:10:46.076 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.260+0000 7f9b3e144700 1 --2- 172.21.15.69:0/2862462192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 0x7f9b380791a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b340098a0 tx=0x7f9b34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39be7a7a29bcd571 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:46.077 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.260+0000 7f9b3d142700 1 -- 172.21.15.69:0/2862462192 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b34004160 con 0x7f9b3807aca0 2023-07-06T20:10:46.077 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.260+0000 7f9b3d142700 1 -- 172.21.15.69:0/2862462192 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b340042c0 con 0x7f9b3807aca0 2023-07-06T20:10:46.077 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.260+0000 7f9b3d142700 1 -- 172.21.15.69:0/2862462192 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b340119b0 con 0x7f9b3807aca0 2023-07-06T20:10:46.077 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.261+0000 7f9b403a8700 1 -- 172.21.15.69:0/2862462192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 msgr2=0x7f9b380791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:46.078 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.261+0000 7f9b403a8700 1 --2- 172.21.15.69:0/2862462192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 0x7f9b380791a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b340098a0 tx=0x7f9b34009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:46.078 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.261+0000 7f9b403a8700 1 -- 172.21.15.69:0/2862462192 shutdown_connections 2023-07-06T20:10:46.078 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.261+0000 7f9b403a8700 1 --2- 172.21.15.69:0/2862462192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 0x7f9b380791a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:46.078 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.261+0000 7f9b403a8700 1 -- 172.21.15.69:0/2862462192 >> 172.21.15.69:0/2862462192 conn(0x7f9b38100b90 msgr2=0x7f9b38102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:46.079 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.261+0000 7f9b403a8700 1 -- 172.21.15.69:0/2862462192 shutdown_connections 2023-07-06T20:10:46.079 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.261+0000 7f9b403a8700 1 -- 172.21.15.69:0/2862462192 wait complete. 2023-07-06T20:10:46.079 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.262+0000 7f9b403a8700 1 Processor -- start 2023-07-06T20:10:46.079 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.262+0000 7f9b403a8700 1 -- start start 2023-07-06T20:10:46.079 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.262+0000 7f9b403a8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 0x7f9b3819b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:46.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.262+0000 7f9b403a8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3819bac0 con 0x7f9b3807aca0 2023-07-06T20:10:46.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.263+0000 7f9b3e144700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 0x7f9b3819b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:46.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.263+0000 7f9b3e144700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 0x7f9b3819b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56578/0 (socket says 172.21.15.69:56578) 2023-07-06T20:10:46.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.263+0000 7f9b3e144700 1 -- 172.21.15.69:0/646838922 learned_addr learned my addr 172.21.15.69:0/646838922 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:46.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.263+0000 7f9b3e144700 1 -- 172.21.15.69:0/646838922 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b34009550 con 0x7f9b3807aca0 2023-07-06T20:10:46.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.263+0000 7f9b3e144700 1 --2- 172.21.15.69:0/646838922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 0x7f9b3819b580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b34003e60 tx=0x7f9b34011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:46.080 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.263+0000 7f9b2f7fe700 1 -- 172.21.15.69:0/646838922 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b34003af0 con 0x7f9b3807aca0 2023-07-06T20:10:46.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.264+0000 7f9b2f7fe700 1 -- 172.21.15.69:0/646838922 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b34003c50 con 0x7f9b3807aca0 2023-07-06T20:10:46.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.264+0000 7f9b403a8700 1 -- 172.21.15.69:0/646838922 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3819bcc0 con 0x7f9b3807aca0 2023-07-06T20:10:46.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.264+0000 7f9b2f7fe700 1 -- 172.21.15.69:0/646838922 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b34019790 con 0x7f9b3807aca0 2023-07-06T20:10:46.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.264+0000 7f9b403a8700 1 -- 172.21.15.69:0/646838922 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3819c0e0 con 0x7f9b3807aca0 2023-07-06T20:10:46.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.265+0000 7f9b2f7fe700 1 -- 172.21.15.69:0/646838922 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f9b3401a430 con 0x7f9b3807aca0 2023-07-06T20:10:46.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.265+0000 7f9b403a8700 1 -- 172.21.15.69:0/646838922 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b38195530 con 0x7f9b3807aca0 2023-07-06T20:10:46.081 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.265+0000 7f9b2f7fe700 1 --2- 172.21.15.69:0/646838922 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f9b24038760 0x7f9b2403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:46.082 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.265+0000 7f9b2f7fe700 1 -- 172.21.15.69:0/646838922 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9b3404ca20 con 0x7f9b3807aca0 2023-07-06T20:10:46.082 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.265+0000 7f9b3d943700 1 --2- 172.21.15.69:0/646838922 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f9b24038760 0x7f9b2403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:46.082 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.266+0000 7f9b3d943700 1 --2- 172.21.15.69:0/646838922 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f9b24038760 0x7f9b2403ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b28006fd0 tx=0x7f9b28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:46.082 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.270+0000 7f9b2f7fe700 1 -- 172.21.15.69:0/646838922 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9b34018b00 con 0x7f9b3807aca0 2023-07-06T20:10:46.082 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.476+0000 7f9b403a8700 1 -- 172.21.15.69:0/646838922 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f9b3802cd70 con 0x7f9b3807aca0 2023-07-06T20:10:46.082 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.482+0000 7f9b2f7fe700 1 -- 172.21.15.69:0/646838922 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f9b34018b00 con 0x7f9b3807aca0 2023-07-06T20:10:46.082 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.485+0000 7f9b403a8700 1 -- 172.21.15.69:0/646838922 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f9b24038760 msgr2=0x7f9b2403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:46.083 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.485+0000 7f9b403a8700 1 --2- 172.21.15.69:0/646838922 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f9b24038760 0x7f9b2403ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b28006fd0 tx=0x7f9b28006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:46.083 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.485+0000 7f9b403a8700 1 -- 172.21.15.69:0/646838922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 msgr2=0x7f9b3819b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:46.083 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.485+0000 7f9b403a8700 1 --2- 172.21.15.69:0/646838922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 0x7f9b3819b580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b34003e60 tx=0x7f9b34011aa0 comp rx=0 tx=0).stop 2023-07-06T20:10:46.083 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.485+0000 7f9b403a8700 1 -- 172.21.15.69:0/646838922 shutdown_connections 2023-07-06T20:10:46.083 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.485+0000 7f9b403a8700 1 --2- 172.21.15.69:0/646838922 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f9b24038760 0x7f9b2403ac10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:46.083 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.485+0000 7f9b403a8700 1 --2- 172.21.15.69:0/646838922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9b3807aca0 0x7f9b3819b580 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:46.083 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.485+0000 7f9b403a8700 1 -- 172.21.15.69:0/646838922 >> 172.21.15.69:0/646838922 conn(0x7f9b38100b90 msgr2=0x7f9b381022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:46.083 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.485+0000 7f9b403a8700 1 -- 172.21.15.69:0/646838922 shutdown_connections 2023-07-06T20:10:46.084 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:45.485+0000 7f9b403a8700 1 -- 172.21.15.69:0/646838922 wait complete. 2023-07-06T20:10:46.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:46 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/646838922' entity='client.admin' 2023-07-06T20:10:47.348 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.668+0000 7fef35203700 1 Processor -- start 2023-07-06T20:10:47.348 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.668+0000 7fef35203700 1 -- start start 2023-07-06T20:10:47.348 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.669+0000 7fef35203700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 0x7fef30079180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:47.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.669+0000 7fef35203700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef300796c0 con 0x7fef3007ac80 2023-07-06T20:10:47.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.669+0000 7fef2ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 0x7fef30079180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:47.349 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.669+0000 7fef2ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 0x7fef30079180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56582/0 (socket says 172.21.15.69:56582) 2023-07-06T20:10:47.353 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.669+0000 7fef2ed9d700 1 -- 172.21.15.69:0/2522892833 learned_addr learned my addr 172.21.15.69:0/2522892833 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:47.353 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.669+0000 7fef2ed9d700 1 -- 172.21.15.69:0/2522892833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef30079800 con 0x7fef3007ac80 2023-07-06T20:10:47.353 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.670+0000 7fef2ed9d700 1 --2- 172.21.15.69:0/2522892833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 0x7fef30079180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fef200098a0 tx=0x7fef20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cb49cdfb892240 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:47.354 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.670+0000 7fef2dd9b700 1 -- 172.21.15.69:0/2522892833 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef20004160 con 0x7fef3007ac80 2023-07-06T20:10:47.354 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.670+0000 7fef2dd9b700 1 -- 172.21.15.69:0/2522892833 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef200042c0 con 0x7fef3007ac80 2023-07-06T20:10:47.354 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.670+0000 7fef2dd9b700 1 -- 172.21.15.69:0/2522892833 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef200119b0 con 0x7fef3007ac80 2023-07-06T20:10:47.354 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.671+0000 7fef35203700 1 -- 172.21.15.69:0/2522892833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 msgr2=0x7fef30079180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:47.354 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.671+0000 7fef35203700 1 --2- 172.21.15.69:0/2522892833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 0x7fef30079180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fef200098a0 tx=0x7fef20009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:47.354 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.671+0000 7fef35203700 1 -- 172.21.15.69:0/2522892833 shutdown_connections 2023-07-06T20:10:47.354 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.671+0000 7fef35203700 1 --2- 172.21.15.69:0/2522892833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 0x7fef30079180 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:47.355 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.671+0000 7fef35203700 1 -- 172.21.15.69:0/2522892833 >> 172.21.15.69:0/2522892833 conn(0x7fef30100b90 msgr2=0x7fef30102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:47.355 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.671+0000 7fef35203700 1 -- 172.21.15.69:0/2522892833 shutdown_connections 2023-07-06T20:10:47.355 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.671+0000 7fef35203700 1 -- 172.21.15.69:0/2522892833 wait complete. 2023-07-06T20:10:47.355 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.672+0000 7fef35203700 1 Processor -- start 2023-07-06T20:10:47.355 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.672+0000 7fef35203700 1 -- start start 2023-07-06T20:10:47.355 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.672+0000 7fef35203700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 0x7fef3019b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:47.355 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.672+0000 7fef35203700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3019bae0 con 0x7fef3007ac80 2023-07-06T20:10:47.355 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.673+0000 7fef2ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 0x7fef3019b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:47.356 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.673+0000 7fef2ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 0x7fef3019b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56594/0 (socket says 172.21.15.69:56594) 2023-07-06T20:10:47.356 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.673+0000 7fef2ed9d700 1 -- 172.21.15.69:0/1306583445 learned_addr learned my addr 172.21.15.69:0/1306583445 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:47.356 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.673+0000 7fef2ed9d700 1 -- 172.21.15.69:0/1306583445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef20009550 con 0x7fef3007ac80 2023-07-06T20:10:47.356 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.673+0000 7fef2ed9d700 1 --2- 172.21.15.69:0/1306583445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 0x7fef3019b5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fef20000c00 tx=0x7fef20011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:47.356 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.673+0000 7fef17fff700 1 -- 172.21.15.69:0/1306583445 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef20003aa0 con 0x7fef3007ac80 2023-07-06T20:10:47.356 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.673+0000 7fef17fff700 1 -- 172.21.15.69:0/1306583445 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef20003c00 con 0x7fef3007ac80 2023-07-06T20:10:47.356 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.673+0000 7fef35203700 1 -- 172.21.15.69:0/1306583445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3019bce0 con 0x7fef3007ac80 2023-07-06T20:10:47.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.673+0000 7fef17fff700 1 -- 172.21.15.69:0/1306583445 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef20024800 con 0x7fef3007ac80 2023-07-06T20:10:47.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.674+0000 7fef35203700 1 -- 172.21.15.69:0/1306583445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3019c100 con 0x7fef3007ac80 2023-07-06T20:10:47.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.675+0000 7fef17fff700 1 -- 172.21.15.69:0/1306583445 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fef2001ab60 con 0x7fef3007ac80 2023-07-06T20:10:47.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.675+0000 7fef35203700 1 -- 172.21.15.69:0/1306583445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef30195550 con 0x7fef3007ac80 2023-07-06T20:10:47.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.675+0000 7fef17fff700 1 --2- 172.21.15.69:0/1306583445 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fef18038710 0x7fef1803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:47.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.675+0000 7fef17fff700 1 -- 172.21.15.69:0/1306583445 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fef2004c830 con 0x7fef3007ac80 2023-07-06T20:10:47.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.675+0000 7fef2e59c700 1 --2- 172.21.15.69:0/1306583445 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fef18038710 0x7fef1803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:47.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.676+0000 7fef2e59c700 1 --2- 172.21.15.69:0/1306583445 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fef18038710 0x7fef1803abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef24006fd0 tx=0x7fef24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:47.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.680+0000 7fef17fff700 1 -- 172.21.15.69:0/1306583445 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fef20018530 con 0x7fef3007ac80 2023-07-06T20:10:47.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.887+0000 7fef35203700 1 -- 172.21.15.69:0/1306583445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fef30061c70 con 0x7fef3007ac80 2023-07-06T20:10:47.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.893+0000 7fef17fff700 1 -- 172.21.15.69:0/1306583445 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fef20018350 con 0x7fef3007ac80 2023-07-06T20:10:47.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.896+0000 7fef35203700 1 -- 172.21.15.69:0/1306583445 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fef18038710 msgr2=0x7fef1803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:47.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.896+0000 7fef35203700 1 --2- 172.21.15.69:0/1306583445 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fef18038710 0x7fef1803abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef24006fd0 tx=0x7fef24006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:47.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.896+0000 7fef35203700 1 -- 172.21.15.69:0/1306583445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 msgr2=0x7fef3019b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:47.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.896+0000 7fef35203700 1 --2- 172.21.15.69:0/1306583445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 0x7fef3019b5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fef20000c00 tx=0x7fef20011a30 comp rx=0 tx=0).stop 2023-07-06T20:10:47.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.896+0000 7fef35203700 1 -- 172.21.15.69:0/1306583445 shutdown_connections 2023-07-06T20:10:47.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.896+0000 7fef35203700 1 --2- 172.21.15.69:0/1306583445 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fef18038710 0x7fef1803abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:47.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.896+0000 7fef35203700 1 --2- 172.21.15.69:0/1306583445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3007ac80 0x7fef3019b5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:47.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.896+0000 7fef35203700 1 -- 172.21.15.69:0/1306583445 >> 172.21.15.69:0/1306583445 conn(0x7fef30100b90 msgr2=0x7fef301022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:47.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.896+0000 7fef35203700 1 -- 172.21.15.69:0/1306583445 shutdown_connections 2023-07-06T20:10:47.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:46.896+0000 7fef35203700 1 -- 172.21.15.69:0/1306583445 wait complete. 2023-07-06T20:10:47.360 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling the dashboard module... 2023-07-06T20:10:48.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:47 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1306583445' entity='client.admin' 2023-07-06T20:10:48.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:47 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:49.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:48 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2078890379' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-06T20:10:49.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:48 smithi069 ceph-mon[126399]: from='mgr.14120 172.21.15.69:0/2672941734' entity='mgr.x' 2023-07-06T20:10:49.211 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:49 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: ignoring --setuser ceph since I am not root 2023-07-06T20:10:49.211 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:49 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: ignoring --setgroup ceph since I am not root 2023-07-06T20:10:49.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.893+0000 7f638d9bd700 1 Processor -- start 2023-07-06T20:10:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.893+0000 7f638d9bd700 1 -- start start 2023-07-06T20:10:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.894+0000 7f638d9bd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 0x7f6388110360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.894+0000 7f638d9bd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881108a0 con 0x7f638810ff50 2023-07-06T20:10:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.894+0000 7f6386ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 0x7f6388110360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.894+0000 7f6386ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 0x7f6388110360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56600/0 (socket says 172.21.15.69:56600) 2023-07-06T20:10:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.894+0000 7f6386ffd700 1 -- 172.21.15.69:0/2992897632 learned_addr learned my addr 172.21.15.69:0/2992897632 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.894+0000 7f6386ffd700 1 -- 172.21.15.69:0/2992897632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63881109e0 con 0x7f638810ff50 2023-07-06T20:10:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.895+0000 7f6386ffd700 1 --2- 172.21.15.69:0/2992897632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 0x7f6388110360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63780098a0 tx=0x7f6378009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f23aa25a09814a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.895+0000 7f6385ffb700 1 -- 172.21.15.69:0/2992897632 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6378004160 con 0x7f638810ff50 2023-07-06T20:10:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.895+0000 7f6385ffb700 1 -- 172.21.15.69:0/2992897632 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63780042c0 con 0x7f638810ff50 2023-07-06T20:10:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.895+0000 7f6385ffb700 1 -- 172.21.15.69:0/2992897632 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63780119b0 con 0x7f638810ff50 2023-07-06T20:10:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.896+0000 7f638d9bd700 1 -- 172.21.15.69:0/2992897632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 msgr2=0x7f6388110360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.896+0000 7f638d9bd700 1 --2- 172.21.15.69:0/2992897632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 0x7f6388110360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63780098a0 tx=0x7f6378009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.896+0000 7f638d9bd700 1 -- 172.21.15.69:0/2992897632 shutdown_connections 2023-07-06T20:10:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.896+0000 7f638d9bd700 1 --2- 172.21.15.69:0/2992897632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 0x7f6388110360 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.896+0000 7f638d9bd700 1 -- 172.21.15.69:0/2992897632 >> 172.21.15.69:0/2992897632 conn(0x7f63880722e0 msgr2=0x7f638810dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.896+0000 7f638d9bd700 1 -- 172.21.15.69:0/2992897632 shutdown_connections 2023-07-06T20:10:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.896+0000 7f638d9bd700 1 -- 172.21.15.69:0/2992897632 wait complete. 2023-07-06T20:10:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.897+0000 7f638d9bd700 1 Processor -- start 2023-07-06T20:10:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.898+0000 7f638d9bd700 1 -- start start 2023-07-06T20:10:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.898+0000 7f638d9bd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 0x7f63881a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.898+0000 7f638d9bd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881a42e0 con 0x7f638810ff50 2023-07-06T20:10:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.898+0000 7f6386ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 0x7f63881a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.898+0000 7f6386ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 0x7f63881a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56614/0 (socket says 172.21.15.69:56614) 2023-07-06T20:10:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.898+0000 7f6386ffd700 1 -- 172.21.15.69:0/2078890379 learned_addr learned my addr 172.21.15.69:0/2078890379 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.899+0000 7f6386ffd700 1 -- 172.21.15.69:0/2078890379 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6378009550 con 0x7f638810ff50 2023-07-06T20:10:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.899+0000 7f6386ffd700 1 --2- 172.21.15.69:0/2078890379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 0x7f63881a3da0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6378003f00 tx=0x7f6378011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.899+0000 7f638c9bb700 1 -- 172.21.15.69:0/2078890379 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6378003b20 con 0x7f638810ff50 2023-07-06T20:10:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.899+0000 7f638c9bb700 1 -- 172.21.15.69:0/2078890379 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6378003c80 con 0x7f638810ff50 2023-07-06T20:10:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.899+0000 7f638d9bd700 1 -- 172.21.15.69:0/2078890379 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63881a44e0 con 0x7f638810ff50 2023-07-06T20:10:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.899+0000 7f638c9bb700 1 -- 172.21.15.69:0/2078890379 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6378019770 con 0x7f638810ff50 2023-07-06T20:10:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.900+0000 7f638d9bd700 1 -- 172.21.15.69:0/2078890379 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63881a4900 con 0x7f638810ff50 2023-07-06T20:10:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.901+0000 7f638d9bd700 1 -- 172.21.15.69:0/2078890379 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6388061c70 con 0x7f638810ff50 2023-07-06T20:10:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.901+0000 7f638c9bb700 1 -- 172.21.15.69:0/2078890379 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f637801ab60 con 0x7f638810ff50 2023-07-06T20:10:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.902+0000 7f638c9bb700 1 --2- 172.21.15.69:0/2078890379 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f6370038760 0x7f637003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.902+0000 7f638c9bb700 1 -- 172.21.15.69:0/2078890379 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f637804ca30 con 0x7f638810ff50 2023-07-06T20:10:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.902+0000 7f63867fc700 1 --2- 172.21.15.69:0/2078890379 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f6370038760 0x7f637003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.902+0000 7f63867fc700 1 --2- 172.21.15.69:0/2078890379 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f6370038760 0x7f637003ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f637c006fd0 tx=0x7f637c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:47.907+0000 7f638c9bb700 1 -- 172.21.15.69:0/2078890379 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f637801a430 con 0x7f638810ff50 2023-07-06T20:10:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.197+0000 7f638d9bd700 1 -- 172.21.15.69:0/2078890379 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f63881a7980 con 0x7f638810ff50 2023-07-06T20:10:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.900+0000 7f638c9bb700 1 -- 172.21.15.69:0/2078890379 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f637802d350 con 0x7f638810ff50 2023-07-06T20:10:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.902+0000 7f638c9bb700 1 -- 172.21.15.69:0/2078890379 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f637804c380 con 0x7f638810ff50 2023-07-06T20:10:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.904+0000 7f638d9bd700 1 -- 172.21.15.69:0/2078890379 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f6370038760 msgr2=0x7f637003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.904+0000 7f638d9bd700 1 --2- 172.21.15.69:0/2078890379 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f6370038760 0x7f637003ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f637c006fd0 tx=0x7f637c006e40 comp rx=0 tx=0).stop 2023-07-06T20:10:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.904+0000 7f638d9bd700 1 -- 172.21.15.69:0/2078890379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 msgr2=0x7f63881a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.904+0000 7f638d9bd700 1 --2- 172.21.15.69:0/2078890379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 0x7f63881a3da0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6378003f00 tx=0x7f6378011a30 comp rx=0 tx=0).stop 2023-07-06T20:10:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.904+0000 7f638d9bd700 1 -- 172.21.15.69:0/2078890379 shutdown_connections 2023-07-06T20:10:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.904+0000 7f638d9bd700 1 --2- 172.21.15.69:0/2078890379 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f6370038760 0x7f637003ac10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.904+0000 7f638d9bd700 1 --2- 172.21.15.69:0/2078890379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f638810ff50 0x7f63881a3da0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.904+0000 7f638d9bd700 1 -- 172.21.15.69:0/2078890379 >> 172.21.15.69:0/2078890379 conn(0x7f63880722e0 msgr2=0x7f638806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.904+0000 7f638d9bd700 1 -- 172.21.15.69:0/2078890379 shutdown_connections 2023-07-06T20:10:49.341 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:48.905+0000 7f638d9bd700 1 -- 172.21.15.69:0/2078890379 wait complete. 2023-07-06T20:10:49.508 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:49 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:49.208+0000 7f3d791e0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T20:10:49.509 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:49 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:49.318+0000 7f3d791e0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T20:10:49.509 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:49 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:49.446+0000 7f3d791e0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T20:10:49.905 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:49 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:49.575+0000 7f3d791e0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T20:10:49.906 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:49 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:49.812+0000 7f3d791e0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T20:10:50.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:49 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2078890379' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-06T20:10:50.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:49 smithi069 ceph-mon[126399]: mgrmap e9: x(active, since 17s) 2023-07-06T20:10:50.508 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:50 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:50.270+0000 7f3d791e0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T20:10:50.509 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:50 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:50.395+0000 7f3d791e0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T20:10:50.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2023-07-06T20:10:50.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-06T20:10:50.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-06T20:10:50.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-06T20:10:50.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-06T20:10:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2023-07-06T20:10:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.886+0000 7f1feefb0700 1 Processor -- start 2023-07-06T20:10:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.887+0000 7f1feefb0700 1 -- start start 2023-07-06T20:10:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.887+0000 7f1feefb0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe8073d00 0x7f1fe81080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.887+0000 7f1feefb0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe8108610 con 0x7f1fe8073d00 2023-07-06T20:10:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.888+0000 7f1fecd4c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe8073d00 0x7f1fe81080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:50.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.888+0000 7f1fecd4c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe8073d00 0x7f1fe81080d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56642/0 (socket says 172.21.15.69:56642) 2023-07-06T20:10:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.888+0000 7f1fecd4c700 1 -- 172.21.15.69:0/3593092550 learned_addr learned my addr 172.21.15.69:0/3593092550 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.888+0000 7f1fecd4c700 1 -- 172.21.15.69:0/3593092550 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe8108750 con 0x7f1fe8073d00 2023-07-06T20:10:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.888+0000 7f1fecd4c700 1 --2- 172.21.15.69:0/3593092550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe8073d00 0x7f1fe81080d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc0098a0 tx=0x7f1fdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e998017c507934 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.889+0000 7f1fe77fe700 1 -- 172.21.15.69:0/3593092550 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fdc004160 con 0x7f1fe8073d00 2023-07-06T20:10:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.889+0000 7f1fe77fe700 1 -- 172.21.15.69:0/3593092550 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fdc0042c0 con 0x7f1fe8073d00 2023-07-06T20:10:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.889+0000 7f1fe77fe700 1 -- 172.21.15.69:0/3593092550 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fdc011960 con 0x7f1fe8073d00 2023-07-06T20:10:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.890+0000 7f1feefb0700 1 -- 172.21.15.69:0/3593092550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe8073d00 msgr2=0x7f1fe81080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:50.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.890+0000 7f1feefb0700 1 --2- 172.21.15.69:0/3593092550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe8073d00 0x7f1fe81080d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc0098a0 tx=0x7f1fdc009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:50.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.890+0000 7f1feefb0700 1 -- 172.21.15.69:0/3593092550 shutdown_connections 2023-07-06T20:10:50.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.890+0000 7f1feefb0700 1 --2- 172.21.15.69:0/3593092550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe8073d00 0x7f1fe81080d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:50.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.890+0000 7f1feefb0700 1 -- 172.21.15.69:0/3593092550 >> 172.21.15.69:0/3593092550 conn(0x7f1fe80ffa80 msgr2=0x7f1fe8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:50.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.890+0000 7f1feefb0700 1 -- 172.21.15.69:0/3593092550 shutdown_connections 2023-07-06T20:10:50.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.890+0000 7f1feefb0700 1 -- 172.21.15.69:0/3593092550 wait complete. 2023-07-06T20:10:50.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.891+0000 7f1feefb0700 1 Processor -- start 2023-07-06T20:10:50.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.892+0000 7f1feefb0700 1 -- start start 2023-07-06T20:10:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.892+0000 7f1feefb0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe81059e0 0x7f1fe8104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.892+0000 7f1feefb0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe8105df0 con 0x7f1fe81059e0 2023-07-06T20:10:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.892+0000 7f1fecd4c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe81059e0 0x7f1fe8104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.892+0000 7f1fecd4c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe81059e0 0x7f1fe8104070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56658/0 (socket says 172.21.15.69:56658) 2023-07-06T20:10:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.892+0000 7f1fecd4c700 1 -- 172.21.15.69:0/1926082910 learned_addr learned my addr 172.21.15.69:0/1926082910 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.893+0000 7f1fecd4c700 1 -- 172.21.15.69:0/1926082910 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc009550 con 0x7f1fe81059e0 2023-07-06T20:10:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.893+0000 7f1fecd4c700 1 --2- 172.21.15.69:0/1926082910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe81059e0 0x7f1fe8104070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc003f00 tx=0x7f1fdc011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:50.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.893+0000 7f1fe5ffb700 1 -- 172.21.15.69:0/1926082910 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fdc0036a0 con 0x7f1fe81059e0 2023-07-06T20:10:50.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.893+0000 7f1fe5ffb700 1 -- 172.21.15.69:0/1926082910 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fdc01a430 con 0x7f1fe81059e0 2023-07-06T20:10:50.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.893+0000 7f1fe5ffb700 1 -- 172.21.15.69:0/1926082910 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fdc0193c0 con 0x7f1fe81059e0 2023-07-06T20:10:50.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.893+0000 7f1feefb0700 1 -- 172.21.15.69:0/1926082910 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe81045b0 con 0x7f1fe81059e0 2023-07-06T20:10:50.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.894+0000 7f1feefb0700 1 -- 172.21.15.69:0/1926082910 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe81049d0 con 0x7f1fe81059e0 2023-07-06T20:10:50.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.895+0000 7f1fe5ffb700 1 -- 172.21.15.69:0/1926082910 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f1fdc019640 con 0x7f1fe81059e0 2023-07-06T20:10:50.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.895+0000 7f1feefb0700 1 -- 172.21.15.69:0/1926082910 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe8193320 con 0x7f1fe81059e0 2023-07-06T20:10:50.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.895+0000 7f1fe5ffb700 1 --2- 172.21.15.69:0/1926082910 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f1fd80387b0 0x7f1fd803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:50.825 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.896+0000 7f1fe5ffb700 1 -- 172.21.15.69:0/1926082910 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1fdc04c410 con 0x7f1fe81059e0 2023-07-06T20:10:50.825 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.896+0000 7f1fe7fff700 1 -- 172.21.15.69:0/1926082910 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f1fd80387b0 msgr2=0x7f1fd803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/9699389 2023-07-06T20:10:50.825 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:50 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:50.559+0000 7f3d791e0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T20:10:50.825 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:50 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:50.688+0000 7f3d791e0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T20:10:50.826 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.896+0000 7f1fe7fff700 1 --2- 172.21.15.69:0/1926082910 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f1fd80387b0 0x7f1fd803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T20:10:50.827 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:49.902+0000 7f1fe5ffb700 1 -- 172.21.15.69:0/1926082910 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1fdc0199e0 con 0x7f1fe81059e0 2023-07-06T20:10:50.827 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.097+0000 7f1fe7fff700 1 -- 172.21.15.69:0/1926082910 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f1fd80387b0 msgr2=0x7f1fd803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/9699389 2023-07-06T20:10:50.827 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.097+0000 7f1fe7fff700 1 --2- 172.21.15.69:0/1926082910 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f1fd80387b0 0x7f1fd803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-06T20:10:50.827 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.171+0000 7f1feefb0700 1 -- 172.21.15.69:0/1926082910 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f1fe8061c70 con 0x7f1fe81059e0 2023-07-06T20:10:50.828 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.172+0000 7f1fe5ffb700 1 -- 172.21.15.69:0/1926082910 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f1fdc023b80 con 0x7f1fe81059e0 2023-07-06T20:10:50.828 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.175+0000 7f1feefb0700 1 -- 172.21.15.69:0/1926082910 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f1fd80387b0 msgr2=0x7f1fd803ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T20:10:50.828 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.175+0000 7f1feefb0700 1 --2- 172.21.15.69:0/1926082910 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f1fd80387b0 0x7f1fd803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:50.828 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.175+0000 7f1feefb0700 1 -- 172.21.15.69:0/1926082910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe81059e0 msgr2=0x7f1fe8104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:50.828 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.175+0000 7f1feefb0700 1 --2- 172.21.15.69:0/1926082910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe81059e0 0x7f1fe8104070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc003f00 tx=0x7f1fdc011eb0 comp rx=0 tx=0).stop 2023-07-06T20:10:50.829 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.175+0000 7f1feefb0700 1 -- 172.21.15.69:0/1926082910 shutdown_connections 2023-07-06T20:10:50.829 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.175+0000 7f1feefb0700 1 --2- 172.21.15.69:0/1926082910 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7f1fd80387b0 0x7f1fd803ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:50.829 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.175+0000 7f1feefb0700 1 --2- 172.21.15.69:0/1926082910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fe81059e0 0x7f1fe8104070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:50.829 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.175+0000 7f1feefb0700 1 -- 172.21.15.69:0/1926082910 >> 172.21.15.69:0/1926082910 conn(0x7f1fe80ffa80 msgr2=0x7f1fe81011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:50.830 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.175+0000 7f1feefb0700 1 -- 172.21.15.69:0/1926082910 shutdown_connections 2023-07-06T20:10:50.830 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:50.175+0000 7f1feefb0700 1 -- 172.21.15.69:0/1926082910 wait complete. 2023-07-06T20:10:50.830 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for the mgr to restart... 2023-07-06T20:10:50.831 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr epoch 9... 2023-07-06T20:10:51.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:50 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1926082910' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-06T20:10:51.508 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:51 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:51.252+0000 7f3d791e0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T20:10:52.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:51 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:51.995+0000 7f3d791e0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T20:10:52.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:52 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:52.203+0000 7f3d791e0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T20:10:53.475 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:53 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:53.134+0000 7f3d791e0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:10:53.476 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:53 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:53.252+0000 7f3d791e0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T20:10:53.476 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:53 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:53.473+0000 7f3d791e0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:10:54.104 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:53 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:53.831+0000 7f3d791e0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T20:10:54.105 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:54 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:54.102+0000 7f3d791e0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T20:10:54.508 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:54 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:54.291+0000 7f3d791e0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T20:10:55.008 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:54 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:54.631+0000 7f3d791e0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T20:10:55.955 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:55 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:55.541+0000 7f3d791e0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T20:10:55.956 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:55 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:55.712+0000 7f3d791e0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T20:10:56.258 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:55 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:55.954+0000 7f3d791e0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T20:10:56.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:56 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:56.101+0000 7f3d791e0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T20:10:56.758 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:56 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:56.371+0000 7f3d791e0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T20:10:57.758 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:10:57 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:10:57.430+0000 7f3d791e0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T20:10:58.475 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: Active manager daemon x restarted 2023-07-06T20:10:58.475 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: Activating manager daemon x 2023-07-06T20:10:58.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: osdmap e3: 0 total, 0 up, 0 in 2023-07-06T20:10:58.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: mgrmap e10: x(active, starting, since 0.00838505s) 2023-07-06T20:10:58.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:10:58.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-06T20:10:58.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-06T20:10:58.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-06T20:10:58.476 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-06T20:10:58.477 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: Manager daemon x is now available 2023-07-06T20:10:58.477 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:10:58.477 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:10:58.477 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-06T20:10:58.477 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-06T20:10:58.477 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: [06/Jul/2023:20:10:57] ENGINE Bus STARTING 2023-07-06T20:10:58.477 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: [06/Jul/2023:20:10:57] ENGINE Serving on http://172.21.15.69:8765 2023-07-06T20:10:58.477 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: [06/Jul/2023:20:10:57] ENGINE Serving on https://172.21.15.69:7150 2023-07-06T20:10:58.478 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:58 smithi069 ceph-mon[126399]: [06/Jul/2023:20:10:57] ENGINE Bus STARTED 2023-07-06T20:10:58.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2023-07-06T20:10:58.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-06T20:10:58.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-06T20:10:58.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2023-07-06T20:10:58.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.360+0000 7fdbade74700 1 Processor -- start 2023-07-06T20:10:58.815 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.360+0000 7fdbade74700 1 -- start start 2023-07-06T20:10:58.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.360+0000 7fdbade74700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 0x7fdba8079160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:58.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.360+0000 7fdbade74700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80796a0 con 0x7fdba807ac60 2023-07-06T20:10:58.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.361+0000 7fdba77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 0x7fdba8079160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:58.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.361+0000 7fdba77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 0x7fdba8079160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56670/0 (socket says 172.21.15.69:56670) 2023-07-06T20:10:58.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.361+0000 7fdba77fe700 1 -- 172.21.15.69:0/3850243367 learned_addr learned my addr 172.21.15.69:0/3850243367 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:58.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.361+0000 7fdba77fe700 1 -- 172.21.15.69:0/3850243367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba80797e0 con 0x7fdba807ac60 2023-07-06T20:10:58.816 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.361+0000 7fdba77fe700 1 --2- 172.21.15.69:0/3850243367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 0x7fdba8079160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdb980098a0 tx=0x7fdb98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e28a68304f58b74 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:58.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.361+0000 7fdba67fc700 1 -- 172.21.15.69:0/3850243367 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb98004160 con 0x7fdba807ac60 2023-07-06T20:10:58.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.361+0000 7fdba67fc700 1 -- 172.21.15.69:0/3850243367 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb980042c0 con 0x7fdba807ac60 2023-07-06T20:10:58.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.361+0000 7fdba67fc700 1 -- 172.21.15.69:0/3850243367 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb980119b0 con 0x7fdba807ac60 2023-07-06T20:10:58.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.362+0000 7fdbade74700 1 -- 172.21.15.69:0/3850243367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 msgr2=0x7fdba8079160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:58.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.362+0000 7fdbade74700 1 --2- 172.21.15.69:0/3850243367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 0x7fdba8079160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdb980098a0 tx=0x7fdb98009bb0 comp rx=0 tx=0).stop 2023-07-06T20:10:58.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.362+0000 7fdbade74700 1 -- 172.21.15.69:0/3850243367 shutdown_connections 2023-07-06T20:10:58.817 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.362+0000 7fdbade74700 1 --2- 172.21.15.69:0/3850243367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 0x7fdba8079160 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:58.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.362+0000 7fdbade74700 1 -- 172.21.15.69:0/3850243367 >> 172.21.15.69:0/3850243367 conn(0x7fdba8100b90 msgr2=0x7fdba8102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:58.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.362+0000 7fdbade74700 1 -- 172.21.15.69:0/3850243367 shutdown_connections 2023-07-06T20:10:58.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.362+0000 7fdbade74700 1 -- 172.21.15.69:0/3850243367 wait complete. 2023-07-06T20:10:58.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.364+0000 7fdbade74700 1 Processor -- start 2023-07-06T20:10:58.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.364+0000 7fdbade74700 1 -- start start 2023-07-06T20:10:58.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.364+0000 7fdbade74700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 0x7fdba819b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:58.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.365+0000 7fdbade74700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba819baa0 con 0x7fdba807ac60 2023-07-06T20:10:58.818 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.365+0000 7fdba77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 0x7fdba819b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:58.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.365+0000 7fdba77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 0x7fdba819b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56672/0 (socket says 172.21.15.69:56672) 2023-07-06T20:10:58.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.365+0000 7fdba77fe700 1 -- 172.21.15.69:0/1683757531 learned_addr learned my addr 172.21.15.69:0/1683757531 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:10:58.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.365+0000 7fdba77fe700 1 -- 172.21.15.69:0/1683757531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb98009550 con 0x7fdba807ac60 2023-07-06T20:10:58.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.365+0000 7fdba77fe700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 0x7fdba819b560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb98003e60 tx=0x7fdb98011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:58.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.366+0000 7fdba4ff9700 1 -- 172.21.15.69:0/1683757531 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb98003a60 con 0x7fdba807ac60 2023-07-06T20:10:58.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.366+0000 7fdba4ff9700 1 -- 172.21.15.69:0/1683757531 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb98003bc0 con 0x7fdba807ac60 2023-07-06T20:10:58.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.366+0000 7fdbade74700 1 -- 172.21.15.69:0/1683757531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba819bca0 con 0x7fdba807ac60 2023-07-06T20:10:58.819 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.366+0000 7fdba4ff9700 1 -- 172.21.15.69:0/1683757531 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb980196c0 con 0x7fdba807ac60 2023-07-06T20:10:58.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.366+0000 7fdbade74700 1 -- 172.21.15.69:0/1683757531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba819c0c0 con 0x7fdba807ac60 2023-07-06T20:10:58.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.367+0000 7fdba4ff9700 1 -- 172.21.15.69:0/1683757531 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fdb98019930 con 0x7fdba807ac60 2023-07-06T20:10:58.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.367+0000 7fdba4ff9700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 0x7fdb9003ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:58.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.367+0000 7fdba4ff9700 1 -- 172.21.15.69:0/1683757531 --> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb9003b350 con 0x7fdb90038790 2023-07-06T20:10:58.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.367+0000 7fdba4ff9700 1 -- 172.21.15.69:0/1683757531 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdb9804c930 con 0x7fdba807ac60 2023-07-06T20:10:58.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.367+0000 7fdba6ffd700 1 -- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 msgr2=0x7fdb9003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/9699389 2023-07-06T20:10:58.820 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.367+0000 7fdba6ffd700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 0x7fdb9003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T20:10:58.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.568+0000 7fdba6ffd700 1 -- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 msgr2=0x7fdb9003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/9699389 2023-07-06T20:10:58.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.568+0000 7fdba6ffd700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 0x7fdb9003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-06T20:10:58.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.969+0000 7fdba6ffd700 1 -- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 msgr2=0x7fdb9003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/9699389 2023-07-06T20:10:58.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:51.969+0000 7fdba6ffd700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 0x7fdb9003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-06T20:10:58.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:52.770+0000 7fdba6ffd700 1 -- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 msgr2=0x7fdb9003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/9699389 2023-07-06T20:10:58.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:52.770+0000 7fdba6ffd700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 0x7fdb9003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-06T20:10:58.821 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:54.371+0000 7fdba6ffd700 1 -- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 msgr2=0x7fdb9003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/9699389 2023-07-06T20:10:58.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:54.373+0000 7fdba6ffd700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 0x7fdb9003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-06T20:10:58.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:57.440+0000 7fdba4ff9700 1 -- 172.21.15.69:0/1683757531 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7fdb9802c9b0 con 0x7fdba807ac60 2023-07-06T20:10:58.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:57.440+0000 7fdba4ff9700 1 -- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 msgr2=0x7fdb9003ac40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T20:10:58.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:57.440+0000 7fdba4ff9700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 0x7fdb9003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:58.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.439+0000 7fdba4ff9700 1 -- 172.21.15.69:0/1683757531 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fdb9804e7a0 con 0x7fdba807ac60 2023-07-06T20:10:58.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.440+0000 7fdba4ff9700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fdb9003bff0 0x7fdb9003e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:10:58.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.440+0000 7fdba4ff9700 1 -- 172.21.15.69:0/1683757531 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb9003b350 con 0x7fdb9003bff0 2023-07-06T20:10:58.822 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.440+0000 7fdba6ffd700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fdb9003bff0 0x7fdb9003e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:10:58.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.442+0000 7fdba6ffd700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fdb9003bff0 0x7fdb9003e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c003a10 tx=0x7fdb9c009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:10:58.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.442+0000 7fdba4ff9700 1 -- 172.21.15.69:0/1683757531 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fdb9003b350 con 0x7fdb9003bff0 2023-07-06T20:10:58.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.449+0000 7fdbade74700 1 -- 172.21.15.69:0/1683757531 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fdba819c8a0 con 0x7fdb9003bff0 2023-07-06T20:10:58.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.450+0000 7fdba4ff9700 1 -- 172.21.15.69:0/1683757531 <== mgr.14152 v2:172.21.15.69:6800/4215258217 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fdba819c8a0 con 0x7fdb9003bff0 2023-07-06T20:10:58.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.450+0000 7fdbade74700 1 -- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fdb9003bff0 msgr2=0x7fdb9003e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:58.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.450+0000 7fdbade74700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fdb9003bff0 0x7fdb9003e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c003a10 tx=0x7fdb9c009270 comp rx=0 tx=0).stop 2023-07-06T20:10:58.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.450+0000 7fdbade74700 1 -- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 msgr2=0x7fdba819b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:10:58.823 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.450+0000 7fdbade74700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 0x7fdba819b560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb98003e60 tx=0x7fdb98011a30 comp rx=0 tx=0).stop 2023-07-06T20:10:58.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.450+0000 7fdbade74700 1 -- 172.21.15.69:0/1683757531 shutdown_connections 2023-07-06T20:10:58.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.450+0000 7fdbade74700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fdb9003bff0 0x7fdb9003e3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:58.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.450+0000 7fdbade74700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdba807ac60 0x7fdba819b560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:58.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.450+0000 7fdbade74700 1 --2- 172.21.15.69:0/1683757531 >> [v2:172.21.15.69:6800/9699389,v1:172.21.15.69:6801/9699389] conn(0x7fdb90038790 0x7fdb9003ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:10:58.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.450+0000 7fdbade74700 1 -- 172.21.15.69:0/1683757531 >> 172.21.15.69:0/1683757531 conn(0x7fdba8100b90 msgr2=0x7fdba81022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:10:58.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.450+0000 7fdbade74700 1 -- 172.21.15.69:0/1683757531 shutdown_connections 2023-07-06T20:10:58.824 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:58.450+0000 7fdbade74700 1 -- 172.21.15.69:0/1683757531 wait complete. 2023-07-06T20:10:58.825 INFO:teuthology.orchestra.run.smithi069.stdout:mgr epoch 9 is available 2023-07-06T20:10:58.825 INFO:teuthology.orchestra.run.smithi069.stdout:Generating a dashboard self-signed certificate... 2023-07-06T20:10:59.713 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:59 smithi069 ceph-mon[126399]: mgrmap e11: x(active, since 1.00932s) 2023-07-06T20:10:59.713 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:59 smithi069 ceph-mon[126399]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-06T20:10:59.713 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:10:59 smithi069 ceph-mon[126399]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-06T20:11:00.198 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-06T20:11:00.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.399+0000 7fb39ba4f700 1 Processor -- start 2023-07-06T20:11:00.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.399+0000 7fb39ba4f700 1 -- start start 2023-07-06T20:11:00.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.399+0000 7fb39ba4f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 0x7fb394079160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:00.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.399+0000 7fb39ba4f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3940796a0 con 0x7fb39407ac60 2023-07-06T20:11:00.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.400+0000 7fb3997eb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 0x7fb394079160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:00.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.400+0000 7fb3997eb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 0x7fb394079160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55616/0 (socket says 172.21.15.69:55616) 2023-07-06T20:11:00.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.400+0000 7fb3997eb700 1 -- 172.21.15.69:0/4110461185 learned_addr learned my addr 172.21.15.69:0/4110461185 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:00.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.400+0000 7fb3997eb700 1 -- 172.21.15.69:0/4110461185 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3940797e0 con 0x7fb39407ac60 2023-07-06T20:11:00.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.400+0000 7fb3997eb700 1 --2- 172.21.15.69:0/4110461185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 0x7fb394079160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb3900098a0 tx=0x7fb390009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57a821e2602c49b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:00.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.401+0000 7fb38bfff700 1 -- 172.21.15.69:0/4110461185 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb390004160 con 0x7fb39407ac60 2023-07-06T20:11:00.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.401+0000 7fb38bfff700 1 -- 172.21.15.69:0/4110461185 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3900042c0 con 0x7fb39407ac60 2023-07-06T20:11:00.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.401+0000 7fb38bfff700 1 -- 172.21.15.69:0/4110461185 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3900119b0 con 0x7fb39407ac60 2023-07-06T20:11:00.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.401+0000 7fb39ba4f700 1 -- 172.21.15.69:0/4110461185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 msgr2=0x7fb394079160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:00.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.401+0000 7fb39ba4f700 1 --2- 172.21.15.69:0/4110461185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 0x7fb394079160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb3900098a0 tx=0x7fb390009bb0 comp rx=0 tx=0).stop 2023-07-06T20:11:00.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.401+0000 7fb39ba4f700 1 -- 172.21.15.69:0/4110461185 shutdown_connections 2023-07-06T20:11:00.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.401+0000 7fb39ba4f700 1 --2- 172.21.15.69:0/4110461185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 0x7fb394079160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:00.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.401+0000 7fb39ba4f700 1 -- 172.21.15.69:0/4110461185 >> 172.21.15.69:0/4110461185 conn(0x7fb394100b90 msgr2=0x7fb394102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:00.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.401+0000 7fb39ba4f700 1 -- 172.21.15.69:0/4110461185 shutdown_connections 2023-07-06T20:11:00.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.401+0000 7fb39ba4f700 1 -- 172.21.15.69:0/4110461185 wait complete. 2023-07-06T20:11:00.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.402+0000 7fb39ba4f700 1 Processor -- start 2023-07-06T20:11:00.202 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.402+0000 7fb39ba4f700 1 -- start start 2023-07-06T20:11:00.202 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.403+0000 7fb39ba4f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 0x7fb39419b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:00.202 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.403+0000 7fb39ba4f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39419baa0 con 0x7fb39407ac60 2023-07-06T20:11:00.202 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.403+0000 7fb3997eb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 0x7fb39419b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:00.202 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.403+0000 7fb3997eb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 0x7fb39419b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55618/0 (socket says 172.21.15.69:55618) 2023-07-06T20:11:00.202 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.403+0000 7fb3997eb700 1 -- 172.21.15.69:0/37066649 learned_addr learned my addr 172.21.15.69:0/37066649 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:00.202 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.403+0000 7fb3997eb700 1 -- 172.21.15.69:0/37066649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb390009550 con 0x7fb39407ac60 2023-07-06T20:11:00.203 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.403+0000 7fb3997eb700 1 --2- 172.21.15.69:0/37066649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 0x7fb39419b560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb390003e60 tx=0x7fb390011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:00.203 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.404+0000 7fb38a7fc700 1 -- 172.21.15.69:0/37066649 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb390003a60 con 0x7fb39407ac60 2023-07-06T20:11:00.203 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.404+0000 7fb38a7fc700 1 -- 172.21.15.69:0/37066649 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb390003bc0 con 0x7fb39407ac60 2023-07-06T20:11:00.203 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.404+0000 7fb38a7fc700 1 -- 172.21.15.69:0/37066649 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3900196c0 con 0x7fb39407ac60 2023-07-06T20:11:00.203 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.404+0000 7fb39ba4f700 1 -- 172.21.15.69:0/37066649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39419bca0 con 0x7fb39407ac60 2023-07-06T20:11:00.203 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.404+0000 7fb39ba4f700 1 -- 172.21.15.69:0/37066649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39419c0c0 con 0x7fb39407ac60 2023-07-06T20:11:00.203 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.405+0000 7fb38a7fc700 1 -- 172.21.15.69:0/37066649 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fb39001ab60 con 0x7fb39407ac60 2023-07-06T20:11:00.203 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.405+0000 7fb39ba4f700 1 -- 172.21.15.69:0/37066649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb394061c70 con 0x7fb39407ac60 2023-07-06T20:11:00.204 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.405+0000 7fb38a7fc700 1 --2- 172.21.15.69:0/37066649 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb380038680 0x7fb38003ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:00.204 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.405+0000 7fb38a7fc700 1 -- 172.21.15.69:0/37066649 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb39004c8d0 con 0x7fb39407ac60 2023-07-06T20:11:00.204 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.406+0000 7fb398fea700 1 --2- 172.21.15.69:0/37066649 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb380038680 0x7fb38003ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:00.204 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.406+0000 7fb398fea700 1 --2- 172.21.15.69:0/37066649 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb380038680 0x7fb38003ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb384006fd0 tx=0x7fb384006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:00.204 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.411+0000 7fb38a7fc700 1 -- 172.21.15.69:0/37066649 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb390018b00 con 0x7fb39407ac60 2023-07-06T20:11:00.204 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.443+0000 7fb38a7fc700 1 -- 172.21.15.69:0/37066649 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fb39001a7b0 con 0x7fb39407ac60 2023-07-06T20:11:00.204 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.626+0000 7fb39ba4f700 1 -- 172.21.15.69:0/37066649 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb394102f00 con 0x7fb380038680 2023-07-06T20:11:00.205 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.685+0000 7fb38a7fc700 1 -- 172.21.15.69:0/37066649 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb394102f00 con 0x7fb380038680 2023-07-06T20:11:00.205 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.687+0000 7fb39ba4f700 1 -- 172.21.15.69:0/37066649 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb380038680 msgr2=0x7fb38003ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:00.205 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.687+0000 7fb39ba4f700 1 --2- 172.21.15.69:0/37066649 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb380038680 0x7fb38003ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb384006fd0 tx=0x7fb384006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:00.205 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.687+0000 7fb39ba4f700 1 -- 172.21.15.69:0/37066649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 msgr2=0x7fb39419b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:00.205 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.687+0000 7fb39ba4f700 1 --2- 172.21.15.69:0/37066649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 0x7fb39419b560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb390003e60 tx=0x7fb390011a30 comp rx=0 tx=0).stop 2023-07-06T20:11:00.205 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.688+0000 7fb39ba4f700 1 -- 172.21.15.69:0/37066649 shutdown_connections 2023-07-06T20:11:00.205 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.688+0000 7fb39ba4f700 1 --2- 172.21.15.69:0/37066649 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb380038680 0x7fb38003ab30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:00.206 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.688+0000 7fb39ba4f700 1 --2- 172.21.15.69:0/37066649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb39407ac60 0x7fb39419b560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:00.206 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.688+0000 7fb39ba4f700 1 -- 172.21.15.69:0/37066649 >> 172.21.15.69:0/37066649 conn(0x7fb394100b90 msgr2=0x7fb3941022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:00.206 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.688+0000 7fb39ba4f700 1 -- 172.21.15.69:0/37066649 shutdown_connections 2023-07-06T20:11:00.206 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:10:59.688+0000 7fb39ba4f700 1 -- 172.21.15.69:0/37066649 wait complete. 2023-07-06T20:11:00.206 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial admin user... 2023-07-06T20:11:00.493 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:00 smithi069 ceph-mon[126399]: mgrmap e12: x(active, since 2s) 2023-07-06T20:11:00.493 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:00 smithi069 ceph-mon[126399]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:00.494 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:00.494 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:00.494 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:01.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:01 smithi069 ceph-mon[126399]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:01.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:01.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:01.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ju0fgVDJHAns/UJQx1PxeeT9OmY/hwkc2BGRGDSBap9rF89pL6hQC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1688674261, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-06T20:11:01.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.738+0000 7f16455e7700 1 Processor -- start 2023-07-06T20:11:01.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.738+0000 7f16455e7700 1 -- start start 2023-07-06T20:11:01.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.738+0000 7f16455e7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 0x7f1640079180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:01.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.738+0000 7f16455e7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16400796c0 con 0x7f164007ac80 2023-07-06T20:11:01.682 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.739+0000 7f163effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 0x7f1640079180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:01.682 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.739+0000 7f163effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 0x7f1640079180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55632/0 (socket says 172.21.15.69:55632) 2023-07-06T20:11:01.682 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.739+0000 7f163effd700 1 -- 172.21.15.69:0/2165290957 learned_addr learned my addr 172.21.15.69:0/2165290957 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:01.682 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.739+0000 7f163effd700 1 -- 172.21.15.69:0/2165290957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1640079800 con 0x7f164007ac80 2023-07-06T20:11:01.682 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.739+0000 7f163effd700 1 --2- 172.21.15.69:0/2165290957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 0x7f1640079180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f16300098a0 tx=0x7f1630009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a82d22c9b75fe3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:01.682 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.740+0000 7f163dffb700 1 -- 172.21.15.69:0/2165290957 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1630004160 con 0x7f164007ac80 2023-07-06T20:11:01.683 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.740+0000 7f163dffb700 1 -- 172.21.15.69:0/2165290957 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16300042c0 con 0x7f164007ac80 2023-07-06T20:11:01.683 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.740+0000 7f163dffb700 1 -- 172.21.15.69:0/2165290957 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16300119b0 con 0x7f164007ac80 2023-07-06T20:11:01.683 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.740+0000 7f16455e7700 1 -- 172.21.15.69:0/2165290957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 msgr2=0x7f1640079180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:01.683 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.740+0000 7f16455e7700 1 --2- 172.21.15.69:0/2165290957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 0x7f1640079180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f16300098a0 tx=0x7f1630009bb0 comp rx=0 tx=0).stop 2023-07-06T20:11:01.683 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.740+0000 7f16455e7700 1 -- 172.21.15.69:0/2165290957 shutdown_connections 2023-07-06T20:11:01.683 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.740+0000 7f16455e7700 1 --2- 172.21.15.69:0/2165290957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 0x7f1640079180 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:01.683 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.740+0000 7f16455e7700 1 -- 172.21.15.69:0/2165290957 >> 172.21.15.69:0/2165290957 conn(0x7f1640100b90 msgr2=0x7f1640102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:01.684 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.740+0000 7f16455e7700 1 -- 172.21.15.69:0/2165290957 shutdown_connections 2023-07-06T20:11:01.684 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.741+0000 7f16455e7700 1 -- 172.21.15.69:0/2165290957 wait complete. 2023-07-06T20:11:01.684 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.742+0000 7f16455e7700 1 Processor -- start 2023-07-06T20:11:01.684 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.742+0000 7f16455e7700 1 -- start start 2023-07-06T20:11:01.684 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.742+0000 7f16455e7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 0x7f164019b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:01.684 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.742+0000 7f16455e7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164019bae0 con 0x7f164007ac80 2023-07-06T20:11:01.685 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.742+0000 7f163effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 0x7f164019b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:01.685 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.742+0000 7f163effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 0x7f164019b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55644/0 (socket says 172.21.15.69:55644) 2023-07-06T20:11:01.685 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.742+0000 7f163effd700 1 -- 172.21.15.69:0/1452439628 learned_addr learned my addr 172.21.15.69:0/1452439628 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:01.685 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.743+0000 7f163effd700 1 -- 172.21.15.69:0/1452439628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1630009550 con 0x7f164007ac80 2023-07-06T20:11:01.685 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.743+0000 7f163effd700 1 --2- 172.21.15.69:0/1452439628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 0x7f164019b5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1630000c00 tx=0x7f1630011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:01.685 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.743+0000 7f1627fff700 1 -- 172.21.15.69:0/1452439628 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1630003aa0 con 0x7f164007ac80 2023-07-06T20:11:01.685 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.743+0000 7f1627fff700 1 -- 172.21.15.69:0/1452439628 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1630003c00 con 0x7f164007ac80 2023-07-06T20:11:01.685 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.743+0000 7f16455e7700 1 -- 172.21.15.69:0/1452439628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164019bce0 con 0x7f164007ac80 2023-07-06T20:11:01.686 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.743+0000 7f1627fff700 1 -- 172.21.15.69:0/1452439628 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1630024800 con 0x7f164007ac80 2023-07-06T20:11:01.686 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.743+0000 7f16455e7700 1 -- 172.21.15.69:0/1452439628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164019c100 con 0x7f164007ac80 2023-07-06T20:11:01.686 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.744+0000 7f1627fff700 1 -- 172.21.15.69:0/1452439628 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f163001a430 con 0x7f164007ac80 2023-07-06T20:11:01.686 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.745+0000 7f16455e7700 1 -- 172.21.15.69:0/1452439628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1640195550 con 0x7f164007ac80 2023-07-06T20:11:01.686 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.745+0000 7f1627fff700 1 --2- 172.21.15.69:0/1452439628 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1628038760 0x7f162803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:01.686 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.745+0000 7f1627fff700 1 -- 172.21.15.69:0/1452439628 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f163004ca20 con 0x7f164007ac80 2023-07-06T20:11:01.686 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.745+0000 7f163e7fc700 1 --2- 172.21.15.69:0/1452439628 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1628038760 0x7f162803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:01.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.746+0000 7f163e7fc700 1 --2- 172.21.15.69:0/1452439628 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1628038760 0x7f162803ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1634006fd0 tx=0x7f1634006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:01.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.750+0000 7f1627fff700 1 -- 172.21.15.69:0/1452439628 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1630018b40 con 0x7f164007ac80 2023-07-06T20:11:01.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:00.968+0000 7f16455e7700 1 -- 172.21.15.69:0/1452439628 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1640061c70 con 0x7f1628038760 2023-07-06T20:11:01.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:01.201+0000 7f1627fff700 1 -- 172.21.15.69:0/1452439628 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f1640061c70 con 0x7f1628038760 2023-07-06T20:11:01.687 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:01.204+0000 7f16455e7700 1 -- 172.21.15.69:0/1452439628 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1628038760 msgr2=0x7f162803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:01.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:01.204+0000 7f16455e7700 1 --2- 172.21.15.69:0/1452439628 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1628038760 0x7f162803ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1634006fd0 tx=0x7f1634006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:01.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:01.204+0000 7f16455e7700 1 -- 172.21.15.69:0/1452439628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 msgr2=0x7f164019b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:01.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:01.204+0000 7f16455e7700 1 --2- 172.21.15.69:0/1452439628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 0x7f164019b5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1630000c00 tx=0x7f1630011a30 comp rx=0 tx=0).stop 2023-07-06T20:11:01.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:01.204+0000 7f16455e7700 1 -- 172.21.15.69:0/1452439628 shutdown_connections 2023-07-06T20:11:01.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:01.204+0000 7f16455e7700 1 --2- 172.21.15.69:0/1452439628 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1628038760 0x7f162803ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:01.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:01.204+0000 7f16455e7700 1 --2- 172.21.15.69:0/1452439628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f164007ac80 0x7f164019b5a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:01.688 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:01.204+0000 7f16455e7700 1 -- 172.21.15.69:0/1452439628 >> 172.21.15.69:0/1452439628 conn(0x7f1640100b90 msgr2=0x7f1640102320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:01.689 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:01.204+0000 7f16455e7700 1 -- 172.21.15.69:0/1452439628 shutdown_connections 2023-07-06T20:11:01.689 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:01.204+0000 7f16455e7700 1 -- 172.21.15.69:0/1452439628 wait complete. 2023-07-06T20:11:01.689 INFO:teuthology.orchestra.run.smithi069.stdout:Fetching dashboard port number... 2023-07-06T20:11:03.163 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 8443 2023-07-06T20:11:03.163 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.235+0000 7fa21ac03700 1 Processor -- start 2023-07-06T20:11:03.163 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.235+0000 7fa21ac03700 1 -- start start 2023-07-06T20:11:03.163 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.236+0000 7fa21ac03700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 0x7fa214105930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:03.164 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.236+0000 7fa21ac03700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa214105e70 con 0x7fa214105520 2023-07-06T20:11:03.164 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.236+0000 7fa21899f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 0x7fa214105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:03.164 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.236+0000 7fa21899f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 0x7fa214105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55660/0 (socket says 172.21.15.69:55660) 2023-07-06T20:11:03.164 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.236+0000 7fa21899f700 1 -- 172.21.15.69:0/489625310 learned_addr learned my addr 172.21.15.69:0/489625310 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:03.164 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.236+0000 7fa21899f700 1 -- 172.21.15.69:0/489625310 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa214105fb0 con 0x7fa214105520 2023-07-06T20:11:03.164 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.237+0000 7fa21899f700 1 --2- 172.21.15.69:0/489625310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 0x7fa214105930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa2080098a0 tx=0x7fa208009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bf3a46381945e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:03.164 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.237+0000 7fa2137fe700 1 -- 172.21.15.69:0/489625310 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa208004160 con 0x7fa214105520 2023-07-06T20:11:03.165 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.237+0000 7fa2137fe700 1 -- 172.21.15.69:0/489625310 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2080042c0 con 0x7fa214105520 2023-07-06T20:11:03.165 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.237+0000 7fa2137fe700 1 -- 172.21.15.69:0/489625310 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2080119b0 con 0x7fa214105520 2023-07-06T20:11:03.165 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.237+0000 7fa21ac03700 1 -- 172.21.15.69:0/489625310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 msgr2=0x7fa214105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:03.165 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.237+0000 7fa21ac03700 1 --2- 172.21.15.69:0/489625310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 0x7fa214105930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa2080098a0 tx=0x7fa208009bb0 comp rx=0 tx=0).stop 2023-07-06T20:11:03.165 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.238+0000 7fa21ac03700 1 -- 172.21.15.69:0/489625310 shutdown_connections 2023-07-06T20:11:03.165 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.238+0000 7fa21ac03700 1 --2- 172.21.15.69:0/489625310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 0x7fa214105930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:03.165 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.238+0000 7fa21ac03700 1 -- 172.21.15.69:0/489625310 >> 172.21.15.69:0/489625310 conn(0x7fa214100bb0 msgr2=0x7fa214102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:03.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.238+0000 7fa21ac03700 1 -- 172.21.15.69:0/489625310 shutdown_connections 2023-07-06T20:11:03.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.238+0000 7fa21ac03700 1 -- 172.21.15.69:0/489625310 wait complete. 2023-07-06T20:11:03.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.239+0000 7fa21ac03700 1 Processor -- start 2023-07-06T20:11:03.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.239+0000 7fa21ac03700 1 -- start start 2023-07-06T20:11:03.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.239+0000 7fa21ac03700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 0x7fa21407bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:03.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.239+0000 7fa21ac03700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21407c120 con 0x7fa214105520 2023-07-06T20:11:03.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.239+0000 7fa21899f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 0x7fa21407bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:03.166 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.240+0000 7fa21899f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 0x7fa21407bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55672/0 (socket says 172.21.15.69:55672) 2023-07-06T20:11:03.167 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.240+0000 7fa21899f700 1 -- 172.21.15.69:0/1301766754 learned_addr learned my addr 172.21.15.69:0/1301766754 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:03.167 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.240+0000 7fa21899f700 1 -- 172.21.15.69:0/1301766754 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa208009550 con 0x7fa214105520 2023-07-06T20:11:03.167 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.240+0000 7fa21899f700 1 --2- 172.21.15.69:0/1301766754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 0x7fa21407bbe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa208011980 tx=0x7fa208011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:03.167 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.240+0000 7fa211ffb700 1 -- 172.21.15.69:0/1301766754 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa208003a60 con 0x7fa214105520 2023-07-06T20:11:03.167 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.240+0000 7fa211ffb700 1 -- 172.21.15.69:0/1301766754 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa208003bc0 con 0x7fa214105520 2023-07-06T20:11:03.167 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.240+0000 7fa21ac03700 1 -- 172.21.15.69:0/1301766754 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21407a230 con 0x7fa214105520 2023-07-06T20:11:03.167 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.240+0000 7fa211ffb700 1 -- 172.21.15.69:0/1301766754 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa20801a670 con 0x7fa214105520 2023-07-06T20:11:03.167 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.240+0000 7fa21ac03700 1 -- 172.21.15.69:0/1301766754 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21407a650 con 0x7fa214105520 2023-07-06T20:11:03.168 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.241+0000 7fa211ffb700 1 -- 172.21.15.69:0/1301766754 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fa208024440 con 0x7fa214105520 2023-07-06T20:11:03.168 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.242+0000 7fa21ac03700 1 -- 172.21.15.69:0/1301766754 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa214061c70 con 0x7fa214105520 2023-07-06T20:11:03.168 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.242+0000 7fa211ffb700 1 --2- 172.21.15.69:0/1301766754 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa2040387b0 0x7fa20403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:03.168 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.242+0000 7fa211ffb700 1 -- 172.21.15.69:0/1301766754 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa20804c9c0 con 0x7fa214105520 2023-07-06T20:11:03.168 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.242+0000 7fa213fff700 1 --2- 172.21.15.69:0/1301766754 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa2040387b0 0x7fa20403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:03.168 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.243+0000 7fa213fff700 1 --2- 172.21.15.69:0/1301766754 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa2040387b0 0x7fa20403ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa200006fd0 tx=0x7fa200006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:03.169 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.247+0000 7fa211ffb700 1 -- 172.21.15.69:0/1301766754 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa2080189a0 con 0x7fa214105520 2023-07-06T20:11:03.169 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.456+0000 7fa21ac03700 1 -- 172.21.15.69:0/1301766754 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fa214107420 con 0x7fa214105520 2023-07-06T20:11:03.169 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.457+0000 7fa211ffb700 1 -- 172.21.15.69:0/1301766754 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fa208018350 con 0x7fa214105520 2023-07-06T20:11:03.169 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.460+0000 7fa21ac03700 1 -- 172.21.15.69:0/1301766754 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa2040387b0 msgr2=0x7fa20403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:03.169 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.460+0000 7fa21ac03700 1 --2- 172.21.15.69:0/1301766754 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa2040387b0 0x7fa20403ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa200006fd0 tx=0x7fa200006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:03.169 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.460+0000 7fa21ac03700 1 -- 172.21.15.69:0/1301766754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 msgr2=0x7fa21407bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:03.169 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.460+0000 7fa21ac03700 1 --2- 172.21.15.69:0/1301766754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 0x7fa21407bbe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa208011980 tx=0x7fa208011a60 comp rx=0 tx=0).stop 2023-07-06T20:11:03.169 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.460+0000 7fa21ac03700 1 -- 172.21.15.69:0/1301766754 shutdown_connections 2023-07-06T20:11:03.170 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.460+0000 7fa21ac03700 1 --2- 172.21.15.69:0/1301766754 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa2040387b0 0x7fa20403ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:03.170 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.460+0000 7fa21ac03700 1 --2- 172.21.15.69:0/1301766754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa214105520 0x7fa21407bbe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:03.170 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.460+0000 7fa21ac03700 1 -- 172.21.15.69:0/1301766754 >> 172.21.15.69:0/1301766754 conn(0x7fa214100bb0 msgr2=0x7fa214102290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:03.170 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.460+0000 7fa21ac03700 1 -- 172.21.15.69:0/1301766754 shutdown_connections 2023-07-06T20:11:03.170 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:02.460+0000 7fa21ac03700 1 -- 172.21.15.69:0/1301766754 wait complete. 2023-07-06T20:11:03.182 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-06T20:11:03.182 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2023-07-06T20:11:03.198 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-06T20:11:03.198 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2023-07-06T20:11:03.198 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2023-07-06T20:11:03.198 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-06T20:11:03.200 INFO:teuthology.orchestra.run.smithi069.stdout:Ceph Dashboard is now available at: 2023-07-06T20:11:03.200 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:11:03.201 INFO:teuthology.orchestra.run.smithi069.stdout: URL: https://smithi069.front.sepia.ceph.com:8443/ 2023-07-06T20:11:03.201 INFO:teuthology.orchestra.run.smithi069.stdout: User: admin 2023-07-06T20:11:03.201 INFO:teuthology.orchestra.run.smithi069.stdout: Password: s9a9iokbjg 2023-07-06T20:11:03.201 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:11:03.201 INFO:teuthology.orchestra.run.smithi069.stdout:Saving cluster configuration to /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config directory 2023-07-06T20:11:03.202 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling autotune for osd_memory_target 2023-07-06T20:11:03.459 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:03 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1301766754' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-06T20:11:04.378 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.719+0000 7f4ec1b13700 1 Processor -- start 2023-07-06T20:11:04.379 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.720+0000 7f4ec1b13700 1 -- start start 2023-07-06T20:11:04.379 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.720+0000 7f4ec1b13700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 0x7f4ebc105950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:04.379 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.720+0000 7f4ec1b13700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc105e90 con 0x7f4ebc105540 2023-07-06T20:11:04.379 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.720+0000 7f4ebb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 0x7f4ebc105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:04.379 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.721+0000 7f4ebb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 0x7f4ebc105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55192/0 (socket says 172.21.15.69:55192) 2023-07-06T20:11:04.379 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.721+0000 7f4ebb7fe700 1 -- 172.21.15.69:0/1991204991 learned_addr learned my addr 172.21.15.69:0/1991204991 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:04.380 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.721+0000 7f4ebb7fe700 1 -- 172.21.15.69:0/1991204991 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc105fd0 con 0x7f4ebc105540 2023-07-06T20:11:04.380 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.721+0000 7f4ebb7fe700 1 --2- 172.21.15.69:0/1991204991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 0x7f4ebc105950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4eac0098a0 tx=0x7f4eac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c348b896a98006b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:04.380 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.721+0000 7f4eba7fc700 1 -- 172.21.15.69:0/1991204991 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4eac004160 con 0x7f4ebc105540 2023-07-06T20:11:04.380 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.721+0000 7f4eba7fc700 1 -- 172.21.15.69:0/1991204991 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eac0042c0 con 0x7f4ebc105540 2023-07-06T20:11:04.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.721+0000 7f4eba7fc700 1 -- 172.21.15.69:0/1991204991 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4eac0119b0 con 0x7f4ebc105540 2023-07-06T20:11:04.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.722+0000 7f4ec1b13700 1 -- 172.21.15.69:0/1991204991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 msgr2=0x7f4ebc105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:04.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.722+0000 7f4ec1b13700 1 --2- 172.21.15.69:0/1991204991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 0x7f4ebc105950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4eac0098a0 tx=0x7f4eac009bb0 comp rx=0 tx=0).stop 2023-07-06T20:11:04.383 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.722+0000 7f4ec1b13700 1 -- 172.21.15.69:0/1991204991 shutdown_connections 2023-07-06T20:11:04.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.722+0000 7f4ec1b13700 1 --2- 172.21.15.69:0/1991204991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 0x7f4ebc105950 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:04.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.722+0000 7f4ec1b13700 1 -- 172.21.15.69:0/1991204991 >> 172.21.15.69:0/1991204991 conn(0x7f4ebc100b90 msgr2=0x7f4ebc102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:04.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.722+0000 7f4ec1b13700 1 -- 172.21.15.69:0/1991204991 shutdown_connections 2023-07-06T20:11:04.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.722+0000 7f4ec1b13700 1 -- 172.21.15.69:0/1991204991 wait complete. 2023-07-06T20:11:04.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.723+0000 7f4ec1b13700 1 Processor -- start 2023-07-06T20:11:04.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.723+0000 7f4ec1b13700 1 -- start start 2023-07-06T20:11:04.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.723+0000 7f4ec1b13700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 0x7f4ebc19b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:04.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.723+0000 7f4ec1b13700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc19bac0 con 0x7f4ebc105540 2023-07-06T20:11:04.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.724+0000 7f4ebb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 0x7f4ebc19b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:04.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.724+0000 7f4ebb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 0x7f4ebc19b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55200/0 (socket says 172.21.15.69:55200) 2023-07-06T20:11:04.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.724+0000 7f4ebb7fe700 1 -- 172.21.15.69:0/2544520563 learned_addr learned my addr 172.21.15.69:0/2544520563 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:04.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.724+0000 7f4ebb7fe700 1 -- 172.21.15.69:0/2544520563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac009550 con 0x7f4ebc105540 2023-07-06T20:11:04.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.724+0000 7f4ebb7fe700 1 --2- 172.21.15.69:0/2544520563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 0x7f4ebc19b580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4eac003f00 tx=0x7f4eac011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:04.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.724+0000 7f4eb8ff9700 1 -- 172.21.15.69:0/2544520563 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4eac003b20 con 0x7f4ebc105540 2023-07-06T20:11:04.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.724+0000 7f4eb8ff9700 1 -- 172.21.15.69:0/2544520563 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eac003c80 con 0x7f4ebc105540 2023-07-06T20:11:04.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.724+0000 7f4ec1b13700 1 -- 172.21.15.69:0/2544520563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ebc19bcc0 con 0x7f4ebc105540 2023-07-06T20:11:04.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.724+0000 7f4eb8ff9700 1 -- 172.21.15.69:0/2544520563 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4eac019770 con 0x7f4ebc105540 2023-07-06T20:11:04.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.725+0000 7f4ec1b13700 1 -- 172.21.15.69:0/2544520563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ebc19c0e0 con 0x7f4ebc105540 2023-07-06T20:11:04.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.726+0000 7f4ec1b13700 1 -- 172.21.15.69:0/2544520563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ebc061c70 con 0x7f4ebc105540 2023-07-06T20:11:04.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.726+0000 7f4eb8ff9700 1 -- 172.21.15.69:0/2544520563 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f4eac01ab60 con 0x7f4ebc105540 2023-07-06T20:11:04.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.726+0000 7f4eb8ff9700 1 --2- 172.21.15.69:0/2544520563 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4ea40387b0 0x7f4ea403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:04.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.726+0000 7f4eb8ff9700 1 -- 172.21.15.69:0/2544520563 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4eac04cb20 con 0x7f4ebc105540 2023-07-06T20:11:04.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.726+0000 7f4ebaffd700 1 --2- 172.21.15.69:0/2544520563 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4ea40387b0 0x7f4ea403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:04.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.727+0000 7f4ebaffd700 1 --2- 172.21.15.69:0/2544520563 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4ea40387b0 0x7f4ea403ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0006fd0 tx=0x7f4eb0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:04.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.731+0000 7f4eb8ff9700 1 -- 172.21.15.69:0/2544520563 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4eac01a430 con 0x7f4ebc105540 2023-07-06T20:11:04.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.925+0000 7f4ec1b13700 1 -- 172.21.15.69:0/2544520563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f4ebc107440 con 0x7f4ebc105540 2023-07-06T20:11:04.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.926+0000 7f4eb8ff9700 1 -- 172.21.15.69:0/2544520563 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f4eac02db40 con 0x7f4ebc105540 2023-07-06T20:11:04.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.928+0000 7f4ec1b13700 1 -- 172.21.15.69:0/2544520563 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4ea40387b0 msgr2=0x7f4ea403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:04.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.928+0000 7f4ec1b13700 1 --2- 172.21.15.69:0/2544520563 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4ea40387b0 0x7f4ea403ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0006fd0 tx=0x7f4eb0006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:04.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.928+0000 7f4ec1b13700 1 -- 172.21.15.69:0/2544520563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 msgr2=0x7f4ebc19b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:04.388 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.928+0000 7f4ec1b13700 1 --2- 172.21.15.69:0/2544520563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 0x7f4ebc19b580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4eac003f00 tx=0x7f4eac011a30 comp rx=0 tx=0).stop 2023-07-06T20:11:04.388 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.928+0000 7f4ec1b13700 1 -- 172.21.15.69:0/2544520563 shutdown_connections 2023-07-06T20:11:04.388 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.928+0000 7f4ec1b13700 1 --2- 172.21.15.69:0/2544520563 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4ea40387b0 0x7f4ea403ac60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:04.388 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.928+0000 7f4ec1b13700 1 --2- 172.21.15.69:0/2544520563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ebc105540 0x7f4ebc19b580 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:04.388 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.928+0000 7f4ec1b13700 1 -- 172.21.15.69:0/2544520563 >> 172.21.15.69:0/2544520563 conn(0x7f4ebc100b90 msgr2=0x7f4ebc102290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:04.388 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.928+0000 7f4ec1b13700 1 -- 172.21.15.69:0/2544520563 shutdown_connections 2023-07-06T20:11:04.389 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:03.928+0000 7f4ec1b13700 1 -- 172.21.15.69:0/2544520563 wait complete. 2023-07-06T20:11:05.363 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:05 smithi069 ceph-mon[126399]: mgrmap e13: x(active, since 6s) 2023-07-06T20:11:05.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.901+0000 7f07a8608700 1 Processor -- start 2023-07-06T20:11:05.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.901+0000 7f07a8608700 1 -- start start 2023-07-06T20:11:05.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.902+0000 7f07a8608700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 0x7f07a0079180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:05.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.902+0000 7f07a8608700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00796c0 con 0x7f07a007ac80 2023-07-06T20:11:05.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.902+0000 7f07a63a4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 0x7f07a0079180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:05.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.902+0000 7f07a63a4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 0x7f07a0079180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55206/0 (socket says 172.21.15.69:55206) 2023-07-06T20:11:05.664 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.902+0000 7f07a63a4700 1 -- 172.21.15.69:0/1447096473 learned_addr learned my addr 172.21.15.69:0/1447096473 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:05.664 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.903+0000 7f07a63a4700 1 -- 172.21.15.69:0/1447096473 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a0079800 con 0x7f07a007ac80 2023-07-06T20:11:05.664 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.903+0000 7f07a63a4700 1 --2- 172.21.15.69:0/1447096473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 0x7f07a0079180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f079c0098a0 tx=0x7f079c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab86e46b9c1b6c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:05.664 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.903+0000 7f07a53a2700 1 -- 172.21.15.69:0/1447096473 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f079c004160 con 0x7f07a007ac80 2023-07-06T20:11:05.664 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.903+0000 7f07a53a2700 1 -- 172.21.15.69:0/1447096473 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f079c0042c0 con 0x7f07a007ac80 2023-07-06T20:11:05.664 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.903+0000 7f07a53a2700 1 -- 172.21.15.69:0/1447096473 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f079c0119b0 con 0x7f07a007ac80 2023-07-06T20:11:05.664 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.904+0000 7f07a8608700 1 -- 172.21.15.69:0/1447096473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 msgr2=0x7f07a0079180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:05.665 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.904+0000 7f07a8608700 1 --2- 172.21.15.69:0/1447096473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 0x7f07a0079180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f079c0098a0 tx=0x7f079c009bb0 comp rx=0 tx=0).stop 2023-07-06T20:11:05.665 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.904+0000 7f07a8608700 1 -- 172.21.15.69:0/1447096473 shutdown_connections 2023-07-06T20:11:05.665 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.904+0000 7f07a8608700 1 --2- 172.21.15.69:0/1447096473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 0x7f07a0079180 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:05.665 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.904+0000 7f07a8608700 1 -- 172.21.15.69:0/1447096473 >> 172.21.15.69:0/1447096473 conn(0x7f07a0100b90 msgr2=0x7f07a0102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:05.665 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.904+0000 7f07a8608700 1 -- 172.21.15.69:0/1447096473 shutdown_connections 2023-07-06T20:11:05.665 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.904+0000 7f07a8608700 1 -- 172.21.15.69:0/1447096473 wait complete. 2023-07-06T20:11:05.665 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.905+0000 7f07a8608700 1 Processor -- start 2023-07-06T20:11:05.665 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.905+0000 7f07a8608700 1 -- start start 2023-07-06T20:11:05.666 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.905+0000 7f07a8608700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 0x7f07a019b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:05.666 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.905+0000 7f07a8608700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a019bae0 con 0x7f07a007ac80 2023-07-06T20:11:05.666 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.906+0000 7f07a63a4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 0x7f07a019b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:05.666 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.906+0000 7f07a63a4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 0x7f07a019b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55216/0 (socket says 172.21.15.69:55216) 2023-07-06T20:11:05.666 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.906+0000 7f07a63a4700 1 -- 172.21.15.69:0/2571816039 learned_addr learned my addr 172.21.15.69:0/2571816039 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:05.666 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.906+0000 7f07a63a4700 1 -- 172.21.15.69:0/2571816039 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c009550 con 0x7f07a007ac80 2023-07-06T20:11:05.666 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.906+0000 7f07a63a4700 1 --2- 172.21.15.69:0/2571816039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 0x7f07a019b5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f079c003e60 tx=0x7f079c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:05.667 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.906+0000 7f07977fe700 1 -- 172.21.15.69:0/2571816039 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f079c0039f0 con 0x7f07a007ac80 2023-07-06T20:11:05.667 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.906+0000 7f07977fe700 1 -- 172.21.15.69:0/2571816039 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f079c003b50 con 0x7f07a007ac80 2023-07-06T20:11:05.667 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.906+0000 7f07a8608700 1 -- 172.21.15.69:0/2571816039 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a019bce0 con 0x7f07a007ac80 2023-07-06T20:11:05.667 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.906+0000 7f07977fe700 1 -- 172.21.15.69:0/2571816039 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f079c01a580 con 0x7f07a007ac80 2023-07-06T20:11:05.667 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.907+0000 7f07a8608700 1 -- 172.21.15.69:0/2571816039 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a019c100 con 0x7f07a007ac80 2023-07-06T20:11:05.667 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.908+0000 7f07a8608700 1 -- 172.21.15.69:0/2571816039 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a0195550 con 0x7f07a007ac80 2023-07-06T20:11:05.667 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.908+0000 7f07977fe700 1 -- 172.21.15.69:0/2571816039 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f079c024430 con 0x7f07a007ac80 2023-07-06T20:11:05.667 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.908+0000 7f07977fe700 1 --2- 172.21.15.69:0/2571816039 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f078c038800 0x7f078c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:05.668 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.908+0000 7f07977fe700 1 -- 172.21.15.69:0/2571816039 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f079c04cab0 con 0x7f07a007ac80 2023-07-06T20:11:05.668 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.909+0000 7f07a5ba3700 1 --2- 172.21.15.69:0/2571816039 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f078c038800 0x7f078c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:05.668 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.909+0000 7f07a5ba3700 1 --2- 172.21.15.69:0/2571816039 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f078c038800 0x7f078c03acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0790006fd0 tx=0x7f0790006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:05.668 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:04.913+0000 7f07977fe700 1 -- 172.21.15.69:0/2571816039 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f079c0189a0 con 0x7f07a007ac80 2023-07-06T20:11:05.668 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:05.216+0000 7f07a8608700 1 -- 172.21.15.69:0/2571816039 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f07a002c8e0 con 0x7f07a007ac80 2023-07-06T20:11:05.668 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:05.218+0000 7f07977fe700 1 -- 172.21.15.69:0/2571816039 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f079c018350 con 0x7f07a007ac80 2023-07-06T20:11:05.668 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:05.220+0000 7f07a8608700 1 -- 172.21.15.69:0/2571816039 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f078c038800 msgr2=0x7f078c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:05.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:05.220+0000 7f07a8608700 1 --2- 172.21.15.69:0/2571816039 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f078c038800 0x7f078c03acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0790006fd0 tx=0x7f0790006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:05.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:05.220+0000 7f07a8608700 1 -- 172.21.15.69:0/2571816039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 msgr2=0x7f07a019b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:05.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:05.220+0000 7f07a8608700 1 --2- 172.21.15.69:0/2571816039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 0x7f07a019b5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f079c003e60 tx=0x7f079c011a30 comp rx=0 tx=0).stop 2023-07-06T20:11:05.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:05.220+0000 7f07a8608700 1 -- 172.21.15.69:0/2571816039 shutdown_connections 2023-07-06T20:11:05.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:05.220+0000 7f07a8608700 1 --2- 172.21.15.69:0/2571816039 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f078c038800 0x7f078c03acb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:05.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:05.220+0000 7f07a8608700 1 --2- 172.21.15.69:0/2571816039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07a007ac80 0x7f07a019b5a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:05.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:05.220+0000 7f07a8608700 1 -- 172.21.15.69:0/2571816039 >> 172.21.15.69:0/2571816039 conn(0x7f07a0100b90 msgr2=0x7f07a01022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:05.669 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:05.221+0000 7f07a8608700 1 -- 172.21.15.69:0/2571816039 shutdown_connections 2023-07-06T20:11:05.670 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2023-07-06T20:11:05.221+0000 7f07a8608700 1 -- 172.21.15.69:0/2571816039 wait complete. 2023-07-06T20:11:05.670 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-06T20:11:05.670 INFO:teuthology.orchestra.run.smithi069.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-06T20:11:05.670 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:11:05.670 INFO:teuthology.orchestra.run.smithi069.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:05.670 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:11:05.670 INFO:teuthology.orchestra.run.smithi069.stdout:Or, if you are only running a single cluster on this host: 2023-07-06T20:11:05.670 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:11:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-06T20:11:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:11:05.671 INFO:teuthology.orchestra.run.smithi069.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-06T20:11:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:11:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: ceph telemetry on 2023-07-06T20:11:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:11:05.671 INFO:teuthology.orchestra.run.smithi069.stdout:For more information see: 2023-07-06T20:11:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:11:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-06T20:11:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:11:05.672 INFO:teuthology.orchestra.run.smithi069.stdout:Bootstrap complete. 2023-07-06T20:11:05.719 INFO:tasks.cephadm:Fetching config... 2023-07-06T20:11:05.721 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:11:05.721 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-06T20:11:05.763 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-06T20:11:05.764 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:11:05.764 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-06T20:11:05.847 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-06T20:11:05.848 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:11:05.848 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-06T20:11:05.955 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-06T20:11:05.955 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:11:05.955 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-06T20:11:06.037 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-06T20:11:06.037 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC2OFHfi5oDewL7brB8IleCloiQOROvtAcE2bJbhzupcDovM/HZH2wyacyaF8AyW920eFAuEb9PHzNZct+ZB+/S4ZPO0AaFrLH7aKekt8sdfch0VJ2nVegkMXRkBVeth3cohXRCIdQfewnlPVo2b6alb5WJCdGerUkj7vOcZCTVswEgOe6puJtSevechEizVTqp1xEM8Q+4Jps8uVVeuZawteRzXHM1hOiaZNdpifRCvI1wuP3mSXSxLNuUbaRDNdKQqux/+QjA0f26fCbn3mggrjMnnN/ikffYQnpC4eCEruHGVq5NA7+2XTEVIsFkl+1OjDOCANI2AfQbmAATCJRKHd+H1T6LTb+1yTrZHuEM6tg9drzyMy3gFl+tBSJesZk1a7EOhCLaQw/Oz6TbAZamTA9Gsv0wcHLtL5dZreCf0a7uts6xMr1QzXtpN6ba4j2sdsCeJ2w7//OFuWt6ZVbHO7oOr8JMoZqbrKuESOJK1iR28n+GAfpClGxn+nKE9c= ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-06T20:11:06.171 INFO:teuthology.orchestra.run.smithi069.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC2OFHfi5oDewL7brB8IleCloiQOROvtAcE2bJbhzupcDovM/HZH2wyacyaF8AyW920eFAuEb9PHzNZct+ZB+/S4ZPO0AaFrLH7aKekt8sdfch0VJ2nVegkMXRkBVeth3cohXRCIdQfewnlPVo2b6alb5WJCdGerUkj7vOcZCTVswEgOe6puJtSevechEizVTqp1xEM8Q+4Jps8uVVeuZawteRzXHM1hOiaZNdpifRCvI1wuP3mSXSxLNuUbaRDNdKQqux/+QjA0f26fCbn3mggrjMnnN/ikffYQnpC4eCEruHGVq5NA7+2XTEVIsFkl+1OjDOCANI2AfQbmAATCJRKHd+H1T6LTb+1yTrZHuEM6tg9drzyMy3gFl+tBSJesZk1a7EOhCLaQw/Oz6TbAZamTA9Gsv0wcHLtL5dZreCf0a7uts6xMr1QzXtpN6ba4j2sdsCeJ2w7//OFuWt6ZVbHO7oOr8JMoZqbrKuESOJK1iR28n+GAfpClGxn+nKE9c= ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:11:06.209 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC2OFHfi5oDewL7brB8IleCloiQOROvtAcE2bJbhzupcDovM/HZH2wyacyaF8AyW920eFAuEb9PHzNZct+ZB+/S4ZPO0AaFrLH7aKekt8sdfch0VJ2nVegkMXRkBVeth3cohXRCIdQfewnlPVo2b6alb5WJCdGerUkj7vOcZCTVswEgOe6puJtSevechEizVTqp1xEM8Q+4Jps8uVVeuZawteRzXHM1hOiaZNdpifRCvI1wuP3mSXSxLNuUbaRDNdKQqux/+QjA0f26fCbn3mggrjMnnN/ikffYQnpC4eCEruHGVq5NA7+2XTEVIsFkl+1OjDOCANI2AfQbmAATCJRKHd+H1T6LTb+1yTrZHuEM6tg9drzyMy3gFl+tBSJesZk1a7EOhCLaQw/Oz6TbAZamTA9Gsv0wcHLtL5dZreCf0a7uts6xMr1QzXtpN6ba4j2sdsCeJ2w7//OFuWt6ZVbHO7oOr8JMoZqbrKuESOJK1iR28n+GAfpClGxn+nKE9c= ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-06T20:11:06.255 INFO:teuthology.orchestra.run.smithi098.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC2OFHfi5oDewL7brB8IleCloiQOROvtAcE2bJbhzupcDovM/HZH2wyacyaF8AyW920eFAuEb9PHzNZct+ZB+/S4ZPO0AaFrLH7aKekt8sdfch0VJ2nVegkMXRkBVeth3cohXRCIdQfewnlPVo2b6alb5WJCdGerUkj7vOcZCTVswEgOe6puJtSevechEizVTqp1xEM8Q+4Jps8uVVeuZawteRzXHM1hOiaZNdpifRCvI1wuP3mSXSxLNuUbaRDNdKQqux/+QjA0f26fCbn3mggrjMnnN/ikffYQnpC4eCEruHGVq5NA7+2XTEVIsFkl+1OjDOCANI2AfQbmAATCJRKHd+H1T6LTb+1yTrZHuEM6tg9drzyMy3gFl+tBSJesZk1a7EOhCLaQw/Oz6TbAZamTA9Gsv0wcHLtL5dZreCf0a7uts6xMr1QzXtpN6ba4j2sdsCeJ2w7//OFuWt6ZVbHO7oOr8JMoZqbrKuESOJK1iR28n+GAfpClGxn+nKE9c= ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:11:06.268 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC2OFHfi5oDewL7brB8IleCloiQOROvtAcE2bJbhzupcDovM/HZH2wyacyaF8AyW920eFAuEb9PHzNZct+ZB+/S4ZPO0AaFrLH7aKekt8sdfch0VJ2nVegkMXRkBVeth3cohXRCIdQfewnlPVo2b6alb5WJCdGerUkj7vOcZCTVswEgOe6puJtSevechEizVTqp1xEM8Q+4Jps8uVVeuZawteRzXHM1hOiaZNdpifRCvI1wuP3mSXSxLNuUbaRDNdKQqux/+QjA0f26fCbn3mggrjMnnN/ikffYQnpC4eCEruHGVq5NA7+2XTEVIsFkl+1OjDOCANI2AfQbmAATCJRKHd+H1T6LTb+1yTrZHuEM6tg9drzyMy3gFl+tBSJesZk1a7EOhCLaQw/Oz6TbAZamTA9Gsv0wcHLtL5dZreCf0a7uts6xMr1QzXtpN6ba4j2sdsCeJ2w7//OFuWt6ZVbHO7oOr8JMoZqbrKuESOJK1iR28n+GAfpClGxn+nKE9c= ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-06T20:11:06.313 INFO:teuthology.orchestra.run.smithi156.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC2OFHfi5oDewL7brB8IleCloiQOROvtAcE2bJbhzupcDovM/HZH2wyacyaF8AyW920eFAuEb9PHzNZct+ZB+/S4ZPO0AaFrLH7aKekt8sdfch0VJ2nVegkMXRkBVeth3cohXRCIdQfewnlPVo2b6alb5WJCdGerUkj7vOcZCTVswEgOe6puJtSevechEizVTqp1xEM8Q+4Jps8uVVeuZawteRzXHM1hOiaZNdpifRCvI1wuP3mSXSxLNuUbaRDNdKQqux/+QjA0f26fCbn3mggrjMnnN/ikffYQnpC4eCEruHGVq5NA7+2XTEVIsFkl+1OjDOCANI2AfQbmAATCJRKHd+H1T6LTb+1yTrZHuEM6tg9drzyMy3gFl+tBSJesZk1a7EOhCLaQw/Oz6TbAZamTA9Gsv0wcHLtL5dZreCf0a7uts6xMr1QzXtpN6ba4j2sdsCeJ2w7//OFuWt6ZVbHO7oOr8JMoZqbrKuESOJK1iR28n+GAfpClGxn+nKE9c= ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:11:06.326 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-06T20:11:06.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:06 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2571816039' entity='client.admin' 2023-07-06T20:11:07.050 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:11:08.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.824+0000 7fba65c58700 1 -- 172.21.15.69:0/2710981950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba600ff710 msgr2=0x7fba600ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:08.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.824+0000 7fba65c58700 1 --2- 172.21.15.69:0/2710981950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba600ff710 0x7fba600ffb20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fba50009910 tx=0x7fba50009c20 comp rx=0 tx=0).stop 2023-07-06T20:11:08.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.825+0000 7fba65c58700 1 -- 172.21.15.69:0/2710981950 shutdown_connections 2023-07-06T20:11:08.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.825+0000 7fba65c58700 1 --2- 172.21.15.69:0/2710981950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba600ff710 0x7fba600ffb20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:08.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.825+0000 7fba65c58700 1 -- 172.21.15.69:0/2710981950 >> 172.21.15.69:0/2710981950 conn(0x7fba600facc0 msgr2=0x7fba600fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:08.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.825+0000 7fba65c58700 1 -- 172.21.15.69:0/2710981950 shutdown_connections 2023-07-06T20:11:08.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.825+0000 7fba65c58700 1 -- 172.21.15.69:0/2710981950 wait complete. 2023-07-06T20:11:08.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.826+0000 7fba65c58700 1 Processor -- start 2023-07-06T20:11:08.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.826+0000 7fba65c58700 1 -- start start 2023-07-06T20:11:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.826+0000 7fba65c58700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba600ff710 0x7fba60196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.826+0000 7fba65c58700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba60196fb0 con 0x7fba600ff710 2023-07-06T20:11:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.826+0000 7fba5f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba600ff710 0x7fba60196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.826+0000 7fba5f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba600ff710 0x7fba60196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55248/0 (socket says 172.21.15.69:55248) 2023-07-06T20:11:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.826+0000 7fba5f7fe700 1 -- 172.21.15.69:0/2708057478 learned_addr learned my addr 172.21.15.69:0/2708057478 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.827+0000 7fba5f7fe700 1 -- 172.21.15.69:0/2708057478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba500095f0 con 0x7fba600ff710 2023-07-06T20:11:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.827+0000 7fba5f7fe700 1 --2- 172.21.15.69:0/2708057478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba600ff710 0x7fba60196a70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fba50004f40 tx=0x7fba500057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.827+0000 7fba5cff9700 1 -- 172.21.15.69:0/2708057478 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba5001d070 con 0x7fba600ff710 2023-07-06T20:11:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.827+0000 7fba5cff9700 1 -- 172.21.15.69:0/2708057478 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba5000bc10 con 0x7fba600ff710 2023-07-06T20:11:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.827+0000 7fba65c58700 1 -- 172.21.15.69:0/2708057478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba601971b0 con 0x7fba600ff710 2023-07-06T20:11:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.827+0000 7fba5cff9700 1 -- 172.21.15.69:0/2708057478 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba50003a70 con 0x7fba600ff710 2023-07-06T20:11:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.827+0000 7fba65c58700 1 -- 172.21.15.69:0/2708057478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba60197650 con 0x7fba600ff710 2023-07-06T20:11:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.829+0000 7fba5cff9700 1 -- 172.21.15.69:0/2708057478 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fba50003d30 con 0x7fba600ff710 2023-07-06T20:11:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.829+0000 7fba65c58700 1 -- 172.21.15.69:0/2708057478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba601906e0 con 0x7fba600ff710 2023-07-06T20:11:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.829+0000 7fba5cff9700 1 --2- 172.21.15.69:0/2708057478 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fba480387b0 0x7fba4803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.829+0000 7fba5cff9700 1 -- 172.21.15.69:0/2708057478 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fba5004c5b0 con 0x7fba600ff710 2023-07-06T20:11:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.829+0000 7fba5effd700 1 --2- 172.21.15.69:0/2708057478 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fba480387b0 0x7fba4803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:08.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.830+0000 7fba5effd700 1 --2- 172.21.15.69:0/2708057478 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fba480387b0 0x7fba4803ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fba54006fd0 tx=0x7fba54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:08.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:08.834+0000 7fba5cff9700 1 -- 172.21.15.69:0/2708057478 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fba50029770 con 0x7fba600ff710 2023-07-06T20:11:09.038 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:09.035+0000 7fba65c58700 1 -- 172.21.15.69:0/2708057478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fba6002c8e0 con 0x7fba600ff710 2023-07-06T20:11:09.041 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:09.039+0000 7fba5cff9700 1 -- 172.21.15.69:0/2708057478 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fba50026030 con 0x7fba600ff710 2023-07-06T20:11:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:09.042+0000 7fba65c58700 1 -- 172.21.15.69:0/2708057478 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fba480387b0 msgr2=0x7fba4803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:09.042+0000 7fba65c58700 1 --2- 172.21.15.69:0/2708057478 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fba480387b0 0x7fba4803ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fba54006fd0 tx=0x7fba54006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:09.042+0000 7fba65c58700 1 -- 172.21.15.69:0/2708057478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba600ff710 msgr2=0x7fba60196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:09.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:09.042+0000 7fba65c58700 1 --2- 172.21.15.69:0/2708057478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba600ff710 0x7fba60196a70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fba50004f40 tx=0x7fba500057c0 comp rx=0 tx=0).stop 2023-07-06T20:11:09.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:09.042+0000 7fba65c58700 1 -- 172.21.15.69:0/2708057478 shutdown_connections 2023-07-06T20:11:09.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:09.042+0000 7fba65c58700 1 --2- 172.21.15.69:0/2708057478 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fba480387b0 0x7fba4803ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:09.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:09.042+0000 7fba65c58700 1 --2- 172.21.15.69:0/2708057478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba600ff710 0x7fba60196a70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:09.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:09.042+0000 7fba65c58700 1 -- 172.21.15.69:0/2708057478 >> 172.21.15.69:0/2708057478 conn(0x7fba600facc0 msgr2=0x7fba600fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:09.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:09.042+0000 7fba65c58700 1 -- 172.21.15.69:0/2708057478 shutdown_connections 2023-07-06T20:11:09.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:09.043+0000 7fba65c58700 1 -- 172.21.15.69:0/2708057478 wait complete. 2023-07-06T20:11:09.447 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-06T20:11:09.448 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-06T20:11:10.176 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:11:10.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:10 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2708057478' entity='client.admin' 2023-07-06T20:11:10.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:10.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:10.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:11:10.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:10.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:11:10.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:11.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.748+0000 7f03ce95e700 1 -- 172.21.15.69:0/179279248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c80745f0 msgr2=0x7f03c8072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:11.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.748+0000 7f03ce95e700 1 --2- 172.21.15.69:0/179279248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c80745f0 0x7f03c8072a50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009910 tx=0x7f03b8009c20 comp rx=0 tx=0).stop 2023-07-06T20:11:11.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.748+0000 7f03ce95e700 1 -- 172.21.15.69:0/179279248 shutdown_connections 2023-07-06T20:11:11.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.748+0000 7f03ce95e700 1 --2- 172.21.15.69:0/179279248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c80745f0 0x7f03c8072a50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:11.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.748+0000 7f03ce95e700 1 -- 172.21.15.69:0/179279248 >> 172.21.15.69:0/179279248 conn(0x7f03c80facc0 msgr2=0x7f03c80fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:11.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.748+0000 7f03ce95e700 1 -- 172.21.15.69:0/179279248 shutdown_connections 2023-07-06T20:11:11.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.748+0000 7f03ce95e700 1 -- 172.21.15.69:0/179279248 wait complete. 2023-07-06T20:11:11.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.749+0000 7f03ce95e700 1 Processor -- start 2023-07-06T20:11:11.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.749+0000 7f03ce95e700 1 -- start start 2023-07-06T20:11:11.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.750+0000 7f03ce95e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c80745f0 0x7f03c8196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:11.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.750+0000 7f03ce95e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c8196fe0 con 0x7f03c80745f0 2023-07-06T20:11:11.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.750+0000 7f03c7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c80745f0 0x7f03c8196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:11.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.750+0000 7f03c7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c80745f0 0x7f03c8196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55274/0 (socket says 172.21.15.69:55274) 2023-07-06T20:11:11.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.750+0000 7f03c7fff700 1 -- 172.21.15.69:0/364850298 learned_addr learned my addr 172.21.15.69:0/364850298 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:11.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.750+0000 7f03c7fff700 1 -- 172.21.15.69:0/364850298 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b80095f0 con 0x7f03c80745f0 2023-07-06T20:11:11.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.750+0000 7f03c7fff700 1 --2- 172.21.15.69:0/364850298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c80745f0 0x7f03c8196aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005f50 tx=0x7f03b80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:11.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.751+0000 7f03c57fa700 1 -- 172.21.15.69:0/364850298 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03b801d070 con 0x7f03c80745f0 2023-07-06T20:11:11.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.751+0000 7f03c57fa700 1 -- 172.21.15.69:0/364850298 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03b8022470 con 0x7f03c80745f0 2023-07-06T20:11:11.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.751+0000 7f03ce95e700 1 -- 172.21.15.69:0/364850298 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c81971e0 con 0x7f03c80745f0 2023-07-06T20:11:11.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.751+0000 7f03c57fa700 1 -- 172.21.15.69:0/364850298 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03b800f5f0 con 0x7f03c80745f0 2023-07-06T20:11:11.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.751+0000 7f03ce95e700 1 -- 172.21.15.69:0/364850298 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c8197600 con 0x7f03c80745f0 2023-07-06T20:11:11.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.752+0000 7f03c57fa700 1 -- 172.21.15.69:0/364850298 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f03b800ba90 con 0x7f03c80745f0 2023-07-06T20:11:11.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.752+0000 7f03ce95e700 1 -- 172.21.15.69:0/364850298 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c8190410 con 0x7f03c80745f0 2023-07-06T20:11:11.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.752+0000 7f03c57fa700 1 --2- 172.21.15.69:0/364850298 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f03b00387b0 0x7f03b003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:11.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.752+0000 7f03c57fa700 1 -- 172.21.15.69:0/364850298 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f03b804cbb0 con 0x7f03c80745f0 2023-07-06T20:11:11.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.753+0000 7f03c77fe700 1 --2- 172.21.15.69:0/364850298 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f03b00387b0 0x7f03b003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:11.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.753+0000 7f03c77fe700 1 --2- 172.21.15.69:0/364850298 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f03b00387b0 0x7f03b003ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f03bc006fd0 tx=0x7f03bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:11.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.757+0000 7f03c57fa700 1 -- 172.21.15.69:0/364850298 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f03b802bba0 con 0x7f03c80745f0 2023-07-06T20:11:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.958+0000 7f03ce95e700 1 -- 172.21.15.69:0/364850298 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f03c8060a80 con 0x7f03b00387b0 2023-07-06T20:11:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.961+0000 7f03c57fa700 1 -- 172.21.15.69:0/364850298 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f03c8060a80 con 0x7f03b00387b0 2023-07-06T20:11:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.964+0000 7f03ce95e700 1 -- 172.21.15.69:0/364850298 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f03b00387b0 msgr2=0x7f03b003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.964+0000 7f03ce95e700 1 --2- 172.21.15.69:0/364850298 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f03b00387b0 0x7f03b003ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f03bc006fd0 tx=0x7f03bc006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.964+0000 7f03ce95e700 1 -- 172.21.15.69:0/364850298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c80745f0 msgr2=0x7f03c8196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.964+0000 7f03ce95e700 1 --2- 172.21.15.69:0/364850298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c80745f0 0x7f03c8196aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005f50 tx=0x7f03b80057c0 comp rx=0 tx=0).stop 2023-07-06T20:11:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.964+0000 7f03ce95e700 1 -- 172.21.15.69:0/364850298 shutdown_connections 2023-07-06T20:11:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.964+0000 7f03ce95e700 1 --2- 172.21.15.69:0/364850298 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f03b00387b0 0x7f03b003ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.964+0000 7f03ce95e700 1 --2- 172.21.15.69:0/364850298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03c80745f0 0x7f03c8196aa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.964+0000 7f03ce95e700 1 -- 172.21.15.69:0/364850298 >> 172.21.15.69:0/364850298 conn(0x7f03c80facc0 msgr2=0x7f03c80fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.964+0000 7f03ce95e700 1 -- 172.21.15.69:0/364850298 shutdown_connections 2023-07-06T20:11:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:11.964+0000 7f03ce95e700 1 -- 172.21.15.69:0/364850298 wait complete. 2023-07-06T20:11:12.547 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi098 2023-07-06T20:11:12.547 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-07-06T20:11:12.547 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.conf 2023-07-06T20:11:12.568 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-07-06T20:11:12.568 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:12.627 INFO:tasks.cephadm:Adding host smithi098 to orchestrator... 2023-07-06T20:11:12.628 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch host add smithi098 2023-07-06T20:11:13.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:12 smithi069 ceph-mon[126399]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:13.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:13.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:11:13.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:13.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:11:13.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:12 smithi069 ceph-mon[126399]: Updating smithi069:/etc/ceph/ceph.conf 2023-07-06T20:11:13.367 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:11:14.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:13 smithi069 ceph-mon[126399]: Updating smithi069:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:14.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.937+0000 7fa5484fe700 1 -- 172.21.15.69:0/3612051347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa540105720 msgr2=0x7fa540105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:14.940 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.937+0000 7fa5484fe700 1 --2- 172.21.15.69:0/3612051347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa540105720 0x7fa540105af0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009910 tx=0x7fa53c009c20 comp rx=0 tx=0).stop 2023-07-06T20:11:14.940 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.937+0000 7fa5484fe700 1 -- 172.21.15.69:0/3612051347 shutdown_connections 2023-07-06T20:11:14.940 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.937+0000 7fa5484fe700 1 --2- 172.21.15.69:0/3612051347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa540105720 0x7fa540105af0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:14.941 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.937+0000 7fa5484fe700 1 -- 172.21.15.69:0/3612051347 >> 172.21.15.69:0/3612051347 conn(0x7fa5400fb260 msgr2=0x7fa5400fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:14.941 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.937+0000 7fa5484fe700 1 -- 172.21.15.69:0/3612051347 shutdown_connections 2023-07-06T20:11:14.941 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.937+0000 7fa5484fe700 1 -- 172.21.15.69:0/3612051347 wait complete. 2023-07-06T20:11:14.942 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.938+0000 7fa5484fe700 1 Processor -- start 2023-07-06T20:11:14.942 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.938+0000 7fa5484fe700 1 -- start start 2023-07-06T20:11:14.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.939+0000 7fa5484fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa540105720 0x7fa54019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:14.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.939+0000 7fa5484fe700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54019ae90 con 0x7fa540105720 2023-07-06T20:11:14.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.939+0000 7fa54629a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa540105720 0x7fa54019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:14.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.939+0000 7fa54629a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa540105720 0x7fa54019a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35184/0 (socket says 172.21.15.69:35184) 2023-07-06T20:11:14.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.939+0000 7fa54629a700 1 -- 172.21.15.69:0/1723836595 learned_addr learned my addr 172.21.15.69:0/1723836595 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:14.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.939+0000 7fa54629a700 1 -- 172.21.15.69:0/1723836595 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c0095f0 con 0x7fa540105720 2023-07-06T20:11:14.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.939+0000 7fa54629a700 1 --2- 172.21.15.69:0/1723836595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa540105720 0x7fa54019a950 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0092e0 tx=0x7fa53c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:14.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.940+0000 7fa5377fe700 1 -- 172.21.15.69:0/1723836595 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa53c01d070 con 0x7fa540105720 2023-07-06T20:11:14.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.940+0000 7fa5377fe700 1 -- 172.21.15.69:0/1723836595 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa53c00bc10 con 0x7fa540105720 2023-07-06T20:11:14.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.940+0000 7fa5484fe700 1 -- 172.21.15.69:0/1723836595 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54019b090 con 0x7fa540105720 2023-07-06T20:11:14.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.940+0000 7fa5377fe700 1 -- 172.21.15.69:0/1723836595 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa53c003930 con 0x7fa540105720 2023-07-06T20:11:14.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.940+0000 7fa5484fe700 1 -- 172.21.15.69:0/1723836595 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5401949d0 con 0x7fa540105720 2023-07-06T20:11:14.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.941+0000 7fa5377fe700 1 -- 172.21.15.69:0/1723836595 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fa53c003b60 con 0x7fa540105720 2023-07-06T20:11:14.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.941+0000 7fa5484fe700 1 -- 172.21.15.69:0/1723836595 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa540061c70 con 0x7fa540105720 2023-07-06T20:11:14.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.941+0000 7fa5377fe700 1 --2- 172.21.15.69:0/1723836595 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa52c038800 0x7fa52c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:14.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.942+0000 7fa5377fe700 1 -- 172.21.15.69:0/1723836595 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa53c04c490 con 0x7fa540105720 2023-07-06T20:11:14.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.942+0000 7fa545a99700 1 --2- 172.21.15.69:0/1723836595 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa52c038800 0x7fa52c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:14.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.942+0000 7fa545a99700 1 --2- 172.21.15.69:0/1723836595 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa52c038800 0x7fa52c03acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa530006fd0 tx=0x7fa530006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:14.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:14.947+0000 7fa5377fe700 1 -- 172.21.15.69:0/1723836595 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa53c00ba70 con 0x7fa540105720 2023-07-06T20:11:15.148 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:15.146+0000 7fa5484fe700 1 -- 172.21.15.69:0/1723836595 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}) v1 -- 0x7fa540195400 con 0x7fa52c038800 2023-07-06T20:11:15.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:14 smithi069 ceph-mon[126399]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:15.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:14 smithi069 ceph-mon[126399]: Updating smithi069:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.client.admin.keyring 2023-07-06T20:11:15.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:15.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:15.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:16.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:16 smithi069 ceph-mon[126399]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:16.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:16 smithi069 ceph-mon[126399]: Deploying cephadm binary to smithi098 2023-07-06T20:11:17.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:17.406+0000 7fa5377fe700 1 -- 172.21.15.69:0/1723836595 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fa540195400 con 0x7fa52c038800 2023-07-06T20:11:17.408 INFO:teuthology.orchestra.run.smithi069.stdout:Added host 'smithi098' with addr '172.21.15.98' 2023-07-06T20:11:17.410 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:17.408+0000 7fa5484fe700 1 -- 172.21.15.69:0/1723836595 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa52c038800 msgr2=0x7fa52c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:17.410 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:17.408+0000 7fa5484fe700 1 --2- 172.21.15.69:0/1723836595 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa52c038800 0x7fa52c03acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa530006fd0 tx=0x7fa530006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:17.410 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:17.408+0000 7fa5484fe700 1 -- 172.21.15.69:0/1723836595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa540105720 msgr2=0x7fa54019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:17.408+0000 7fa5484fe700 1 --2- 172.21.15.69:0/1723836595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa540105720 0x7fa54019a950 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0092e0 tx=0x7fa53c004ff0 comp rx=0 tx=0).stop 2023-07-06T20:11:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:17.408+0000 7fa5484fe700 1 -- 172.21.15.69:0/1723836595 shutdown_connections 2023-07-06T20:11:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:17.408+0000 7fa5484fe700 1 --2- 172.21.15.69:0/1723836595 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa52c038800 0x7fa52c03acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:17.408+0000 7fa5484fe700 1 --2- 172.21.15.69:0/1723836595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa540105720 0x7fa54019a950 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:17.408+0000 7fa5484fe700 1 -- 172.21.15.69:0/1723836595 >> 172.21.15.69:0/1723836595 conn(0x7fa5400fb260 msgr2=0x7fa5400fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:17.411 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:17.408+0000 7fa5484fe700 1 -- 172.21.15.69:0/1723836595 shutdown_connections 2023-07-06T20:11:17.412 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:17.408+0000 7fa5484fe700 1 -- 172.21.15.69:0/1723836595 wait complete. 2023-07-06T20:11:17.846 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch host ls --format=json 2023-07-06T20:11:18.563 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:11:18.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:18 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:18.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:18 smithi069 ceph-mon[126399]: Added host smithi098 2023-07-06T20:11:18.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:18 smithi069 ceph-mon[126399]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:18.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:18 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:11:18.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:18 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:19.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:20.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.044+0000 7fd9540d9700 1 -- 172.21.15.69:0/2380848780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c105720 msgr2=0x7fd94c105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:20.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.044+0000 7fd9540d9700 1 --2- 172.21.15.69:0/2380848780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c105720 0x7fd94c105af0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd948009910 tx=0x7fd948009c20 comp rx=0 tx=0).stop 2023-07-06T20:11:20.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.044+0000 7fd9540d9700 1 -- 172.21.15.69:0/2380848780 shutdown_connections 2023-07-06T20:11:20.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.044+0000 7fd9540d9700 1 --2- 172.21.15.69:0/2380848780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c105720 0x7fd94c105af0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:20.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.044+0000 7fd9540d9700 1 -- 172.21.15.69:0/2380848780 >> 172.21.15.69:0/2380848780 conn(0x7fd94c0fb260 msgr2=0x7fd94c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:20.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.045+0000 7fd9540d9700 1 -- 172.21.15.69:0/2380848780 shutdown_connections 2023-07-06T20:11:20.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.045+0000 7fd9540d9700 1 -- 172.21.15.69:0/2380848780 wait complete. 2023-07-06T20:11:20.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.046+0000 7fd9540d9700 1 Processor -- start 2023-07-06T20:11:20.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.046+0000 7fd9540d9700 1 -- start start 2023-07-06T20:11:20.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.046+0000 7fd9540d9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c105720 0x7fd94c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:20.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.046+0000 7fd9540d9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c19ae90 con 0x7fd94c105720 2023-07-06T20:11:20.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.046+0000 7fd951e75700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c105720 0x7fd94c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:20.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.046+0000 7fd951e75700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c105720 0x7fd94c19a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35194/0 (socket says 172.21.15.69:35194) 2023-07-06T20:11:20.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.046+0000 7fd951e75700 1 -- 172.21.15.69:0/376823153 learned_addr learned my addr 172.21.15.69:0/376823153 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:20.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.047+0000 7fd951e75700 1 -- 172.21.15.69:0/376823153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9480095f0 con 0x7fd94c105720 2023-07-06T20:11:20.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.047+0000 7fd951e75700 1 --2- 172.21.15.69:0/376823153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c105720 0x7fd94c19a950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd948004f40 tx=0x7fd948005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:20.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.047+0000 7fd942ffd700 1 -- 172.21.15.69:0/376823153 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd94801d070 con 0x7fd94c105720 2023-07-06T20:11:20.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.047+0000 7fd9540d9700 1 -- 172.21.15.69:0/376823153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94c19b090 con 0x7fd94c105720 2023-07-06T20:11:20.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.047+0000 7fd942ffd700 1 -- 172.21.15.69:0/376823153 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd94800be20 con 0x7fd94c105720 2023-07-06T20:11:20.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.047+0000 7fd942ffd700 1 -- 172.21.15.69:0/376823153 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd948022d00 con 0x7fd94c105720 2023-07-06T20:11:20.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.047+0000 7fd9540d9700 1 -- 172.21.15.69:0/376823153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94c1949d0 con 0x7fd94c105720 2023-07-06T20:11:20.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.048+0000 7fd942ffd700 1 -- 172.21.15.69:0/376823153 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd948022500 con 0x7fd94c105720 2023-07-06T20:11:20.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.048+0000 7fd9540d9700 1 -- 172.21.15.69:0/376823153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94c061c70 con 0x7fd94c105720 2023-07-06T20:11:20.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.049+0000 7fd942ffd700 1 --2- 172.21.15.69:0/376823153 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd9380387b0 0x7fd93803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:20.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.049+0000 7fd942ffd700 1 -- 172.21.15.69:0/376823153 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd94804c9c0 con 0x7fd94c105720 2023-07-06T20:11:20.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.049+0000 7fd951674700 1 --2- 172.21.15.69:0/376823153 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd9380387b0 0x7fd93803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:20.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.050+0000 7fd951674700 1 --2- 172.21.15.69:0/376823153 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd9380387b0 0x7fd93803ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd93c006fd0 tx=0x7fd93c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:20.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.054+0000 7fd942ffd700 1 -- 172.21.15.69:0/376823153 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd94802ab50 con 0x7fd94c105720 2023-07-06T20:11:20.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.253+0000 7fd9540d9700 1 -- 172.21.15.69:0/376823153 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd94c194c80 con 0x7fd9380387b0 2023-07-06T20:11:20.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.254+0000 7fd942ffd700 1 -- 172.21.15.69:0/376823153 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fd94c194c80 con 0x7fd9380387b0 2023-07-06T20:11:20.256 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:11:20.256 INFO:teuthology.orchestra.run.smithi069.stdout:[{"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}] 2023-07-06T20:11:20.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.256+0000 7fd9540d9700 1 -- 172.21.15.69:0/376823153 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd9380387b0 msgr2=0x7fd93803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:20.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.256+0000 7fd9540d9700 1 --2- 172.21.15.69:0/376823153 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd9380387b0 0x7fd93803ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd93c006fd0 tx=0x7fd93c006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:20.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.256+0000 7fd9540d9700 1 -- 172.21.15.69:0/376823153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c105720 msgr2=0x7fd94c19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:20.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.256+0000 7fd9540d9700 1 --2- 172.21.15.69:0/376823153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c105720 0x7fd94c19a950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd948004f40 tx=0x7fd948005020 comp rx=0 tx=0).stop 2023-07-06T20:11:20.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.256+0000 7fd9540d9700 1 -- 172.21.15.69:0/376823153 shutdown_connections 2023-07-06T20:11:20.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.256+0000 7fd9540d9700 1 --2- 172.21.15.69:0/376823153 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd9380387b0 0x7fd93803ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:20.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.256+0000 7fd9540d9700 1 --2- 172.21.15.69:0/376823153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c105720 0x7fd94c19a950 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:20.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.256+0000 7fd9540d9700 1 -- 172.21.15.69:0/376823153 >> 172.21.15.69:0/376823153 conn(0x7fd94c0fb260 msgr2=0x7fd94c0fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:20.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.257+0000 7fd9540d9700 1 -- 172.21.15.69:0/376823153 shutdown_connections 2023-07-06T20:11:20.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:20.257+0000 7fd9540d9700 1 -- 172.21.15.69:0/376823153 wait complete. 2023-07-06T20:11:20.743 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi156 2023-07-06T20:11:20.744 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-06T20:11:20.744 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.conf 2023-07-06T20:11:20.765 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-06T20:11:20.766 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:20.824 INFO:tasks.cephadm:Adding host smithi156 to orchestrator... 2023-07-06T20:11:20.825 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch host add smithi156 2023-07-06T20:11:21.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:20 smithi069 ceph-mon[126399]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:21.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:21.544 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:11:22.153 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:21 smithi069 ceph-mon[126399]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:11:23.041 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.038+0000 7f099e3a8700 1 -- 172.21.15.69:0/3687329343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0998105720 msgr2=0x7f0998105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:23.041 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.038+0000 7f099e3a8700 1 --2- 172.21.15.69:0/3687329343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0998105720 0x7f0998105af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0988009910 tx=0x7f0988009c20 comp rx=0 tx=0).stop 2023-07-06T20:11:23.041 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.038+0000 7f099e3a8700 1 -- 172.21.15.69:0/3687329343 shutdown_connections 2023-07-06T20:11:23.042 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.038+0000 7f099e3a8700 1 --2- 172.21.15.69:0/3687329343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0998105720 0x7f0998105af0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:23.042 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.038+0000 7f099e3a8700 1 -- 172.21.15.69:0/3687329343 >> 172.21.15.69:0/3687329343 conn(0x7f09980fb260 msgr2=0x7f09980fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:23.042 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.039+0000 7f099e3a8700 1 -- 172.21.15.69:0/3687329343 shutdown_connections 2023-07-06T20:11:23.042 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.039+0000 7f099e3a8700 1 -- 172.21.15.69:0/3687329343 wait complete. 2023-07-06T20:11:23.043 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.040+0000 7f099e3a8700 1 Processor -- start 2023-07-06T20:11:23.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.040+0000 7f099e3a8700 1 -- start start 2023-07-06T20:11:23.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.040+0000 7f099e3a8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0998105720 0x7f0998072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:23.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.040+0000 7f099e3a8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09980748c0 con 0x7f0998105720 2023-07-06T20:11:23.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.040+0000 7f0997fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0998105720 0x7f0998072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:23.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.040+0000 7f0997fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0998105720 0x7f0998072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44794/0 (socket says 172.21.15.69:44794) 2023-07-06T20:11:23.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.040+0000 7f0997fff700 1 -- 172.21.15.69:0/2650625974 learned_addr learned my addr 172.21.15.69:0/2650625974 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:23.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.041+0000 7f0997fff700 1 -- 172.21.15.69:0/2650625974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09880095f0 con 0x7f0998105720 2023-07-06T20:11:23.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.041+0000 7f0997fff700 1 --2- 172.21.15.69:0/2650625974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0998105720 0x7f0998072940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f09880092e0 tx=0x7f0988004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:23.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.041+0000 7f09957fa700 1 -- 172.21.15.69:0/2650625974 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f098801d070 con 0x7f0998105720 2023-07-06T20:11:23.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.041+0000 7f09957fa700 1 -- 172.21.15.69:0/2650625974 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f098800bc10 con 0x7f0998105720 2023-07-06T20:11:23.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.041+0000 7f099e3a8700 1 -- 172.21.15.69:0/2650625974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0998072e80 con 0x7f0998105720 2023-07-06T20:11:23.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.041+0000 7f09957fa700 1 -- 172.21.15.69:0/2650625974 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0988003930 con 0x7f0998105720 2023-07-06T20:11:23.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.041+0000 7f099e3a8700 1 -- 172.21.15.69:0/2650625974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0998073380 con 0x7f0998105720 2023-07-06T20:11:23.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.043+0000 7f09957fa700 1 -- 172.21.15.69:0/2650625974 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f0988003bb0 con 0x7f0998105720 2023-07-06T20:11:23.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.043+0000 7f099e3a8700 1 -- 172.21.15.69:0/2650625974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0998104a10 con 0x7f0998105720 2023-07-06T20:11:23.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.043+0000 7f09957fa700 1 --2- 172.21.15.69:0/2650625974 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f09800387b0 0x7f098003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:23.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.043+0000 7f09957fa700 1 -- 172.21.15.69:0/2650625974 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f098804c520 con 0x7f0998105720 2023-07-06T20:11:23.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.043+0000 7f09977fe700 1 --2- 172.21.15.69:0/2650625974 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f09800387b0 0x7f098003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:23.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.044+0000 7f09977fe700 1 --2- 172.21.15.69:0/2650625974 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f09800387b0 0x7f098003ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f098c006fd0 tx=0x7f098c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:23.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.048+0000 7f09957fa700 1 -- 172.21.15.69:0/2650625974 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f098800ba70 con 0x7f0998105720 2023-07-06T20:11:23.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:23.252+0000 7f099e3a8700 1 -- 172.21.15.69:0/2650625974 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}) v1 -- 0x7f0998073610 con 0x7f09800387b0 2023-07-06T20:11:23.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:22 smithi069 ceph-mon[126399]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:11:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:11:24.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:23 smithi069 ceph-mon[126399]: Updating smithi098:/etc/ceph/ceph.conf 2023-07-06T20:11:24.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:23 smithi069 ceph-mon[126399]: Updating smithi098:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:24.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:23 smithi069 ceph-mon[126399]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:24.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:23 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:24.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:23 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:24.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:23 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:25.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:24 smithi069 ceph-mon[126399]: Updating smithi098:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.client.admin.keyring 2023-07-06T20:11:25.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:24 smithi069 ceph-mon[126399]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:25.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:24 smithi069 ceph-mon[126399]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:25.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:24 smithi069 ceph-mon[126399]: Deploying cephadm binary to smithi156 2023-07-06T20:11:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:25.511+0000 7f09957fa700 1 -- 172.21.15.69:0/2650625974 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0998073610 con 0x7f09800387b0 2023-07-06T20:11:25.513 INFO:teuthology.orchestra.run.smithi069.stdout:Added host 'smithi156' with addr '172.21.15.156' 2023-07-06T20:11:25.516 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:25.513+0000 7f099e3a8700 1 -- 172.21.15.69:0/2650625974 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f09800387b0 msgr2=0x7f098003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:25.516 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:25.513+0000 7f099e3a8700 1 --2- 172.21.15.69:0/2650625974 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f09800387b0 0x7f098003ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f098c006fd0 tx=0x7f098c006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:25.516 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:25.513+0000 7f099e3a8700 1 -- 172.21.15.69:0/2650625974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0998105720 msgr2=0x7f0998072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:25.516 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:25.513+0000 7f099e3a8700 1 --2- 172.21.15.69:0/2650625974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0998105720 0x7f0998072940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f09880092e0 tx=0x7f0988004ff0 comp rx=0 tx=0).stop 2023-07-06T20:11:25.516 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:25.514+0000 7f099e3a8700 1 -- 172.21.15.69:0/2650625974 shutdown_connections 2023-07-06T20:11:25.516 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:25.514+0000 7f099e3a8700 1 --2- 172.21.15.69:0/2650625974 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f09800387b0 0x7f098003ac60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:25.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:25.514+0000 7f099e3a8700 1 --2- 172.21.15.69:0/2650625974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0998105720 0x7f0998072940 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:25.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:25.514+0000 7f099e3a8700 1 -- 172.21.15.69:0/2650625974 >> 172.21.15.69:0/2650625974 conn(0x7f09980fb260 msgr2=0x7f09980fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:25.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:25.514+0000 7f099e3a8700 1 -- 172.21.15.69:0/2650625974 shutdown_connections 2023-07-06T20:11:25.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:25.514+0000 7f099e3a8700 1 -- 172.21.15.69:0/2650625974 wait complete. 2023-07-06T20:11:25.930 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch host ls --format=json 2023-07-06T20:11:26.645 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:11:26.903 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:26 smithi069 ceph-mon[126399]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:26.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:26 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:26.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:26 smithi069 ceph-mon[126399]: Added host smithi156 2023-07-06T20:11:26.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:26 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:11:26.904 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:26 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:27.863 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:28.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.063+0000 7fc5ed674700 1 -- 172.21.15.69:0/692014680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5e8105720 msgr2=0x7fc5e8105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:28.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.063+0000 7fc5ed674700 1 --2- 172.21.15.69:0/692014680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5e8105720 0x7fc5e8105af0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009910 tx=0x7fc5d8009c20 comp rx=0 tx=0).stop 2023-07-06T20:11:28.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.063+0000 7fc5ed674700 1 -- 172.21.15.69:0/692014680 shutdown_connections 2023-07-06T20:11:28.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.063+0000 7fc5ed674700 1 --2- 172.21.15.69:0/692014680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5e8105720 0x7fc5e8105af0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:28.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.063+0000 7fc5ed674700 1 -- 172.21.15.69:0/692014680 >> 172.21.15.69:0/692014680 conn(0x7fc5e80fb260 msgr2=0x7fc5e80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:28.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.063+0000 7fc5ed674700 1 -- 172.21.15.69:0/692014680 shutdown_connections 2023-07-06T20:11:28.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.063+0000 7fc5ed674700 1 -- 172.21.15.69:0/692014680 wait complete. 2023-07-06T20:11:28.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.064+0000 7fc5ed674700 1 Processor -- start 2023-07-06T20:11:28.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.064+0000 7fc5ed674700 1 -- start start 2023-07-06T20:11:28.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.065+0000 7fc5ed674700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5e8105720 0x7fc5e8196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:28.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.065+0000 7fc5ed674700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e81972b0 con 0x7fc5e8105720 2023-07-06T20:11:28.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.065+0000 7fc5e6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5e8105720 0x7fc5e8196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:28.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.065+0000 7fc5e6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5e8105720 0x7fc5e8196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44812/0 (socket says 172.21.15.69:44812) 2023-07-06T20:11:28.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.065+0000 7fc5e6ffd700 1 -- 172.21.15.69:0/4152018494 learned_addr learned my addr 172.21.15.69:0/4152018494 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:28.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.065+0000 7fc5e6ffd700 1 -- 172.21.15.69:0/4152018494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d80095f0 con 0x7fc5e8105720 2023-07-06T20:11:28.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.065+0000 7fc5e6ffd700 1 --2- 172.21.15.69:0/4152018494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5e8105720 0x7fc5e8196d70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80092e0 tx=0x7fc5d8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:28.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.066+0000 7fc5cffff700 1 -- 172.21.15.69:0/4152018494 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5d801d070 con 0x7fc5e8105720 2023-07-06T20:11:28.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.066+0000 7fc5cffff700 1 -- 172.21.15.69:0/4152018494 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5d800bc10 con 0x7fc5e8105720 2023-07-06T20:11:28.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.066+0000 7fc5ed674700 1 -- 172.21.15.69:0/4152018494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e81974b0 con 0x7fc5e8105720 2023-07-06T20:11:28.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.066+0000 7fc5cffff700 1 -- 172.21.15.69:0/4152018494 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5d8003930 con 0x7fc5e8105720 2023-07-06T20:11:28.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.066+0000 7fc5ed674700 1 -- 172.21.15.69:0/4152018494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e8197950 con 0x7fc5e8105720 2023-07-06T20:11:28.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.067+0000 7fc5cffff700 1 -- 172.21.15.69:0/4152018494 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fc5d8003b60 con 0x7fc5e8105720 2023-07-06T20:11:28.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.067+0000 7fc5ed674700 1 -- 172.21.15.69:0/4152018494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e8061c70 con 0x7fc5e8105720 2023-07-06T20:11:28.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.067+0000 7fc5cffff700 1 --2- 172.21.15.69:0/4152018494 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5d0038800 0x7fc5d003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:28.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.067+0000 7fc5cffff700 1 -- 172.21.15.69:0/4152018494 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc5d804c3c0 con 0x7fc5e8105720 2023-07-06T20:11:28.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.068+0000 7fc5e67fc700 1 --2- 172.21.15.69:0/4152018494 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5d0038800 0x7fc5d003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:28.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.068+0000 7fc5e67fc700 1 --2- 172.21.15.69:0/4152018494 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5d0038800 0x7fc5d003acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc006fd0 tx=0x7fc5dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:28.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.073+0000 7fc5cffff700 1 -- 172.21.15.69:0/4152018494 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc5d800ba70 con 0x7fc5e8105720 2023-07-06T20:11:28.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.273+0000 7fc5ed674700 1 -- 172.21.15.69:0/4152018494 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5e8102590 con 0x7fc5d0038800 2023-07-06T20:11:28.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.275+0000 7fc5cffff700 1 -- 172.21.15.69:0/4152018494 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7fc5e8102590 con 0x7fc5d0038800 2023-07-06T20:11:28.277 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:11:28.277 INFO:teuthology.orchestra.run.smithi069.stdout:[{"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}] 2023-07-06T20:11:28.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.277+0000 7fc5ed674700 1 -- 172.21.15.69:0/4152018494 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5d0038800 msgr2=0x7fc5d003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:28.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.277+0000 7fc5ed674700 1 --2- 172.21.15.69:0/4152018494 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5d0038800 0x7fc5d003acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc006fd0 tx=0x7fc5dc006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:28.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.277+0000 7fc5ed674700 1 -- 172.21.15.69:0/4152018494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5e8105720 msgr2=0x7fc5e8196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:28.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.277+0000 7fc5ed674700 1 --2- 172.21.15.69:0/4152018494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5e8105720 0x7fc5e8196d70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80092e0 tx=0x7fc5d8004ff0 comp rx=0 tx=0).stop 2023-07-06T20:11:28.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.277+0000 7fc5ed674700 1 -- 172.21.15.69:0/4152018494 shutdown_connections 2023-07-06T20:11:28.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.277+0000 7fc5ed674700 1 --2- 172.21.15.69:0/4152018494 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5d0038800 0x7fc5d003acb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:28.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.277+0000 7fc5ed674700 1 --2- 172.21.15.69:0/4152018494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5e8105720 0x7fc5e8196d70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:28.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.277+0000 7fc5ed674700 1 -- 172.21.15.69:0/4152018494 >> 172.21.15.69:0/4152018494 conn(0x7fc5e80fb260 msgr2=0x7fc5e8074330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:28.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.278+0000 7fc5ed674700 1 -- 172.21.15.69:0/4152018494 shutdown_connections 2023-07-06T20:11:28.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:28.278+0000 7fc5ed674700 1 -- 172.21.15.69:0/4152018494 wait complete. 2023-07-06T20:11:28.725 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-06T20:11:28.726 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd crush tunables default 2023-07-06T20:11:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:28 smithi069 ceph-mon[126399]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:28 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:29.440 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:11:29.985 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:29 smithi069 ceph-mon[126399]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:11:30.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.846+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2338643872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc105720 msgr2=0x7f3fbc105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:30.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.846+0000 7f3fc1bab700 1 --2- 172.21.15.69:0/2338643872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc105720 0x7f3fbc105af0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009910 tx=0x7f3fac009c20 comp rx=0 tx=0).stop 2023-07-06T20:11:30.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.846+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2338643872 shutdown_connections 2023-07-06T20:11:30.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.846+0000 7f3fc1bab700 1 --2- 172.21.15.69:0/2338643872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc105720 0x7f3fbc105af0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:30.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.846+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2338643872 >> 172.21.15.69:0/2338643872 conn(0x7f3fbc0fb260 msgr2=0x7f3fbc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:30.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.846+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2338643872 shutdown_connections 2023-07-06T20:11:30.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.846+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2338643872 wait complete. 2023-07-06T20:11:30.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.847+0000 7f3fc1bab700 1 Processor -- start 2023-07-06T20:11:30.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.847+0000 7f3fc1bab700 1 -- start start 2023-07-06T20:11:30.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.848+0000 7f3fc1bab700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc105720 0x7f3fbc196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:30.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.848+0000 7f3fc1bab700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1972b0 con 0x7f3fbc105720 2023-07-06T20:11:30.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.848+0000 7f3fbb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc105720 0x7f3fbc196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:30.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.848+0000 7f3fbb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc105720 0x7f3fbc196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44834/0 (socket says 172.21.15.69:44834) 2023-07-06T20:11:30.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.848+0000 7f3fbb7fe700 1 -- 172.21.15.69:0/2293341203 learned_addr learned my addr 172.21.15.69:0/2293341203 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:30.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.848+0000 7f3fbb7fe700 1 -- 172.21.15.69:0/2293341203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac0095f0 con 0x7f3fbc105720 2023-07-06T20:11:30.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.848+0000 7f3fbb7fe700 1 --2- 172.21.15.69:0/2293341203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc105720 0x7f3fbc196d70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3fac004f40 tx=0x7f3fac005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:30.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.849+0000 7f3fb8ff9700 1 -- 172.21.15.69:0/2293341203 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fac01d070 con 0x7f3fbc105720 2023-07-06T20:11:30.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.849+0000 7f3fb8ff9700 1 -- 172.21.15.69:0/2293341203 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fac00be20 con 0x7f3fbc105720 2023-07-06T20:11:30.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.849+0000 7f3fb8ff9700 1 -- 172.21.15.69:0/2293341203 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fac022d00 con 0x7f3fbc105720 2023-07-06T20:11:30.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.849+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2293341203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc1974b0 con 0x7f3fbc105720 2023-07-06T20:11:30.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.849+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2293341203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc197950 con 0x7f3fbc105720 2023-07-06T20:11:30.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.850+0000 7f3fb8ff9700 1 -- 172.21.15.69:0/2293341203 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f3fac022500 con 0x7f3fbc105720 2023-07-06T20:11:30.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.850+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2293341203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc061c70 con 0x7f3fbc105720 2023-07-06T20:11:30.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.850+0000 7f3fb8ff9700 1 --2- 172.21.15.69:0/2293341203 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fa40387b0 0x7f3fa403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:30.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.850+0000 7f3fb8ff9700 1 -- 172.21.15.69:0/2293341203 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3fac04c890 con 0x7f3fbc105720 2023-07-06T20:11:30.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.851+0000 7f3fbaffd700 1 --2- 172.21.15.69:0/2293341203 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fa40387b0 0x7f3fa403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:30.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.851+0000 7f3fbaffd700 1 --2- 172.21.15.69:0/2293341203 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fa40387b0 0x7f3fa403ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006fd0 tx=0x7f3fb0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:30.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:30.855+0000 7f3fb8ff9700 1 -- 172.21.15.69:0/2293341203 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3fac02a540 con 0x7f3fbc105720 2023-07-06T20:11:31.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:31.063+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2293341203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3fbc1980d0 con 0x7f3fbc105720 2023-07-06T20:11:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:30 smithi069 ceph-mon[126399]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:31.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:11:31.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:31.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:11:31.916 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:31.915+0000 7f3fb8ff9700 1 -- 172.21.15.69:0/2293341203 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3fac027020 con 0x7f3fbc105720 2023-07-06T20:11:31.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:31.917+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2293341203 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fa40387b0 msgr2=0x7f3fa403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:31.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:31.917+0000 7f3fc1bab700 1 --2- 172.21.15.69:0/2293341203 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fa40387b0 0x7f3fa403ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006fd0 tx=0x7f3fb0006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:31.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:31.917+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2293341203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc105720 msgr2=0x7f3fbc196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:31.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:31.917+0000 7f3fc1bab700 1 --2- 172.21.15.69:0/2293341203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc105720 0x7f3fbc196d70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3fac004f40 tx=0x7f3fac005020 comp rx=0 tx=0).stop 2023-07-06T20:11:31.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:31.917+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2293341203 shutdown_connections 2023-07-06T20:11:31.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:31.917+0000 7f3fc1bab700 1 --2- 172.21.15.69:0/2293341203 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fa40387b0 0x7f3fa403ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:31.921 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:31.917+0000 7f3fc1bab700 1 --2- 172.21.15.69:0/2293341203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fbc105720 0x7f3fbc196d70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:31.921 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:31.917+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2293341203 >> 172.21.15.69:0/2293341203 conn(0x7f3fbc0fb260 msgr2=0x7f3fbc074330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:31.921 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:31.918+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2293341203 shutdown_connections 2023-07-06T20:11:31.921 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:31.918+0000 7f3fc1bab700 1 -- 172.21.15.69:0/2293341203 wait complete. 2023-07-06T20:11:31.921 INFO:teuthology.orchestra.run.smithi069.stderr:adjusted tunables profile to default 2023-07-06T20:11:32.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:31 smithi069 ceph-mon[126399]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-06T20:11:32.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:31 smithi069 ceph-mon[126399]: Updating smithi156:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:32.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:31 smithi069 ceph-mon[126399]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:32.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:31 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2293341203' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-06T20:11:32.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:32.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:32.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:32.312 INFO:tasks.cephadm:Adding mon.a on smithi069 2023-07-06T20:11:32.313 INFO:tasks.cephadm:Adding mon.b on smithi098 2023-07-06T20:11:32.313 INFO:tasks.cephadm:Adding mon.c on smithi156 2023-07-06T20:11:32.314 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch apply mon '3;smithi069:172.21.15.69=a;smithi098:172.21.15.98=b;smithi156:172.21.15.156=c' 2023-07-06T20:11:32.895 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:33.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:32 smithi069 ceph-mon[126399]: Updating smithi156:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.client.admin.keyring 2023-07-06T20:11:33.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:32 smithi069 ceph-mon[126399]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:33.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:32 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2293341203' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-06T20:11:33.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:32 smithi069 ceph-mon[126399]: osdmap e4: 0 total, 0 up, 0 in 2023-07-06T20:11:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.127+0000 7fe7a975a700 1 -- 172.21.15.156:0/1168256019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a41008e0 msgr2=0x7fe7a4100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.127+0000 7fe7a975a700 1 --2- 172.21.15.156:0/1168256019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a41008e0 0x7fe7a4100cb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe798009910 tx=0x7fe798009c20 comp rx=0 tx=0).stop 2023-07-06T20:11:34.130 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.127+0000 7fe7a975a700 1 -- 172.21.15.156:0/1168256019 shutdown_connections 2023-07-06T20:11:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.127+0000 7fe7a975a700 1 --2- 172.21.15.156:0/1168256019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a41008e0 0x7fe7a4100cb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.127+0000 7fe7a975a700 1 -- 172.21.15.156:0/1168256019 >> 172.21.15.156:0/1168256019 conn(0x7fe7a40fa140 msgr2=0x7fe7a40fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.127+0000 7fe7a975a700 1 -- 172.21.15.156:0/1168256019 shutdown_connections 2023-07-06T20:11:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.127+0000 7fe7a975a700 1 -- 172.21.15.156:0/1168256019 wait complete. 2023-07-06T20:11:34.131 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.128+0000 7fe7a975a700 1 Processor -- start 2023-07-06T20:11:34.132 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.128+0000 7fe7a975a700 1 -- start start 2023-07-06T20:11:34.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.128+0000 7fe7a975a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a41008e0 0x7fe7a4194ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:34.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.128+0000 7fe7a975a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a41951e0 con 0x7fe7a41008e0 2023-07-06T20:11:34.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.129+0000 7fe7a2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a41008e0 0x7fe7a4194ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:34.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.129+0000 7fe7a2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a41008e0 0x7fe7a4194ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.156:59140/0 (socket says 172.21.15.156:59140) 2023-07-06T20:11:34.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.129+0000 7fe7a2ffd700 1 -- 172.21.15.156:0/1808898410 learned_addr learned my addr 172.21.15.156:0/1808898410 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-06T20:11:34.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.129+0000 7fe7a2ffd700 1 -- 172.21.15.156:0/1808898410 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7980095f0 con 0x7fe7a41008e0 2023-07-06T20:11:34.133 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.129+0000 7fe7a2ffd700 1 --2- 172.21.15.156:0/1808898410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a41008e0 0x7fe7a4194ca0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe798006010 tx=0x7fe798004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:34.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.129+0000 7fe78bfff700 1 -- 172.21.15.156:0/1808898410 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe79801d070 con 0x7fe7a41008e0 2023-07-06T20:11:34.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.129+0000 7fe7a975a700 1 -- 172.21.15.156:0/1808898410 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a41953e0 con 0x7fe7a41008e0 2023-07-06T20:11:34.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.129+0000 7fe78bfff700 1 -- 172.21.15.156:0/1808898410 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7980044a0 con 0x7fe7a41008e0 2023-07-06T20:11:34.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.129+0000 7fe78bfff700 1 -- 172.21.15.156:0/1808898410 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe798003a70 con 0x7fe7a41008e0 2023-07-06T20:11:34.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.129+0000 7fe7a975a700 1 -- 172.21.15.156:0/1808898410 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a4195800 con 0x7fe7a41008e0 2023-07-06T20:11:34.134 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.130+0000 7fe78bfff700 1 -- 172.21.15.156:0/1808898410 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fe798003cb0 con 0x7fe7a41008e0 2023-07-06T20:11:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.130+0000 7fe7a975a700 1 -- 172.21.15.156:0/1808898410 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a4061c70 con 0x7fe7a41008e0 2023-07-06T20:11:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.130+0000 7fe78bfff700 1 --2- 172.21.15.156:0/1808898410 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe78c0387b0 0x7fe78c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:34.135 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.130+0000 7fe78bfff700 1 -- 172.21.15.156:0/1808898410 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe79804c700 con 0x7fe7a41008e0 2023-07-06T20:11:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.131+0000 7fe7a27fc700 1 --2- 172.21.15.156:0/1808898410 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe78c0387b0 0x7fe78c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:34.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.131+0000 7fe7a27fc700 1 --2- 172.21.15.156:0/1808898410 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe78c0387b0 0x7fe78c03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe794006fd0 tx=0x7fe794006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:34.138 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.136+0000 7fe78bfff700 1 -- 172.21.15.156:0/1808898410 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe798029540 con 0x7fe7a41008e0 2023-07-06T20:11:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.328+0000 7fe7a975a700 1 -- 172.21.15.156:0/1808898410 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi098:172.21.15.98=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7a4106900 con 0x7fe78c0387b0 2023-07-06T20:11:34.338 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.336+0000 7fe78bfff700 1 -- 172.21.15.156:0/1808898410 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe7a4106900 con 0x7fe78c0387b0 2023-07-06T20:11:34.338 INFO:teuthology.orchestra.run.smithi156.stdout:Scheduled mon update... 2023-07-06T20:11:34.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.337+0000 7fe7a975a700 1 -- 172.21.15.156:0/1808898410 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe78c0387b0 msgr2=0x7fe78c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:34.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.337+0000 7fe7a975a700 1 --2- 172.21.15.156:0/1808898410 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe78c0387b0 0x7fe78c03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe794006fd0 tx=0x7fe794006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:34.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.337+0000 7fe7a975a700 1 -- 172.21.15.156:0/1808898410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a41008e0 msgr2=0x7fe7a4194ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:34.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.337+0000 7fe7a975a700 1 --2- 172.21.15.156:0/1808898410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a41008e0 0x7fe7a4194ca0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe798006010 tx=0x7fe798004ff0 comp rx=0 tx=0).stop 2023-07-06T20:11:34.340 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.337+0000 7fe7a975a700 1 -- 172.21.15.156:0/1808898410 shutdown_connections 2023-07-06T20:11:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.337+0000 7fe7a975a700 1 --2- 172.21.15.156:0/1808898410 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe78c0387b0 0x7fe78c03ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.337+0000 7fe7a975a700 1 --2- 172.21.15.156:0/1808898410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7a41008e0 0x7fe7a4194ca0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.337+0000 7fe7a975a700 1 -- 172.21.15.156:0/1808898410 >> 172.21.15.156:0/1808898410 conn(0x7fe7a40fa140 msgr2=0x7fe7a4103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.337+0000 7fe7a975a700 1 -- 172.21.15.156:0/1808898410 shutdown_connections 2023-07-06T20:11:34.341 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:34.337+0000 7fe7a975a700 1 -- 172.21.15.156:0/1808898410 wait complete. 2023-07-06T20:11:34.754 DEBUG:teuthology.orchestra.run.smithi098:mon.b> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.b.service 2023-07-06T20:11:34.758 DEBUG:teuthology.orchestra.run.smithi156:mon.c> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.c.service 2023-07-06T20:11:34.760 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-06T20:11:34.760 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph mon dump -f json 2023-07-06T20:11:34.789 INFO:journalctl@ceph.mon.c.smithi156.stdout:-- Logs begin at Thu 2023-07-06 19:42:21 UTC. -- 2023-07-06T20:11:34.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:-- Logs begin at Thu 2023-07-06 19:39:25 UTC. -- 2023-07-06T20:11:35.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:34 smithi069 ceph-mon[126399]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:35.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:35.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:11:35.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:35.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:11:35.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:35.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:35.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:35.761 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:36.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:35 smithi069 ceph-mon[126399]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi098:172.21.15.98=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:36.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:35 smithi069 ceph-mon[126399]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi098:172.21.15.98=b;smithi156:172.21.15.156=c;count:3 2023-07-06T20:11:36.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:35 smithi069 ceph-mon[126399]: Deploying daemon mon.c on smithi156 2023-07-06T20:11:37.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:36 smithi069 ceph-mon[126399]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:37.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.502+0000 7fbd84d0c700 1 -- 172.21.15.156:0/3988127024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd800fe580 msgr2=0x7fbd800fe950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:37.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.502+0000 7fbd84d0c700 1 --2- 172.21.15.156:0/3988127024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd800fe580 0x7fbd800fe950 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000b0a0 tx=0x7fbd7000b3b0 comp rx=0 tx=0).stop 2023-07-06T20:11:37.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.502+0000 7fbd84d0c700 1 -- 172.21.15.156:0/3988127024 shutdown_connections 2023-07-06T20:11:37.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.502+0000 7fbd84d0c700 1 --2- 172.21.15.156:0/3988127024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd800fe580 0x7fbd800fe950 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:37.505 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.502+0000 7fbd84d0c700 1 -- 172.21.15.156:0/3988127024 >> 172.21.15.156:0/3988127024 conn(0x7fbd80075040 msgr2=0x7fbd80075440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:37.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.502+0000 7fbd84d0c700 1 -- 172.21.15.156:0/3988127024 shutdown_connections 2023-07-06T20:11:37.506 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.502+0000 7fbd84d0c700 1 -- 172.21.15.156:0/3988127024 wait complete. 2023-07-06T20:11:37.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.503+0000 7fbd84d0c700 1 Processor -- start 2023-07-06T20:11:37.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.503+0000 7fbd84d0c700 1 -- start start 2023-07-06T20:11:37.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.503+0000 7fbd84d0c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd800fe580 0x7fbd8019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:37.507 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.503+0000 7fbd84d0c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8019ae40 con 0x7fbd800fe580 2023-07-06T20:11:37.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.504+0000 7fbd7e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd800fe580 0x7fbd8019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:37.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.504+0000 7fbd7e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd800fe580 0x7fbd8019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.156:59146/0 (socket says 172.21.15.156:59146) 2023-07-06T20:11:37.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.504+0000 7fbd7e59c700 1 -- 172.21.15.156:0/1411805241 learned_addr learned my addr 172.21.15.156:0/1411805241 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-06T20:11:37.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.504+0000 7fbd7e59c700 1 -- 172.21.15.156:0/1411805241 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd70009d00 con 0x7fbd800fe580 2023-07-06T20:11:37.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.504+0000 7fbd7e59c700 1 --2- 172.21.15.156:0/1411805241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd800fe580 0x7fbd8019a900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009610 tx=0x7fbd7000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:37.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.504+0000 7fbd6f7fe700 1 -- 172.21.15.156:0/1411805241 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd7000e040 con 0x7fbd800fe580 2023-07-06T20:11:37.508 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.504+0000 7fbd6f7fe700 1 -- 172.21.15.156:0/1411805241 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd70003c10 con 0x7fbd800fe580 2023-07-06T20:11:37.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.504+0000 7fbd6f7fe700 1 -- 172.21.15.156:0/1411805241 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd7001d600 con 0x7fbd800fe580 2023-07-06T20:11:37.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.504+0000 7fbd84d0c700 1 -- 172.21.15.156:0/1411805241 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8019b040 con 0x7fbd800fe580 2023-07-06T20:11:37.509 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.504+0000 7fbd84d0c700 1 -- 172.21.15.156:0/1411805241 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd80194980 con 0x7fbd800fe580 2023-07-06T20:11:37.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.505+0000 7fbd84d0c700 1 -- 172.21.15.156:0/1411805241 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd80061c70 con 0x7fbd800fe580 2023-07-06T20:11:37.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.505+0000 7fbd6f7fe700 1 -- 172.21.15.156:0/1411805241 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fbd70019070 con 0x7fbd800fe580 2023-07-06T20:11:37.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.506+0000 7fbd6f7fe700 1 --2- 172.21.15.156:0/1411805241 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbd680387b0 0x7fbd6803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:37.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.506+0000 7fbd6f7fe700 1 -- 172.21.15.156:0/1411805241 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbd7004c8c0 con 0x7fbd800fe580 2023-07-06T20:11:37.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.506+0000 7fbd7dd9b700 1 --2- 172.21.15.156:0/1411805241 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbd680387b0 0x7fbd6803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:37.510 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.506+0000 7fbd7dd9b700 1 --2- 172.21.15.156:0/1411805241 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbd680387b0 0x7fbd6803ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbd74006fd0 tx=0x7fbd74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:37.513 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.511+0000 7fbd6f7fe700 1 -- 172.21.15.156:0/1411805241 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbd7001c5b0 con 0x7fbd800fe580 2023-07-06T20:11:37.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.812+0000 7fbd84d0c700 1 -- 172.21.15.156:0/1411805241 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbd80195340 con 0x7fbd800fe580 2023-07-06T20:11:37.816 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.813+0000 7fbd6f7fe700 1 -- 172.21.15.156:0/1411805241 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7fbd70017020 con 0x7fbd800fe580 2023-07-06T20:11:37.816 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:11:37.816 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":1,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","modified":"2023-07-06T20:09:53.314871Z","created":"2023-07-06T20:09:53.314871Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-06T20:11:37.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.816+0000 7fbd84d0c700 1 -- 172.21.15.156:0/1411805241 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbd680387b0 msgr2=0x7fbd6803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:37.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.816+0000 7fbd84d0c700 1 --2- 172.21.15.156:0/1411805241 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbd680387b0 0x7fbd6803ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbd74006fd0 tx=0x7fbd74006e40 comp rx=0 tx=0).stop 2023-07-06T20:11:37.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.816+0000 7fbd84d0c700 1 -- 172.21.15.156:0/1411805241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd800fe580 msgr2=0x7fbd8019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:37.819 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.816+0000 7fbd84d0c700 1 --2- 172.21.15.156:0/1411805241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd800fe580 0x7fbd8019a900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009610 tx=0x7fbd7000bd40 comp rx=0 tx=0).stop 2023-07-06T20:11:37.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.816+0000 7fbd84d0c700 1 -- 172.21.15.156:0/1411805241 shutdown_connections 2023-07-06T20:11:37.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.816+0000 7fbd84d0c700 1 --2- 172.21.15.156:0/1411805241 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbd680387b0 0x7fbd6803ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:37.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.816+0000 7fbd84d0c700 1 --2- 172.21.15.156:0/1411805241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd800fe580 0x7fbd8019a900 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:37.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.816+0000 7fbd84d0c700 1 -- 172.21.15.156:0/1411805241 >> 172.21.15.156:0/1411805241 conn(0x7fbd80075040 msgr2=0x7fbd80103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:37.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.816+0000 7fbd84d0c700 1 -- 172.21.15.156:0/1411805241 shutdown_connections 2023-07-06T20:11:37.820 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:37.816+0000 7fbd84d0c700 1 -- 172.21.15.156:0/1411805241 wait complete. 2023-07-06T20:11:37.821 INFO:teuthology.orchestra.run.smithi156.stderr:dumped monmap epoch 1 2023-07-06T20:11:38.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:38 smithi069 ceph-mon[126399]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:38.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:38 smithi069 ceph-mon[126399]: from='client.? 172.21.15.156:0/1411805241' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-06T20:11:38.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 systemd[1]: Started Ceph mon.c for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:11:38.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).mds e1 new map 2023-07-06T20:11:38.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-06T20:11:38.607 INFO:journalctl@ceph.mon.c.smithi156.stdout: e1 2023-07-06T20:11:38.607 INFO:journalctl@ceph.mon.c.smithi156.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-06T20:11:38.608 INFO:journalctl@ceph.mon.c.smithi156.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-06T20:11:38.608 INFO:journalctl@ceph.mon.c.smithi156.stdout: legacy client fscid: -1 2023-07-06T20:11:38.608 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-07-06T20:11:38.608 INFO:journalctl@ceph.mon.c.smithi156.stdout: No filesystems configured 2023-07-06T20:11:38.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-06T20:11:38.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-06T20:11:38.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-06T20:11:38.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-06T20:11:38.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-06T20:11:38.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-06T20:11:38.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-06T20:11:38.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-06T20:11:38.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-06T20:11:38.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-06T20:11:38.612 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: Updating smithi098:/etc/ceph/ceph.conf 2023-07-06T20:11:38.612 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: Updating smithi098:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:38.612 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:38.612 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.613 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.613 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.613 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: Updating smithi098:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.client.admin.keyring 2023-07-06T20:11:38.614 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:38.614 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:38.614 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: Deploying cephadm binary to smithi156 2023-07-06T20:11:38.614 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:38.615 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.615 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: Added host smithi156 2023-07-06T20:11:38.615 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:11:38.615 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.616 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.616 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:38.616 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.617 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:11:38.617 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:38.617 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.617 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.618 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.618 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.618 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:11:38.618 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:38.619 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:11:38.619 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-06T20:11:38.619 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: Updating smithi156:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:38.619 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:38.620 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2293341203' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-06T20:11:38.620 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.620 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.620 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.621 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: Updating smithi156:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.client.admin.keyring 2023-07-06T20:11:38.621 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:38.621 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2293341203' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-06T20:11:38.622 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: osdmap e4: 0 total, 0 up, 0 in 2023-07-06T20:11:38.622 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:38.622 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.622 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:11:38.623 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:38.623 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:11:38.623 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:38.623 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:38.624 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:38.624 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi098:172.21.15.98=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:38.625 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi098:172.21.15.98=b;smithi156:172.21.15.156=c;count:3 2023-07-06T20:11:38.625 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: Deploying daemon mon.c on smithi156 2023-07-06T20:11:38.625 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:38.625 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:38.626 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: from='client.? 172.21.15.156:0/1411805241' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-06T20:11:38.626 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-06T20:11:38.626 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-06T20:11:38.626 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: expand_channel_meta expand map: {default=false} 2023-07-06T20:11:38.627 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: expand_channel_meta from 'false' to 'false' 2023-07-06T20:11:38.627 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: expand_channel_meta expanded map: {default=false} 2023-07-06T20:11:38.627 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: expand_channel_meta expand map: {default=info} 2023-07-06T20:11:38.627 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: expand_channel_meta from 'info' to 'info' 2023-07-06T20:11:38.628 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: expand_channel_meta expanded map: {default=info} 2023-07-06T20:11:38.628 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: expand_channel_meta expand map: {default=daemon} 2023-07-06T20:11:38.628 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-06T20:11:38.628 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: expand_channel_meta expanded map: {default=daemon} 2023-07-06T20:11:38.629 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: expand_channel_meta expand map: {default=debug} 2023-07-06T20:11:38.629 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: expand_channel_meta from 'debug' to 'debug' 2023-07-06T20:11:38.629 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:38 smithi156 ceph-mon[129280]: expand_channel_meta expanded map: {default=debug} 2023-07-06T20:11:39.289 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-06T20:11:39.289 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph mon dump -f json 2023-07-06T20:11:39.853 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:11:43.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:11:43.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: mon.a calling monitor election 2023-07-06T20:11:43.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:43.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:43.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:43.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:43.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: mon.c calling monitor election 2023-07-06T20:11:43.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:43.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:43.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:43.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:43.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-06T20:11:43.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: monmap e2: 2 mons at {a=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-06T20:11:43.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: fsmap 2023-07-06T20:11:43.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: osdmap e4: 0 total, 0 up, 0 in 2023-07-06T20:11:43.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: mgrmap e13: x(active, since 46s) 2023-07-06T20:11:43.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: overall HEALTH_OK 2023-07-06T20:11:43.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:43.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:43.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:43.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:43.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:44.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:11:44.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: mon.a calling monitor election 2023-07-06T20:11:44.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:44.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:44.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:44.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:44.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: mon.c calling monitor election 2023-07-06T20:11:44.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:44.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:44.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:44.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:44.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-06T20:11:44.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: monmap e2: 2 mons at {a=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-06T20:11:44.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: fsmap 2023-07-06T20:11:44.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: osdmap e4: 0 total, 0 up, 0 in 2023-07-06T20:11:44.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: mgrmap e13: x(active, since 46s) 2023-07-06T20:11:44.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: overall HEALTH_OK 2023-07-06T20:11:44.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:44.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:44.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:44.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:44.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:44.175 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.172+0000 7fec77b58700 1 -- 172.21.15.156:0/3317093282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec5c0035c0 msgr2=0x7fec5c005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:44.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.172+0000 7fec77b58700 1 --2- 172.21.15.156:0/3317093282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec5c0035c0 0x7fec5c005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec700ffc70 tx=0x7fec6c00fe60 comp rx=0 tx=0).stop 2023-07-06T20:11:44.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.172+0000 7fec77b58700 1 -- 172.21.15.156:0/3317093282 shutdown_connections 2023-07-06T20:11:44.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.172+0000 7fec77b58700 1 --2- 172.21.15.156:0/3317093282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec5c0035c0 0x7fec5c005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:44.176 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.172+0000 7fec77b58700 1 -- 172.21.15.156:0/3317093282 >> 172.21.15.156:0/3317093282 conn(0x7fec70074500 msgr2=0x7fec70074900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:44.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.172+0000 7fec77b58700 1 -- 172.21.15.156:0/3317093282 shutdown_connections 2023-07-06T20:11:44.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.172+0000 7fec77b58700 1 -- 172.21.15.156:0/3317093282 wait complete. 2023-07-06T20:11:44.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.173+0000 7fec77b58700 1 Processor -- start 2023-07-06T20:11:44.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.173+0000 7fec77b58700 1 -- start start 2023-07-06T20:11:44.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.173+0000 7fec77b58700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec5c0035c0 0x7fec701999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:44.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.173+0000 7fec77b58700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec70199ef0 0x7fec70193a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:44.177 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.173+0000 7fec77b58700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7019a420 con 0x7fec5c0035c0 2023-07-06T20:11:44.178 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.173+0000 7fec77b58700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7019a590 con 0x7fec70199ef0 2023-07-06T20:11:44.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.173+0000 7fec750f3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec70199ef0 0x7fec70193a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:44.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.173+0000 7fec750f3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec70199ef0 0x7fec70193a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59640/0 (socket says 172.21.15.156:59640) 2023-07-06T20:11:44.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.174+0000 7fec750f3700 1 -- 172.21.15.156:0/1685627934 learned_addr learned my addr 172.21.15.156:0/1685627934 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-06T20:11:44.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.174+0000 7fec750f3700 1 -- 172.21.15.156:0/1685627934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec70199ef0 msgr2=0x7fec70193a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-06T20:11:44.179 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.174+0000 7fec750f3700 1 -- 172.21.15.156:0/1685627934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec70199ef0 msgr2=0x7fec70193a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-06T20:11:44.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.174+0000 7fec750f3700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec70199ef0 0x7fec70193a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-06T20:11:44.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.174+0000 7fec758f4700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec5c0035c0 0x7fec701999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:44.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.174+0000 7fec750f3700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec70199ef0 0x7fec70193a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T20:11:44.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.174+0000 7fec758f4700 1 -- 172.21.15.156:0/1685627934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec70199ef0 msgr2=0x7fec70193a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T20:11:44.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.174+0000 7fec758f4700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec70199ef0 0x7fec70193a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:44.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.174+0000 7fec758f4700 1 -- 172.21.15.156:0/1685627934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c00f8b0 con 0x7fec5c0035c0 2023-07-06T20:11:44.180 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.174+0000 7fec758f4700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec5c0035c0 0x7fec701999b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fec700ffc70 tx=0x7fec6c013c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:44.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.175+0000 7fec66ffd700 1 -- 172.21.15.156:0/1685627934 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fec6c023070 con 0x7fec5c0035c0 2023-07-06T20:11:44.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.175+0000 7fec66ffd700 1 -- 172.21.15.156:0/1685627934 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec6c004040 con 0x7fec5c0035c0 2023-07-06T20:11:44.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.175+0000 7fec77b58700 1 -- 172.21.15.156:0/1685627934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec70194030 con 0x7fec5c0035c0 2023-07-06T20:11:44.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.175+0000 7fec66ffd700 1 -- 172.21.15.156:0/1685627934 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fec6c01cb50 con 0x7fec5c0035c0 2023-07-06T20:11:44.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.175+0000 7fec77b58700 1 -- 172.21.15.156:0/1685627934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec70194600 con 0x7fec5c0035c0 2023-07-06T20:11:44.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.176+0000 7fec66ffd700 1 -- 172.21.15.156:0/1685627934 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fec6c01ccd0 con 0x7fec5c0035c0 2023-07-06T20:11:44.181 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.176+0000 7fec77b58700 1 -- 172.21.15.156:0/1685627934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec70194870 con 0x7fec5c0035c0 2023-07-06T20:11:44.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.176+0000 7fec66ffd700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fec540385f0 0x7fec5403aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:44.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.176+0000 7fec66ffd700 1 -- 172.21.15.156:0/1685627934 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fec6c04d2e0 con 0x7fec5c0035c0 2023-07-06T20:11:44.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.176+0000 7fec750f3700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fec540385f0 0x7fec5403aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:44.182 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.177+0000 7fec750f3700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fec540385f0 0x7fec5403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fec60009180 tx=0x7fec60008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:44.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.182+0000 7fec66ffd700 1 -- 172.21.15.156:0/1685627934 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fec6c01c480 con 0x7fec5c0035c0 2023-07-06T20:11:44.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.446+0000 7fec77b58700 1 -- 172.21.15.156:0/1685627934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fec7019e820 con 0x7fec5c0035c0 2023-07-06T20:11:44.449 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.447+0000 7fec66ffd700 1 -- 172.21.15.156:0/1685627934 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fec6c01b3e0 con 0x7fec5c0035c0 2023-07-06T20:11:44.450 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:11:44.450 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":2,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","modified":"2023-07-06T20:11:38.497639Z","created":"2023-07-06T20:09:53.314871Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-06T20:11:44.451 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.449+0000 7fec77b58700 1 -- 172.21.15.156:0/1685627934 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fec540385f0 msgr2=0x7fec5403aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:44.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.449+0000 7fec77b58700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fec540385f0 0x7fec5403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fec60009180 tx=0x7fec60008040 comp rx=0 tx=0).stop 2023-07-06T20:11:44.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.449+0000 7fec77b58700 1 -- 172.21.15.156:0/1685627934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec5c0035c0 msgr2=0x7fec701999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:44.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.449+0000 7fec77b58700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec5c0035c0 0x7fec701999b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fec700ffc70 tx=0x7fec6c013c90 comp rx=0 tx=0).stop 2023-07-06T20:11:44.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.449+0000 7fec77b58700 1 -- 172.21.15.156:0/1685627934 shutdown_connections 2023-07-06T20:11:44.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.449+0000 7fec77b58700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fec540385f0 0x7fec5403aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:44.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.449+0000 7fec77b58700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec5c0035c0 0x7fec701999b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:44.452 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.449+0000 7fec77b58700 1 --2- 172.21.15.156:0/1685627934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec70199ef0 0x7fec70193a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:44.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.449+0000 7fec77b58700 1 -- 172.21.15.156:0/1685627934 >> 172.21.15.156:0/1685627934 conn(0x7fec70074500 msgr2=0x7fec70102330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:44.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.449+0000 7fec77b58700 1 -- 172.21.15.156:0/1685627934 shutdown_connections 2023-07-06T20:11:44.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:44.449+0000 7fec77b58700 1 -- 172.21.15.156:0/1685627934 wait complete. 2023-07-06T20:11:44.453 INFO:teuthology.orchestra.run.smithi156.stderr:dumped monmap epoch 2 2023-07-06T20:11:44.838 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:44 smithi156 ceph-mon[129280]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:44.838 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:44 smithi156 ceph-mon[129280]: Deploying daemon mon.b on smithi098 2023-07-06T20:11:44.838 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:44 smithi156 ceph-mon[129280]: from='client.? 172.21.15.156:0/1685627934' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-06T20:11:44.838 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:44 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:45.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:44 smithi069 ceph-mon[126399]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:45.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:44 smithi069 ceph-mon[126399]: Deploying daemon mon.b on smithi098 2023-07-06T20:11:45.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:44 smithi069 ceph-mon[126399]: from='client.? 172.21.15.156:0/1685627934' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-06T20:11:45.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:44 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:45.915 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-06T20:11:45.916 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph mon dump -f json 2023-07-06T20:11:46.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 systemd[1]: Starting Ceph mon.b for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:11:46.478 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:11:46.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:46 smithi156 ceph-mon[129280]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 podman[128928]: 2023-07-06T20:11:46.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: set uid:gid to 167:167 (ceph:ceph) 2023-07-06T20:11:46.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev), process ceph-mon, pid 7 2023-07-06T20:11:46.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pidfile_write: ignore empty --pid-file 2023-07-06T20:11:46.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 bash[128928]: cd355ba922fb42181d6866dcc58c346e72753477b1e69f48ec314a070ba66a79 2023-07-06T20:11:46.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: load: jerasure load: lrc 2023-07-06T20:11:46.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: RocksDB version: 7.9.2 2023-07-06T20:11:46.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Git sha 0 2023-07-06T20:11:46.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Compile date 2023-07-05 19:32:32 2023-07-06T20:11:46.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: DB SUMMARY 2023-07-06T20:11:46.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: DB Session ID: 41JKXO4R9KM9IPPCIIH5 2023-07-06T20:11:46.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: CURRENT file: CURRENT 2023-07-06T20:11:46.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: IDENTITY file: IDENTITY 2023-07-06T20:11:46.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-06T20:11:46.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-06T20:11:46.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-06T20:11:46.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.error_if_exists: 0 2023-07-06T20:11:46.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.create_if_missing: 0 2023-07-06T20:11:46.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.paranoid_checks: 1 2023-07-06T20:11:46.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-06T20:11:46.879 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-06T20:11:46.879 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-06T20:11:46.879 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.env: 0x5640655b6720 2023-07-06T20:11:46.879 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.fs: PosixFileSystem 2023-07-06T20:11:46.879 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.info_log: 0x564066d97280 2023-07-06T20:11:46.879 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_file_opening_threads: 16 2023-07-06T20:11:46.879 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.statistics: (nil) 2023-07-06T20:11:46.879 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.use_fsync: 0 2023-07-06T20:11:46.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_log_file_size: 0 2023-07-06T20:11:46.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-06T20:11:46.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-06T20:11:46.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.keep_log_file_num: 1000 2023-07-06T20:11:46.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.recycle_log_file_num: 0 2023-07-06T20:11:46.881 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.allow_fallocate: 1 2023-07-06T20:11:46.881 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.allow_mmap_reads: 0 2023-07-06T20:11:46.881 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.allow_mmap_writes: 0 2023-07-06T20:11:46.881 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.use_direct_reads: 0 2023-07-06T20:11:46.881 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-06T20:11:46.881 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.create_missing_column_families: 0 2023-07-06T20:11:46.881 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.db_log_dir: 2023-07-06T20:11:46.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.wal_dir: 2023-07-06T20:11:46.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-06T20:11:46.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-06T20:11:46.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-06T20:11:46.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-06T20:11:46.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-06T20:11:46.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-06T20:11:46.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.advise_random_on_open: 1 2023-07-06T20:11:46.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.db_write_buffer_size: 0 2023-07-06T20:11:46.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.write_buffer_manager: 0x564066cb83c0 2023-07-06T20:11:46.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-06T20:11:46.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-06T20:11:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-06T20:11:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.rate_limiter: (nil) 2023-07-06T20:11:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-06T20:11:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.wal_recovery_mode: 2 2023-07-06T20:11:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.enable_thread_tracking: 0 2023-07-06T20:11:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.enable_pipelined_write: 0 2023-07-06T20:11:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.unordered_write: 0 2023-07-06T20:11:46.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-06T20:11:46.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-06T20:11:46.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-06T20:11:46.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-06T20:11:46.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.row_cache: None 2023-07-06T20:11:46.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.wal_filter: None 2023-07-06T20:11:46.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-06T20:11:46.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.allow_ingest_behind: 0 2023-07-06T20:11:46.886 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.two_write_queues: 0 2023-07-06T20:11:46.886 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.manual_wal_flush: 0 2023-07-06T20:11:46.886 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.wal_compression: 0 2023-07-06T20:11:46.886 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.atomic_flush: 0 2023-07-06T20:11:46.886 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-06T20:11:46.887 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-06T20:11:46.887 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-06T20:11:46.887 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.log_readahead_size: 0 2023-07-06T20:11:46.887 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-06T20:11:46.887 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.best_efforts_recovery: 0 2023-07-06T20:11:46.887 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-06T20:11:46.887 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-06T20:11:46.889 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.allow_data_in_errors: 0 2023-07-06T20:11:46.889 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.db_host_id: __hostname__ 2023-07-06T20:11:46.889 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-06T20:11:46.889 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_background_jobs: 2 2023-07-06T20:11:46.889 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_background_compactions: -1 2023-07-06T20:11:46.890 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_subcompactions: 1 2023-07-06T20:11:46.890 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-06T20:11:46.890 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-06T20:11:46.890 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-06T20:11:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_total_wal_size: 0 2023-07-06T20:11:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-06T20:11:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-06T20:11:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-06T20:11:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-06T20:11:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_open_files: -1 2023-07-06T20:11:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bytes_per_sync: 0 2023-07-06T20:11:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-06T20:11:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-06T20:11:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_readahead_size: 0 2023-07-06T20:11:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_background_flushes: -1 2023-07-06T20:11:46.894 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Compression algorithms supported: 2023-07-06T20:11:46.894 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-06T20:11:46.894 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: kZSTD supported: 0 2023-07-06T20:11:46.894 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: kXpressCompression supported: 0 2023-07-06T20:11:46.895 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: kLZ4HCCompression supported: 1 2023-07-06T20:11:46.895 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: kZlibCompression supported: 1 2023-07-06T20:11:46.895 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: kSnappyCompression supported: 1 2023-07-06T20:11:46.896 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: kLZ4Compression supported: 1 2023-07-06T20:11:46.896 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: kBZip2Compression supported: 0 2023-07-06T20:11:46.896 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-06T20:11:46.897 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-06T20:11:46.897 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-06T20:11:46.897 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-06T20:11:46.897 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-06T20:11:46.898 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.merge_operator: 2023-07-06T20:11:46.898 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_filter: None 2023-07-06T20:11:46.898 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_filter_factory: None 2023-07-06T20:11:46.898 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.sst_partitioner_factory: None 2023-07-06T20:11:46.898 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-06T20:11:46.899 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-06T20:11:46.899 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564066d97380) 2023-07-06T20:11:46.899 INFO:journalctl@ceph.mon.b.smithi098.stdout: cache_index_and_filter_blocks: 1 2023-07-06T20:11:46.899 INFO:journalctl@ceph.mon.b.smithi098.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-06T20:11:46.899 INFO:journalctl@ceph.mon.b.smithi098.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-06T20:11:46.899 INFO:journalctl@ceph.mon.b.smithi098.stdout: pin_top_level_index_and_filter: 1 2023-07-06T20:11:46.899 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_type: 0 2023-07-06T20:11:46.900 INFO:journalctl@ceph.mon.b.smithi098.stdout: data_block_index_type: 0 2023-07-06T20:11:46.900 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_shortening: 1 2023-07-06T20:11:46.900 INFO:journalctl@ceph.mon.b.smithi098.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-06T20:11:46.900 INFO:journalctl@ceph.mon.b.smithi098.stdout: checksum: 4 2023-07-06T20:11:46.900 INFO:journalctl@ceph.mon.b.smithi098.stdout: no_block_cache: 0 2023-07-06T20:11:46.900 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache: 0x564066d571f0 2023-07-06T20:11:46.900 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_name: BinnedLRUCache 2023-07-06T20:11:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_options: 2023-07-06T20:11:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout: capacity : 536870912 2023-07-06T20:11:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout: num_shard_bits : 4 2023-07-06T20:11:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout: strict_capacity_limit : 0 2023-07-06T20:11:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout: high_pri_pool_ratio: 0.000 2023-07-06T20:11:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_compressed: (nil) 2023-07-06T20:11:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout: persistent_cache: (nil) 2023-07-06T20:11:46.902 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_size: 4096 2023-07-06T20:11:46.902 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_size_deviation: 10 2023-07-06T20:11:46.902 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_restart_interval: 16 2023-07-06T20:11:46.902 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_block_restart_interval: 1 2023-07-06T20:11:46.902 INFO:journalctl@ceph.mon.b.smithi098.stdout: metadata_block_size: 4096 2023-07-06T20:11:46.902 INFO:journalctl@ceph.mon.b.smithi098.stdout: partition_filters: 0 2023-07-06T20:11:46.902 INFO:journalctl@ceph.mon.b.smithi098.stdout: use_delta_encoding: 1 2023-07-06T20:11:46.902 INFO:journalctl@ceph.mon.b.smithi098.stdout: filter_policy: bloomfilter 2023-07-06T20:11:46.903 INFO:journalctl@ceph.mon.b.smithi098.stdout: whole_key_filtering: 1 2023-07-06T20:11:46.903 INFO:journalctl@ceph.mon.b.smithi098.stdout: verify_compression: 0 2023-07-06T20:11:46.903 INFO:journalctl@ceph.mon.b.smithi098.stdout: read_amp_bytes_per_bit: 0 2023-07-06T20:11:46.903 INFO:journalctl@ceph.mon.b.smithi098.stdout: format_version: 5 2023-07-06T20:11:46.903 INFO:journalctl@ceph.mon.b.smithi098.stdout: enable_index_compression: 1 2023-07-06T20:11:46.904 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_align: 0 2023-07-06T20:11:46.904 INFO:journalctl@ceph.mon.b.smithi098.stdout: max_auto_readahead_size: 262144 2023-07-06T20:11:46.904 INFO:journalctl@ceph.mon.b.smithi098.stdout: prepopulate_block_cache: 0 2023-07-06T20:11:46.904 INFO:journalctl@ceph.mon.b.smithi098.stdout: initial_auto_readahead_size: 8192 2023-07-06T20:11:46.904 INFO:journalctl@ceph.mon.b.smithi098.stdout: num_file_reads_for_auto_readahead: 2 2023-07-06T20:11:46.904 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.write_buffer_size: 33554432 2023-07-06T20:11:46.905 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_write_buffer_number: 2 2023-07-06T20:11:46.905 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compression: NoCompression 2023-07-06T20:11:46.905 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bottommost_compression: Disabled 2023-07-06T20:11:46.905 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.prefix_extractor: nullptr 2023-07-06T20:11:46.905 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-06T20:11:46.905 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.num_levels: 7 2023-07-06T20:11:46.906 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-06T20:11:46.906 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-06T20:11:46.906 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-06T20:11:46.906 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-06T20:11:46.906 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-06T20:11:46.906 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-06T20:11:46.906 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-06T20:11:46.907 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-06T20:11:46.907 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-06T20:11:46.907 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-06T20:11:46.907 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-06T20:11:46.907 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-06T20:11:46.907 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-06T20:11:46.907 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compression_opts.level: 32767 2023-07-06T20:11:46.908 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compression_opts.strategy: 0 2023-07-06T20:11:46.908 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-06T20:11:46.908 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-06T20:11:46.908 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-06T20:11:46.908 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-06T20:11:46.909 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compression_opts.enabled: false 2023-07-06T20:11:46.909 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-06T20:11:46.909 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-06T20:11:46.909 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-06T20:11:46.909 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-06T20:11:46.909 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.target_file_size_base: 67108864 2023-07-06T20:11:46.909 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-06T20:11:46.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-06T20:11:46.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-06T20:11:46.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-06T20:11:46.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-06T20:11:46.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-06T20:11:46.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-06T20:11:46.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-06T20:11:46.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-06T20:11:46.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-06T20:11:46.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-06T20:11:46.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-06T20:11:46.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-06T20:11:46.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-06T20:11:46.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.arena_block_size: 1048576 2023-07-06T20:11:46.912 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-06T20:11:46.912 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-06T20:11:46.912 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.disable_auto_compactions: 0 2023-07-06T20:11:46.912 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-06T20:11:46.912 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-06T20:11:46.912 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-06T20:11:46.912 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-06T20:11:46.913 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-06T20:11:46.913 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-06T20:11:46.913 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-06T20:11:46.913 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-06T20:11:46.913 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-06T20:11:46.913 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-06T20:11:46.913 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.table_properties_collectors: 2023-07-06T20:11:46.914 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.inplace_update_support: 0 2023-07-06T20:11:46.914 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-06T20:11:46.914 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-06T20:11:46.914 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-06T20:11:46.914 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-06T20:11:46.914 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.bloom_locality: 0 2023-07-06T20:11:46.914 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.max_successive_merges: 0 2023-07-06T20:11:46.914 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-06T20:11:46.915 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.paranoid_file_checks: 0 2023-07-06T20:11:46.915 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.force_consistency_checks: 1 2023-07-06T20:11:46.915 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.report_bg_io_stats: 0 2023-07-06T20:11:46.915 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.ttl: 2592000 2023-07-06T20:11:46.915 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-06T20:11:46.915 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-06T20:11:46.915 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-06T20:11:46.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.enable_blob_files: false 2023-07-06T20:11:46.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.min_blob_size: 0 2023-07-06T20:11:46.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.blob_file_size: 268435456 2023-07-06T20:11:46.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-06T20:11:46.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-06T20:11:46.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-06T20:11:46.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-06T20:11:46.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-06T20:11:46.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.blob_file_starting_level: 0 2023-07-06T20:11:46.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-06T20:11:46.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-06T20:11:46.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-06T20:11:46.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9f3b7042-f184-475e-8ec3-35eda0be34a8 2023-07-06T20:11:46.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: EVENT_LOG_v1 {"time_micros": 1688674306606252, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-06T20:11:46.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-06T20:11:46.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: EVENT_LOG_v1 {"time_micros": 1688674306606663, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1688674306, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9f3b7042-f184-475e-8ec3-35eda0be34a8", "db_session_id": "41JKXO4R9KM9IPPCIIH5", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-06T20:11:46.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: EVENT_LOG_v1 {"time_micros": 1688674306606748, "job": 1, "event": "recovery_finished"} 2023-07-06T20:11:46.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-06T20:11:46.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-06T20:11:46.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564066df0000 2023-07-06T20:11:46.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: DB pointer 0x564066ddc000 2023-07-06T20:11:46.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-06T20:11:46.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-06T20:11:46.919 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** DB Stats ** 2023-07-06T20:11:46.919 INFO:journalctl@ceph.mon.b.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-06T20:11:46.919 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-06T20:11:46.919 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-06T20:11:46.920 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-06T20:11:46.920 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-06T20:11:46.920 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-06T20:11:46.920 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-06T20:11:46.920 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-07-06T20:11:46.920 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** Compaction Stats [default] ** 2023-07-06T20:11:46.920 INFO:journalctl@ceph.mon.b.smithi098.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-06T20:11:46.921 INFO:journalctl@ceph.mon.b.smithi098.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-06T20:11:46.921 INFO:journalctl@ceph.mon.b.smithi098.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-06T20:11:46.921 INFO:journalctl@ceph.mon.b.smithi098.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-06T20:11:46.921 INFO:journalctl@ceph.mon.b.smithi098.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-06T20:11:46.921 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-07-06T20:11:46.921 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** Compaction Stats [default] ** 2023-07-06T20:11:46.921 INFO:journalctl@ceph.mon.b.smithi098.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-06T20:11:46.922 INFO:journalctl@ceph.mon.b.smithi098.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-06T20:11:46.922 INFO:journalctl@ceph.mon.b.smithi098.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-06T20:11:46.922 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-07-06T20:11:46.922 INFO:journalctl@ceph.mon.b.smithi098.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-06T20:11:46.922 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-07-06T20:11:46.922 INFO:journalctl@ceph.mon.b.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-06T20:11:46.922 INFO:journalctl@ceph.mon.b.smithi098.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-06T20:11:46.923 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-06T20:11:46.923 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-06T20:11:46.923 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-06T20:11:46.923 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-06T20:11:46.923 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-06T20:11:46.923 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-06T20:11:46.923 INFO:journalctl@ceph.mon.b.smithi098.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-06T20:11:46.924 INFO:journalctl@ceph.mon.b.smithi098.stdout: Block cache BinnedLRUCache@0x564066d571f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-07-06T20:11:46.924 INFO:journalctl@ceph.mon.b.smithi098.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-06T20:11:46.924 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-07-06T20:11:46.924 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-06T20:11:46.924 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-06T20:11:46.924 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: using public_addr v2:172.21.15.98:0/0 -> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] 2023-07-06T20:11:46.924 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: starting mon.b rank -1 at public addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] at bind addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:11:46.925 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 systemd[1]: Started Ceph mon.b for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:11:46.925 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(???) e0 preinit fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:11:46.925 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).mds e1 new map 2023-07-06T20:11:46.925 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-06T20:11:46.925 INFO:journalctl@ceph.mon.b.smithi098.stdout: e1 2023-07-06T20:11:46.925 INFO:journalctl@ceph.mon.b.smithi098.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-06T20:11:46.925 INFO:journalctl@ceph.mon.b.smithi098.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-06T20:11:46.926 INFO:journalctl@ceph.mon.b.smithi098.stdout: legacy client fscid: -1 2023-07-06T20:11:46.926 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-07-06T20:11:46.926 INFO:journalctl@ceph.mon.b.smithi098.stdout: No filesystems configured 2023-07-06T20:11:46.926 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-06T20:11:46.926 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-06T20:11:46.926 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-06T20:11:46.926 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-06T20:11:46.926 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-06T20:11:46.927 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-06T20:11:46.927 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-06T20:11:46.927 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-06T20:11:46.927 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-06T20:11:46.927 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-06T20:11:46.927 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Updating smithi098:/etc/ceph/ceph.conf 2023-07-06T20:11:46.927 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Updating smithi098:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:46.928 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:46.928 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.928 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.928 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.928 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Updating smithi098:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.client.admin.keyring 2023-07-06T20:11:46.928 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:46.928 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.929 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Deploying cephadm binary to smithi156 2023-07-06T20:11:46.929 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.929 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.929 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Added host smithi156 2023-07-06T20:11:46.929 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:11:46.929 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.930 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.930 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.930 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.930 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:11:46.930 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.930 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.930 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.931 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.931 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.931 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:11:46.931 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:46.931 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:11:46.931 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-06T20:11:46.931 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Updating smithi156:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:46.931 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:46.932 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2293341203' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-06T20:11:46.932 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.932 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.932 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.932 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Updating smithi156:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.client.admin.keyring 2023-07-06T20:11:46.932 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2293341203' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-06T20:11:46.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: osdmap e4: 0 total, 0 up, 0 in 2023-07-06T20:11:46.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:11:46.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:46.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:11:46.933 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.934 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:46.934 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:46.934 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi098:172.21.15.98=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:46.934 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi098:172.21.15.98=b;smithi156:172.21.15.156=c;count:3 2023-07-06T20:11:46.934 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Deploying daemon mon.c on smithi156 2023-07-06T20:11:46.934 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.934 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.935 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='client.? 172.21.15.156:0/1411805241' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-06T20:11:46.935 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:11:46.935 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.a calling monitor election 2023-07-06T20:11:46.935 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:46.935 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.935 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:46.935 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:46.935 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.c calling monitor election 2023-07-06T20:11:46.936 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.936 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:46.936 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:46.936 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:46.936 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-06T20:11:46.936 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: monmap e2: 2 mons at {a=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-06T20:11:46.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: fsmap 2023-07-06T20:11:46.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: osdmap e4: 0 total, 0 up, 0 in 2023-07-06T20:11:46.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mgrmap e13: x(active, since 46s) 2023-07-06T20:11:46.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: overall HEALTH_OK 2023-07-06T20:11:46.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:46.937 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:46.938 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:46.938 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.938 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: Deploying daemon mon.b on smithi098 2023-07-06T20:11:46.938 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='client.? 172.21.15.156:0/1685627934' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-06T20:11:46.938 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:46.938 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:46.938 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-06T20:11:46.939 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: expand_channel_meta expand map: {default=false} 2023-07-06T20:11:46.939 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: expand_channel_meta from 'false' to 'false' 2023-07-06T20:11:46.939 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: expand_channel_meta expanded map: {default=false} 2023-07-06T20:11:46.939 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: expand_channel_meta expand map: {default=info} 2023-07-06T20:11:46.939 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: expand_channel_meta from 'info' to 'info' 2023-07-06T20:11:46.939 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: expand_channel_meta expanded map: {default=info} 2023-07-06T20:11:46.939 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: expand_channel_meta expand map: {default=daemon} 2023-07-06T20:11:46.939 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-06T20:11:46.940 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: expand_channel_meta expanded map: {default=daemon} 2023-07-06T20:11:46.940 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: expand_channel_meta expand map: {default=debug} 2023-07-06T20:11:46.940 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: expand_channel_meta from 'debug' to 'debug' 2023-07-06T20:11:46.940 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: expand_channel_meta expanded map: {default=debug} 2023-07-06T20:11:46.941 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:46 smithi098 ceph-mon[128968]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-06T20:11:47.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:46 smithi069 ceph-mon[126399]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:51.980 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.978+0000 7f48d9b6b700 1 -- 172.21.15.156:0/1422464999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48bc0035c0 msgr2=0x7f48bc005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:51.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.978+0000 7f48d9b6b700 1 --2- 172.21.15.156:0/1422464999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48bc0035c0 0x7f48bc005a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48d4076800 tx=0x7f48c800fe60 comp rx=0 tx=0).stop 2023-07-06T20:11:51.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.978+0000 7f48d9b6b700 1 -- 172.21.15.156:0/1422464999 shutdown_connections 2023-07-06T20:11:51.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.978+0000 7f48d9b6b700 1 --2- 172.21.15.156:0/1422464999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48bc0035c0 0x7f48bc005a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:51.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.978+0000 7f48d9b6b700 1 -- 172.21.15.156:0/1422464999 >> 172.21.15.156:0/1422464999 conn(0x7f48d4073ea0 msgr2=0x7f48d40742a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:51.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.978+0000 7f48d9b6b700 1 -- 172.21.15.156:0/1422464999 shutdown_connections 2023-07-06T20:11:51.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.978+0000 7f48d9b6b700 1 -- 172.21.15.156:0/1422464999 wait complete. 2023-07-06T20:11:51.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.979+0000 7f48d9b6b700 1 Processor -- start 2023-07-06T20:11:51.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.979+0000 7f48d9b6b700 1 -- start start 2023-07-06T20:11:51.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d9b6b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48bc0035c0 0x7f48d4195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:51.983 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d9b6b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d41961c0 0x7f48d4196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:51.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d9b6b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48d419ecf0 0x7f48d41a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:51.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d9b6b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d41a1810 con 0x7f48bc0035c0 2023-07-06T20:11:51.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d9b6b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d41a1980 con 0x7f48d419ecf0 2023-07-06T20:11:51.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d9b6b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d41a1b40 con 0x7f48d41961c0 2023-07-06T20:11:51.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d41961c0 0x7f48d4196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:51.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48bc0035c0 0x7f48d4195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:51.984 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d41961c0 0x7f48d4196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59668/0 (socket says 172.21.15.156:59668) 2023-07-06T20:11:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d2ffd700 1 -- 172.21.15.156:0/696857159 learned_addr learned my addr 172.21.15.156:0/696857159 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-06T20:11:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d3fff700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48d419ecf0 0x7f48d41a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d2ffd700 1 -- 172.21.15.156:0/696857159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d41961c0 msgr2=0x7f48d4196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-06T20:11:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d2ffd700 1 -- 172.21.15.156:0/696857159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d41961c0 msgr2=0x7f48d4196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-06T20:11:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d2ffd700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d41961c0 0x7f48d4196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-06T20:11:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d2ffd700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d41961c0 0x7f48d4196630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T20:11:51.985 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d3fff700 1 -- 172.21.15.156:0/696857159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48d419ecf0 msgr2=0x7f48d41a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-07-06T20:11:51.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d3fff700 1 -- 172.21.15.156:0/696857159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48d419ecf0 msgr2=0x7f48d41a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-06T20:11:51.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d3fff700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48d419ecf0 0x7f48d41a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-06T20:11:51.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d3fff700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48d419ecf0 0x7f48d41a10d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T20:11:51.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d37fe700 1 -- 172.21.15.156:0/696857159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d41961c0 msgr2=0x7f48d4196630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T20:11:51.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d37fe700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d41961c0 0x7f48d4196630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:51.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d37fe700 1 -- 172.21.15.156:0/696857159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48d419ecf0 msgr2=0x7f48d41a10d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T20:11:51.986 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d37fe700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48d419ecf0 0x7f48d41a10d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:51.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.980+0000 7f48d37fe700 1 -- 172.21.15.156:0/696857159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c800f8b0 con 0x7f48bc0035c0 2023-07-06T20:11:51.987 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.981+0000 7f48d37fe700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48bc0035c0 0x7f48d4195c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f48c8015040 tx=0x7f48c8012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.981+0000 7f48d0ff9700 1 -- 172.21.15.156:0/696857159 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8004130 con 0x7f48bc0035c0 2023-07-06T20:11:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.981+0000 7f48d0ff9700 1 -- 172.21.15.156:0/696857159 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48c8004290 con 0x7f48bc0035c0 2023-07-06T20:11:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.981+0000 7f48d9b6b700 1 -- 172.21.15.156:0/696857159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d41a1dc0 con 0x7f48bc0035c0 2023-07-06T20:11:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.981+0000 7f48d0ff9700 1 -- 172.21.15.156:0/696857159 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c80278b0 con 0x7f48bc0035c0 2023-07-06T20:11:51.988 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.981+0000 7f48d9b6b700 1 -- 172.21.15.156:0/696857159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d41a2260 con 0x7f48bc0035c0 2023-07-06T20:11:51.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.982+0000 7f48d0ff9700 1 -- 172.21.15.156:0/696857159 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f48c8024050 con 0x7f48bc0035c0 2023-07-06T20:11:51.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.982+0000 7f48d9b6b700 1 -- 172.21.15.156:0/696857159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d4061c70 con 0x7f48bc0035c0 2023-07-06T20:11:51.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.982+0000 7f48d0ff9700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f48c0038c80 0x7f48c003b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:51.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.982+0000 7f48d0ff9700 1 -- 172.21.15.156:0/696857159 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f48c8000e40 con 0x7f48bc0035c0 2023-07-06T20:11:51.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.983+0000 7f48d2ffd700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f48c0038c80 0x7f48c003b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:51.989 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.983+0000 7f48d2ffd700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f48c0038c80 0x7f48c003b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f48d40ff9d0 tx=0x7f48c4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:51.990 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:51.987+0000 7f48d0ff9700 1 -- 172.21.15.156:0/696857159 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f48c8026730 con 0x7f48bc0035c0 2023-07-06T20:11:52.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: mon.c calling monitor election 2023-07-06T20:11:52.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:11:52.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: mon.a calling monitor election 2023-07-06T20:11:52.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:52.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:52.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:52.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:52.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:52.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:52.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:52.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:52.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:52.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-06T20:11:52.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: monmap e3: 3 mons at {a=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-06T20:11:52.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: fsmap 2023-07-06T20:11:52.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: osdmap e4: 0 total, 0 up, 0 in 2023-07-06T20:11:52.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: mgrmap e13: x(active, since 54s) 2023-07-06T20:11:52.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-06T20:11:52.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-06T20:11:52.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-06T20:11:52.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: mon.b (rank 2) addr [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] is down (out of quorum) 2023-07-06T20:11:52.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:52.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:52.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:52.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:52.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: mon.c calling monitor election 2023-07-06T20:11:52.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:11:52.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: mon.a calling monitor election 2023-07-06T20:11:52.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:52.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:52.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:52.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:52.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:52.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:52.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:52.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:52.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:52.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-06T20:11:52.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: monmap e3: 3 mons at {a=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-06T20:11:52.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: fsmap 2023-07-06T20:11:52.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: osdmap e4: 0 total, 0 up, 0 in 2023-07-06T20:11:52.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: mgrmap e13: x(active, since 54s) 2023-07-06T20:11:52.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-06T20:11:52.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-06T20:11:52.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-06T20:11:52.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: mon.b (rank 2) addr [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] is down (out of quorum) 2023-07-06T20:11:52.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:52.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:52.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:52.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:52.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.259+0000 7f48d9b6b700 1 -- 172.21.15.156:0/696857159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f48d40feee0 con 0x7f48bc0035c0 2023-07-06T20:11:52.262 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.260+0000 7f48d0ff9700 1 -- 172.21.15.156:0/696857159 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7f48c8022070 con 0x7f48bc0035c0 2023-07-06T20:11:52.262 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-06T20:11:52.263 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":3,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","modified":"2023-07-06T20:11:46.633432Z","created":"2023-07-06T20:09:53.314871Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-06T20:11:52.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.262+0000 7f48d9b6b700 1 -- 172.21.15.156:0/696857159 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f48c0038c80 msgr2=0x7f48c003b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:52.264 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.262+0000 7f48d9b6b700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f48c0038c80 0x7f48c003b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f48d40ff9d0 tx=0x7f48c4008040 comp rx=0 tx=0).stop 2023-07-06T20:11:52.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.262+0000 7f48d9b6b700 1 -- 172.21.15.156:0/696857159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48bc0035c0 msgr2=0x7f48d4195c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:52.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.262+0000 7f48d9b6b700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48bc0035c0 0x7f48d4195c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f48c8015040 tx=0x7f48c8012d70 comp rx=0 tx=0).stop 2023-07-06T20:11:52.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.262+0000 7f48d9b6b700 1 -- 172.21.15.156:0/696857159 shutdown_connections 2023-07-06T20:11:52.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.262+0000 7f48d9b6b700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f48c0038c80 0x7f48c003b130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:52.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.262+0000 7f48d9b6b700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48d419ecf0 0x7f48d41a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:52.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.262+0000 7f48d9b6b700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48bc0035c0 0x7f48d4195c80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:52.265 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.262+0000 7f48d9b6b700 1 --2- 172.21.15.156:0/696857159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d41961c0 0x7f48d4196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.262+0000 7f48d9b6b700 1 -- 172.21.15.156:0/696857159 >> 172.21.15.156:0/696857159 conn(0x7f48d4073ea0 msgr2=0x7f48d4100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.262+0000 7f48d9b6b700 1 -- 172.21.15.156:0/696857159 shutdown_connections 2023-07-06T20:11:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:52.262+0000 7f48d9b6b700 1 -- 172.21.15.156:0/696857159 wait complete. 2023-07-06T20:11:52.266 INFO:teuthology.orchestra.run.smithi156.stderr:dumped monmap epoch 3 2023-07-06T20:11:52.632 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-06T20:11:52.633 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph config generate-minimal-conf 2023-07-06T20:11:52.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:52 smithi156 ceph-mon[129280]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:52.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:11:52.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:52 smithi156 ceph-mon[129280]: from='client.? 172.21.15.156:0/696857159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-06T20:11:52.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:52.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:53.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:52 smithi069 ceph-mon[126399]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:11:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:52 smithi069 ceph-mon[126399]: from='client.? 172.21.15.156:0/696857159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-06T20:11:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:53.357 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:11:54.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: mon.c calling monitor election 2023-07-06T20:11:54.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:11:54.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: mon.a calling monitor election 2023-07-06T20:11:54.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:54.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:11:54.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:54.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:54.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:54.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:54.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:54.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:54.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:54.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-06T20:11:54.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: monmap e3: 3 mons at {a=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-06T20:11:54.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: fsmap 2023-07-06T20:11:54.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: osdmap e4: 0 total, 0 up, 0 in 2023-07-06T20:11:54.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: mgrmap e13: x(active, since 54s) 2023-07-06T20:11:54.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-06T20:11:54.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-06T20:11:54.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-06T20:11:54.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: mon.b (rank 2) addr [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] is down (out of quorum) 2023-07-06T20:11:54.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:54.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:11:54.130 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='client.? 172.21.15.156:0/696857159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-06T20:11:54.130 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.130 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:54.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: mon.b calling monitor election 2023-07-06T20:11:54.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: Updating smithi069:/etc/ceph/ceph.conf 2023-07-06T20:11:54.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: Updating smithi098:/etc/ceph/ceph.conf 2023-07-06T20:11:54.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-06T20:11:54.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:54.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: mon.b calling monitor election 2023-07-06T20:11:54.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: mon.c calling monitor election 2023-07-06T20:11:54.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: mon.a calling monitor election 2023-07-06T20:11:54.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-06T20:11:54.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: monmap e3: 3 mons at {a=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-06T20:11:54.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: fsmap 2023-07-06T20:11:54.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: osdmap e4: 0 total, 0 up, 0 in 2023-07-06T20:11:54.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: mgrmap e13: x(active, since 56s) 2023-07-06T20:11:54.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-06T20:11:54.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: Cluster is now healthy 2023-07-06T20:11:54.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: overall HEALTH_OK 2023-07-06T20:11:54.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: Updating smithi098:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:54.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: Updating smithi156:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:54.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:54 smithi069 ceph-mon[126399]: Updating smithi069:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:54.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.847+0000 7f0106016700 1 -- 172.21.15.69:0/2330952594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0100105720 msgr2=0x7f0100105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:54.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.847+0000 7f0106016700 1 --2- 172.21.15.69:0/2330952594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0100105720 0x7f0100105af0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009910 tx=0x7f00f0009c20 comp rx=0 tx=0).stop 2023-07-06T20:11:54.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.847+0000 7f0106016700 1 -- 172.21.15.69:0/2330952594 shutdown_connections 2023-07-06T20:11:54.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.847+0000 7f0106016700 1 --2- 172.21.15.69:0/2330952594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0100105720 0x7f0100105af0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:54.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.847+0000 7f0106016700 1 -- 172.21.15.69:0/2330952594 >> 172.21.15.69:0/2330952594 conn(0x7f01000fb260 msgr2=0x7f01000fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:54.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.847+0000 7f0106016700 1 -- 172.21.15.69:0/2330952594 shutdown_connections 2023-07-06T20:11:54.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.847+0000 7f0106016700 1 -- 172.21.15.69:0/2330952594 wait complete. 2023-07-06T20:11:54.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.848+0000 7f0106016700 1 Processor -- start 2023-07-06T20:11:54.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.849+0000 7f0106016700 1 -- start start 2023-07-06T20:11:54.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.849+0000 7f0106016700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0100105720 0x7f0100194da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:54.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.849+0000 7f0106016700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01001952e0 0x7f0100199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:54.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.849+0000 7f0106016700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0100199c90 0x7f010019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:54.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.849+0000 7f0106016700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001959b0 con 0x7f0100199c90 2023-07-06T20:11:54.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.849+0000 7f0106016700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001a1a00 con 0x7f01001952e0 2023-07-06T20:11:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.849+0000 7f0106016700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001a1b40 con 0x7f0100105720 2023-07-06T20:11:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.850+0000 7f00fffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0100199c90 0x7f010019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.850+0000 7f00fffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0100199c90 0x7f010019a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55202/0 (socket says 172.21.15.69:55202) 2023-07-06T20:11:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.850+0000 7f00fffff700 1 -- 172.21.15.69:0/3171057339 learned_addr learned my addr 172.21.15.69:0/3171057339 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:11:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.850+0000 7f00ff7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0100105720 0x7f0100194da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.850+0000 7f00feffd700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01001952e0 0x7f0100199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.850+0000 7f00fffff700 1 -- 172.21.15.69:0/3171057339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0100105720 msgr2=0x7f0100194da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.850+0000 7f00fffff700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0100105720 0x7f0100194da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.850+0000 7f00fffff700 1 -- 172.21.15.69:0/3171057339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01001952e0 msgr2=0x7f0100199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.850+0000 7f00fffff700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01001952e0 0x7f0100199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.850+0000 7f00fffff700 1 -- 172.21.15.69:0/3171057339 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f00095f0 con 0x7f0100199c90 2023-07-06T20:11:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.850+0000 7f00ff7fe700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0100105720 0x7f0100194da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:11:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.851+0000 7f00fffff700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0100199c90 0x7f010019a130 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f00f800ba80 tx=0x7f00f800bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.851+0000 7f00fcff9700 1 -- 172.21.15.69:0/3171057339 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800c930 con 0x7f0100199c90 2023-07-06T20:11:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.851+0000 7f00fcff9700 1 -- 172.21.15.69:0/3171057339 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f8014440 con 0x7f0100199c90 2023-07-06T20:11:54.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.851+0000 7f0106016700 1 -- 172.21.15.69:0/3171057339 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01001a1df0 con 0x7f0100199c90 2023-07-06T20:11:54.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.851+0000 7f00fcff9700 1 -- 172.21.15.69:0/3171057339 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8007570 con 0x7f0100199c90 2023-07-06T20:11:54.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.851+0000 7f0106016700 1 -- 172.21.15.69:0/3171057339 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01001a2210 con 0x7f0100199c90 2023-07-06T20:11:54.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.852+0000 7f00fcff9700 1 -- 172.21.15.69:0/3171057339 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f00f80076d0 con 0x7f0100199c90 2023-07-06T20:11:54.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: mon.b calling monitor election 2023-07-06T20:11:54.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: Updating smithi069:/etc/ceph/ceph.conf 2023-07-06T20:11:54.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: Updating smithi098:/etc/ceph/ceph.conf 2023-07-06T20:11:54.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-06T20:11:54.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:54.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: mon.b calling monitor election 2023-07-06T20:11:54.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: mon.c calling monitor election 2023-07-06T20:11:54.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: mon.a calling monitor election 2023-07-06T20:11:54.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-06T20:11:54.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: monmap e3: 3 mons at {a=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-06T20:11:54.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: fsmap 2023-07-06T20:11:54.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: osdmap e4: 0 total, 0 up, 0 in 2023-07-06T20:11:54.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: mgrmap e13: x(active, since 56s) 2023-07-06T20:11:54.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-06T20:11:54.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: Cluster is now healthy 2023-07-06T20:11:54.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: overall HEALTH_OK 2023-07-06T20:11:54.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: Updating smithi098:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:54.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: Updating smithi156:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:54.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:54 smithi156 ceph-mon[129280]: Updating smithi069:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:54.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.852+0000 7f0106016700 1 -- 172.21.15.69:0/3171057339 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0100061c70 con 0x7f0100199c90 2023-07-06T20:11:54.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.852+0000 7f00fcff9700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f00e8038c80 0x7f00e803b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:54.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.852+0000 7f00fcff9700 1 -- 172.21.15.69:0/3171057339 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f00f804a930 con 0x7f0100199c90 2023-07-06T20:11:54.861 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.853+0000 7f00ff7fe700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f00e8038c80 0x7f00e803b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:54.861 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.853+0000 7f00ff7fe700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f00e8038c80 0x7f00e803b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f00f000b5c0 tx=0x7f00f00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:54.861 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:54.858+0000 7f00fcff9700 1 -- 172.21.15.69:0/3171057339 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f00f801b8b0 con 0x7f0100199c90 2023-07-06T20:11:54.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: mon.b calling monitor election 2023-07-06T20:11:54.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: Updating smithi069:/etc/ceph/ceph.conf 2023-07-06T20:11:54.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: Updating smithi098:/etc/ceph/ceph.conf 2023-07-06T20:11:54.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-06T20:11:54.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:54.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: mon.b calling monitor election 2023-07-06T20:11:54.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: mon.c calling monitor election 2023-07-06T20:11:54.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: mon.a calling monitor election 2023-07-06T20:11:54.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-06T20:11:54.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: monmap e3: 3 mons at {a=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-06T20:11:54.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: fsmap 2023-07-06T20:11:54.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: osdmap e4: 0 total, 0 up, 0 in 2023-07-06T20:11:54.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: mgrmap e13: x(active, since 56s) 2023-07-06T20:11:54.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-06T20:11:54.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: Cluster is now healthy 2023-07-06T20:11:54.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: overall HEALTH_OK 2023-07-06T20:11:54.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: Updating smithi098:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:54.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: Updating smithi156:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:54.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:54.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:54 smithi098 ceph-mon[128968]: Updating smithi069:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/config/ceph.conf 2023-07-06T20:11:55.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.060+0000 7f0106016700 1 -- 172.21.15.69:0/3171057339 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f0100196100 con 0x7f0100199c90 2023-07-06T20:11:55.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.060+0000 7f00fcff9700 1 -- 172.21.15.69:0/3171057339 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f00f801b2c0 con 0x7f0100199c90 2023-07-06T20:11:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:# minimal ceph.conf for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:11:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:[global] 2023-07-06T20:11:55.063 INFO:teuthology.orchestra.run.smithi069.stdout: fsid = e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:11:55.063 INFO:teuthology.orchestra.run.smithi069.stdout: mon_host = [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] 2023-07-06T20:11:55.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.063+0000 7f0106016700 1 -- 172.21.15.69:0/3171057339 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f00e8038c80 msgr2=0x7f00e803b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:55.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.063+0000 7f0106016700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f00e8038c80 0x7f00e803b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f00f000b5c0 tx=0x7f00f00058e0 comp rx=0 tx=0).stop 2023-07-06T20:11:55.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.063+0000 7f0106016700 1 -- 172.21.15.69:0/3171057339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0100199c90 msgr2=0x7f010019a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:55.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.063+0000 7f0106016700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0100199c90 0x7f010019a130 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f00f800ba80 tx=0x7f00f800bd90 comp rx=0 tx=0).stop 2023-07-06T20:11:55.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.063+0000 7f0106016700 1 -- 172.21.15.69:0/3171057339 shutdown_connections 2023-07-06T20:11:55.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.063+0000 7f0106016700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f00e8038c80 0x7f00e803b130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:55.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.063+0000 7f0106016700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0100199c90 0x7f010019a130 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:55.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.063+0000 7f0106016700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0100105720 0x7f0100194da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:55.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.063+0000 7f0106016700 1 --2- 172.21.15.69:0/3171057339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01001952e0 0x7f0100199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:55.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.063+0000 7f0106016700 1 -- 172.21.15.69:0/3171057339 >> 172.21.15.69:0/3171057339 conn(0x7f01000fb260 msgr2=0x7f01000fb630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:55.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.063+0000 7f0106016700 1 -- 172.21.15.69:0/3171057339 shutdown_connections 2023-07-06T20:11:55.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:11:55.063+0000 7f0106016700 1 -- 172.21.15.69:0/3171057339 wait complete. 2023-07-06T20:11:55.596 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-06T20:11:55.597 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:11:55.597 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.conf 2023-07-06T20:11:55.679 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:11:55.680 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:55.752 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:55 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:55.752 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:55 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:55.753 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:55 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:55.753 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:55 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:55.753 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:55 smithi069 ceph-mon[126399]: Reconfiguring mon.a (unknown last config time)... 2023-07-06T20:11:55.753 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:55 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:55.753 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:55 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-06T20:11:55.753 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:55 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:55.753 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:55 smithi069 ceph-mon[126399]: Reconfiguring daemon mon.a on smithi069 2023-07-06T20:11:55.754 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:55 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3171057339' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:55.789 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-07-06T20:11:55.789 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.conf 2023-07-06T20:11:55.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:55 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:55.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:55 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:55.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:55 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:55.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:55 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:55.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:55 smithi098 ceph-mon[128968]: Reconfiguring mon.a (unknown last config time)... 2023-07-06T20:11:55.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:55 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:55.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:55 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-06T20:11:55.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:55 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:55.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:55 smithi098 ceph-mon[128968]: Reconfiguring daemon mon.a on smithi069 2023-07-06T20:11:55.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:55 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3171057339' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:55.824 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-07-06T20:11:55.825 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:55.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:55 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:55.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:55 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:55.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:55 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:55.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:55 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:11:55.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:55 smithi156 ceph-mon[129280]: Reconfiguring mon.a (unknown last config time)... 2023-07-06T20:11:55.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:55 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:55.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:55 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-06T20:11:55.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:55 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:55.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:55 smithi156 ceph-mon[129280]: Reconfiguring daemon mon.a on smithi069 2023-07-06T20:11:55.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:55 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3171057339' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:55.894 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-06T20:11:55.894 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.conf 2023-07-06T20:11:55.930 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-06T20:11:55.931 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-06T20:11:56.000 INFO:tasks.cephadm:Adding mgr.x on smithi069 2023-07-06T20:11:56.001 INFO:tasks.cephadm:Adding mgr.y on smithi098 2023-07-06T20:11:56.001 INFO:tasks.cephadm:Adding mgr.z on smithi156 2023-07-06T20:11:56.001 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch apply mgr '3;smithi069=x;smithi098=y;smithi156=z' 2023-07-06T20:11:56.589 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:11:56.768 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:56 smithi156 ceph-mon[129280]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:56.768 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:56.768 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:56.768 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:56.768 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-06T20:11:56.768 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:56.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:56 smithi098 ceph-mon[128968]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:56.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:56.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:56.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:56.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-06T20:11:56.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:57.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:56 smithi069 ceph-mon[126399]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-06T20:11:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:57.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.828+0000 7ff61d037700 1 -- 172.21.15.156:0/2883672647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180fe730 msgr2=0x7ff6180feb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:57.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.828+0000 7ff61d037700 1 --2- 172.21.15.156:0/2883672647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180fe730 0x7ff6180feb40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff608009910 tx=0x7ff608009c20 comp rx=0 tx=0).stop 2023-07-06T20:11:57.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.828+0000 7ff61d037700 1 -- 172.21.15.156:0/2883672647 shutdown_connections 2023-07-06T20:11:57.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.828+0000 7ff61d037700 1 --2- 172.21.15.156:0/2883672647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180fe730 0x7ff6180feb40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:57.830 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.828+0000 7ff61d037700 1 -- 172.21.15.156:0/2883672647 >> 172.21.15.156:0/2883672647 conn(0x7ff6180f9ce0 msgr2=0x7ff6180fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:57.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.828+0000 7ff61d037700 1 -- 172.21.15.156:0/2883672647 shutdown_connections 2023-07-06T20:11:57.831 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.828+0000 7ff61d037700 1 -- 172.21.15.156:0/2883672647 wait complete. 2023-07-06T20:11:57.832 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.829+0000 7ff61d037700 1 Processor -- start 2023-07-06T20:11:57.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.829+0000 7ff61d037700 1 -- start start 2023-07-06T20:11:57.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.829+0000 7ff61d037700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180fe730 0x7ff618068900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:57.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.829+0000 7ff61d037700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff618199c40 0x7ff61819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:57.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.829+0000 7ff61d037700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff61819c560 0x7ff61819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:57.833 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.829+0000 7ff61d037700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61819eff0 con 0x7ff6180fe730 2023-07-06T20:11:57.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.829+0000 7ff61d037700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61819f160 con 0x7ff61819c560 2023-07-06T20:11:57.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.829+0000 7ff61d037700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61819f320 con 0x7ff618199c40 2023-07-06T20:11:57.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.829+0000 7ff61659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff618199c40 0x7ff61819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:57.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.829+0000 7ff61659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff618199c40 0x7ff61819c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59678/0 (socket says 172.21.15.156:59678) 2023-07-06T20:11:57.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.829+0000 7ff61659c700 1 -- 172.21.15.156:0/2458709302 learned_addr learned my addr 172.21.15.156:0/2458709302 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-06T20:11:57.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.830+0000 7ff61759e700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff61819c560 0x7ff61819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:57.834 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.830+0000 7ff61659c700 1 -- 172.21.15.156:0/2458709302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff61819c560 msgr2=0x7ff61819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:57.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.830+0000 7ff61659c700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff61819c560 0x7ff61819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:57.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.830+0000 7ff61659c700 1 -- 172.21.15.156:0/2458709302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180fe730 msgr2=0x7ff618068900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:57.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.830+0000 7ff61659c700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180fe730 0x7ff618068900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:57.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.830+0000 7ff61659c700 1 -- 172.21.15.156:0/2458709302 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6080095f0 con 0x7ff618199c40 2023-07-06T20:11:57.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.830+0000 7ff61659c700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff618199c40 0x7ff61819c020 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00b780 tx=0x7ff60c00bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:57.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.830+0000 7ff5fffff700 1 -- 172.21.15.156:0/2458709302 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c004230 con 0x7ff618199c40 2023-07-06T20:11:57.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.830+0000 7ff61d037700 1 -- 172.21.15.156:0/2458709302 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61819f600 con 0x7ff618199c40 2023-07-06T20:11:57.835 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.830+0000 7ff5fffff700 1 -- 172.21.15.156:0/2458709302 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff60c004390 con 0x7ff618199c40 2023-07-06T20:11:57.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.830+0000 7ff5fffff700 1 -- 172.21.15.156:0/2458709302 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c011b30 con 0x7ff618199c40 2023-07-06T20:11:57.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.830+0000 7ff61d037700 1 -- 172.21.15.156:0/2458709302 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61819fb80 con 0x7ff618199c40 2023-07-06T20:11:57.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.831+0000 7ff5fffff700 1 -- 172.21.15.156:0/2458709302 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7ff60c011cd0 con 0x7ff618199c40 2023-07-06T20:11:57.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.831+0000 7ff61d037700 1 -- 172.21.15.156:0/2458709302 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61818d300 con 0x7ff618199c40 2023-07-06T20:11:57.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.831+0000 7ff5fffff700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff600038c80 0x7ff60003b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:11:57.836 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.831+0000 7ff5fffff700 1 -- 172.21.15.156:0/2458709302 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff60c04ad80 con 0x7ff618199c40 2023-07-06T20:11:57.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.831+0000 7ff616d9d700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff600038c80 0x7ff60003b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:11:57.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.832+0000 7ff616d9d700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff600038c80 0x7ff60003b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff608004690 tx=0x7ff608005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:11:57.837 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:57.836+0000 7ff5fffff700 1 -- 172.21.15.156:0/2458709302 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff60c00f6a0 con 0x7ff618199c40 2023-07-06T20:11:57.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:57 smithi156 ceph-mon[129280]: Reconfiguring mon.b (monmap changed)... 2023-07-06T20:11:57.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:57 smithi156 ceph-mon[129280]: Reconfiguring daemon mon.b on smithi098 2023-07-06T20:11:57.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:57 smithi098 ceph-mon[128968]: Reconfiguring mon.b (monmap changed)... 2023-07-06T20:11:57.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:57 smithi098 ceph-mon[128968]: Reconfiguring daemon mon.b on smithi098 2023-07-06T20:11:58.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:57 smithi069 ceph-mon[126399]: Reconfiguring mon.b (monmap changed)... 2023-07-06T20:11:58.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:57 smithi069 ceph-mon[126399]: Reconfiguring daemon mon.b on smithi098 2023-07-06T20:11:58.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.041+0000 7ff61d037700 1 -- 172.21.15.156:0/2458709302 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi098=y;smithi156=z", "target": ["mon-mgr", ""]}) v1 -- 0x7ff618060a80 con 0x7ff600038c80 2023-07-06T20:11:58.051 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.049+0000 7ff5fffff700 1 -- 172.21.15.156:0/2458709302 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff618060a80 con 0x7ff600038c80 2023-07-06T20:11:58.051 INFO:teuthology.orchestra.run.smithi156.stdout:Scheduled mgr update... 2023-07-06T20:11:58.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.052+0000 7ff61d037700 1 -- 172.21.15.156:0/2458709302 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff600038c80 msgr2=0x7ff60003b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:58.053 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.052+0000 7ff61d037700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff600038c80 0x7ff60003b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff608004690 tx=0x7ff608005fb0 comp rx=0 tx=0).stop 2023-07-06T20:11:58.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.052+0000 7ff61d037700 1 -- 172.21.15.156:0/2458709302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff618199c40 msgr2=0x7ff61819c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:11:58.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.052+0000 7ff61d037700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff618199c40 0x7ff61819c020 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00b780 tx=0x7ff60c00bb40 comp rx=0 tx=0).stop 2023-07-06T20:11:58.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.052+0000 7ff61d037700 1 -- 172.21.15.156:0/2458709302 shutdown_connections 2023-07-06T20:11:58.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.052+0000 7ff61d037700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff600038c80 0x7ff60003b130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:58.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.052+0000 7ff61d037700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff61819c560 0x7ff61819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:58.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.052+0000 7ff61d037700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180fe730 0x7ff618068900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:58.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.052+0000 7ff61d037700 1 --2- 172.21.15.156:0/2458709302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff618199c40 0x7ff61819c020 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:11:58.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.052+0000 7ff61d037700 1 -- 172.21.15.156:0/2458709302 >> 172.21.15.156:0/2458709302 conn(0x7ff6180f9ce0 msgr2=0x7ff6180fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:11:58.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.052+0000 7ff61d037700 1 -- 172.21.15.156:0/2458709302 shutdown_connections 2023-07-06T20:11:58.055 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:11:58.052+0000 7ff61d037700 1 -- 172.21.15.156:0/2458709302 wait complete. 2023-07-06T20:11:58.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:58 smithi156 ceph-mon[129280]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:58.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:58.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:58.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:58 smithi156 ceph-mon[129280]: Reconfiguring mon.c (monmap changed)... 2023-07-06T20:11:58.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:58.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-06T20:11:58.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:58.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:58 smithi156 ceph-mon[129280]: Reconfiguring daemon mon.c on smithi156 2023-07-06T20:11:58.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:58 smithi156 ceph-mon[129280]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi098=y;smithi156=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:58.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:58 smithi156 ceph-mon[129280]: Saving service mgr spec with placement smithi069=x;smithi098=y;smithi156=z;count:3 2023-07-06T20:11:58.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:11:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:58.744 DEBUG:teuthology.orchestra.run.smithi098:mgr.y> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.y.service 2023-07-06T20:11:58.747 DEBUG:teuthology.orchestra.run.smithi156:mgr.z> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.z.service 2023-07-06T20:11:58.750 INFO:tasks.cephadm:Deploying OSDs... 2023-07-06T20:11:58.750 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:11:58.750 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T20:11:58.779 INFO:journalctl@ceph.mgr.z.smithi156.stdout:-- Logs begin at Thu 2023-07-06 19:42:21 UTC. -- 2023-07-06T20:11:58.782 INFO:journalctl@ceph.mgr.y.smithi098.stdout:-- Logs begin at Thu 2023-07-06 19:39:25 UTC. -- 2023-07-06T20:11:58.797 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T20:11:58.798 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2023-07-06T20:11:58.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:58 smithi098 ceph-mon[128968]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:58.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:58.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:58.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:58 smithi098 ceph-mon[128968]: Reconfiguring mon.c (monmap changed)... 2023-07-06T20:11:58.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:58.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-06T20:11:58.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:58.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:58 smithi098 ceph-mon[128968]: Reconfiguring daemon mon.c on smithi156 2023-07-06T20:11:58.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:58 smithi098 ceph-mon[128968]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi098=y;smithi156=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:58.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:58 smithi098 ceph-mon[128968]: Saving service mgr spec with placement smithi069=x;smithi098=y;smithi156=z;count:3 2023-07-06T20:11:58.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:11:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:58.885 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T20:11:58.885 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:11:58.885 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-07-06T20:11:58.886 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:11:58.886 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:11:58.886 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-07-06 20:11:08.898803361 +0000 2023-07-06T20:11:58.886 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-07-06 19:58:43.427889455 +0000 2023-07-06T20:11:58.886 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-07-06 19:58:43.427889455 +0000 2023-07-06T20:11:58.886 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2023-07-06 19:58:43.399890072 +0000 2023-07-06T20:11:58.887 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T20:11:58.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:58 smithi069 ceph-mon[126399]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:11:58.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:58.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:58.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:58 smithi069 ceph-mon[126399]: Reconfiguring mon.c (monmap changed)... 2023-07-06T20:11:58.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-06T20:11:58.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-06T20:11:58.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:11:58.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:58 smithi069 ceph-mon[126399]: Reconfiguring daemon mon.c on smithi156 2023-07-06T20:11:58.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:58 smithi069 ceph-mon[126399]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi098=y;smithi156=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:11:58.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:58 smithi069 ceph-mon[126399]: Saving service mgr spec with placement smithi069=x;smithi098=y;smithi156=z;count:3 2023-07-06T20:11:58.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:11:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:11:58.953 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-07-06T20:11:58.953 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-07-06T20:11:58.954 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000268617 s, 1.9 MB/s 2023-07-06T20:11:58.956 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T20:11:59.002 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2023-07-06T20:11:59.087 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T20:11:59.087 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:11:59.087 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-07-06T20:11:59.087 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:11:59.087 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:11:59.088 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-07-06 20:11:08.898803361 +0000 2023-07-06T20:11:59.088 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-07-06 19:58:44.337869400 +0000 2023-07-06T20:11:59.088 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-07-06 19:58:44.337869400 +0000 2023-07-06T20:11:59.088 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2023-07-06 19:58:44.311869973 +0000 2023-07-06T20:11:59.088 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T20:11:59.191 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-07-06T20:11:59.191 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-07-06T20:11:59.191 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.00032528 s, 1.6 MB/s 2023-07-06T20:11:59.193 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T20:11:59.238 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2023-07-06T20:11:59.321 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T20:11:59.321 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:11:59.321 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-07-06T20:11:59.321 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:11:59.321 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:11:59.321 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-07-06 20:11:08.898803361 +0000 2023-07-06T20:11:59.322 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-07-06 19:58:45.247849346 +0000 2023-07-06T20:11:59.322 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-07-06 19:58:45.247849346 +0000 2023-07-06T20:11:59.322 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2023-07-06 19:58:45.221849919 +0000 2023-07-06T20:11:59.322 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T20:11:59.424 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-07-06T20:11:59.424 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-07-06T20:11:59.425 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000299112 s, 1.7 MB/s 2023-07-06T20:11:59.427 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T20:11:59.471 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2023-07-06T20:11:59.554 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T20:11:59.554 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:11:59.554 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-07-06T20:11:59.555 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:11:59.555 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:11:59.555 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-07-06 20:11:08.898803361 +0000 2023-07-06T20:11:59.555 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-07-06 19:58:46.214828035 +0000 2023-07-06T20:11:59.555 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-07-06 19:58:46.214828035 +0000 2023-07-06T20:11:59.556 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2023-07-06 19:58:46.188828609 +0000 2023-07-06T20:11:59.556 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T20:11:59.659 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-07-06T20:11:59.659 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-07-06T20:11:59.659 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000316871 s, 1.6 MB/s 2023-07-06T20:11:59.662 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T20:11:59.706 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-07-06T20:11:59.707 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T20:11:59.727 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T20:11:59.728 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2023-07-06T20:11:59.787 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T20:11:59.788 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:11:59.788 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 378349 Links: 1 2023-07-06T20:11:59.788 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:11:59.788 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:11:59.788 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-07-06 20:11:46.080170597 +0000 2023-07-06T20:11:59.788 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-07-06 20:02:28.574487908 +0000 2023-07-06T20:11:59.789 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-07-06 20:02:28.574487908 +0000 2023-07-06T20:11:59.789 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-07-06T20:11:59.789 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T20:11:59.857 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-07-06T20:11:59.858 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-07-06T20:11:59.858 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000329621 s, 1.6 MB/s 2023-07-06T20:11:59.859 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T20:11:59.919 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2023-07-06T20:11:59.978 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T20:11:59.978 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:11:59.979 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 373544 Links: 1 2023-07-06T20:11:59.979 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:11:59.979 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:11:59.979 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-07-06 20:11:46.080170597 +0000 2023-07-06T20:11:59.979 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-07-06 20:02:29.125480840 +0000 2023-07-06T20:11:59.979 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-07-06 20:02:29.125480840 +0000 2023-07-06T20:11:59.980 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-07-06T20:11:59.980 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T20:12:00.045 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-07-06T20:12:00.046 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-07-06T20:12:00.046 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.00023815 s, 2.1 MB/s 2023-07-06T20:12:00.047 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T20:12:00.106 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2023-07-06T20:12:00.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:00.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:00.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:00.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:00.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:00.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:00.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-06T20:12:00.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-06T20:12:00.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-06T20:12:00.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:00.143 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T20:12:00.144 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:00.144 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 383209 Links: 1 2023-07-06T20:12:00.144 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:00.144 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:00.144 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-07-06 20:11:46.080170597 +0000 2023-07-06T20:12:00.144 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-07-06 20:02:29.651474093 +0000 2023-07-06T20:12:00.144 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-07-06 20:02:29.651474093 +0000 2023-07-06T20:12:00.145 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-07-06T20:12:00.145 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T20:12:00.209 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-07-06T20:12:00.210 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-07-06T20:12:00.210 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000244096 s, 2.1 MB/s 2023-07-06T20:12:00.211 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T20:12:00.241 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:00.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:00.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:00.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:00.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:00.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:00.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-06T20:12:00.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-06T20:12:00.243 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-06T20:12:00.243 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:00.269 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2023-07-06T20:12:00.326 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T20:12:00.327 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:00.327 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 376407 Links: 1 2023-07-06T20:12:00.327 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:00.327 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:00.327 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-07-06 20:11:46.080170597 +0000 2023-07-06T20:12:00.327 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-07-06 20:02:30.208466949 +0000 2023-07-06T20:12:00.327 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-07-06 20:02:30.208466949 +0000 2023-07-06T20:12:00.328 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-07-06T20:12:00.328 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T20:12:00.392 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-07-06T20:12:00.392 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-07-06T20:12:00.393 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000247864 s, 2.1 MB/s 2023-07-06T20:12:00.393 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T20:12:00.451 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-06T20:12:00.451 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T20:12:00.469 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T20:12:00.469 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2023-07-06T20:12:00.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:00.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:00.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:00.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:00.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:00.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:00.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-06T20:12:00.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-06T20:12:00.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-06T20:12:00.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:00.528 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T20:12:00.528 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:00.528 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 378130 Links: 1 2023-07-06T20:12:00.528 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:00.528 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:00.529 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-07-06 20:12:00.471551571 +0000 2023-07-06T20:12:00.529 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-07-06 19:55:05.887441290 +0000 2023-07-06T20:12:00.529 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-07-06 19:55:05.887441290 +0000 2023-07-06T20:12:00.529 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-07-06T20:12:00.529 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T20:12:00.599 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-07-06T20:12:00.599 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-07-06T20:12:00.599 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000125529 s, 4.1 MB/s 2023-07-06T20:12:00.600 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T20:12:00.658 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2023-07-06T20:12:00.717 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T20:12:00.717 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:00.717 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 377627 Links: 1 2023-07-06T20:12:00.717 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:00.717 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:00.717 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-07-06 20:12:00.471551571 +0000 2023-07-06T20:12:00.718 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-07-06 19:55:06.415433432 +0000 2023-07-06T20:12:00.718 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-07-06 19:55:06.415433432 +0000 2023-07-06T20:12:00.718 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-07-06T20:12:00.718 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T20:12:00.780 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:00 smithi156 systemd[1]: Starting Ceph mgr.z for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:12:00.783 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-07-06T20:12:00.783 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-07-06T20:12:00.783 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 9.075e-05 s, 5.6 MB/s 2023-07-06T20:12:00.784 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T20:12:00.843 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2023-07-06T20:12:00.900 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T20:12:00.901 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:00.901 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 373561 Links: 1 2023-07-06T20:12:00.901 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:00.901 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:00.901 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-07-06 20:12:00.471551571 +0000 2023-07-06T20:12:00.901 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-07-06 19:55:06.963425276 +0000 2023-07-06T20:12:00.901 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-07-06 19:55:06.963425276 +0000 2023-07-06T20:12:00.902 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-07-06T20:12:00.902 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T20:12:00.966 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-07-06T20:12:00.966 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-07-06T20:12:00.966 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000104559 s, 4.9 MB/s 2023-07-06T20:12:00.967 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T20:12:01.025 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2023-07-06T20:12:01.082 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T20:12:01.083 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T20:12:01.083 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 383027 Links: 1 2023-07-06T20:12:01.083 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T20:12:01.083 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T20:12:01.083 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-07-06 20:12:00.471551571 +0000 2023-07-06T20:12:01.084 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-07-06 19:55:07.500417285 +0000 2023-07-06T20:12:01.084 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-07-06 19:55:07.500417285 +0000 2023-07-06T20:12:01.084 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-07-06T20:12:01.084 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T20:12:01.106 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:00 smithi156 podman[131194]: 2023-07-06T20:12:01.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:01 smithi156 ceph-mon[129280]: Deploying daemon mgr.z on smithi156 2023-07-06T20:12:01.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:01 smithi156 ceph-mon[129280]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:01.133 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-07-06T20:12:01.133 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-07-06T20:12:01.133 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.00011507 s, 4.4 MB/s 2023-07-06T20:12:01.134 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T20:12:01.191 INFO:tasks.cephadm:Deploying osd.0 on smithi069 with /dev/vg_nvme/lv_4... 2023-07-06T20:12:01.192 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-06T20:12:01.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:01 smithi098 ceph-mon[128968]: Deploying daemon mgr.z on smithi156 2023-07-06T20:12:01.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:01 smithi098 ceph-mon[128968]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:01.406 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:01 smithi156 bash[131194]: 2761d1a14d2b31540e89f87fee8556e83ca6ab617426b03018485e3a404a9eba 2023-07-06T20:12:01.407 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:01 smithi156 ceph-mgr[131377]: -- 172.21.15.156:0/2666988532 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56088e4531e0 con 0x56088e428800 2023-07-06T20:12:01.407 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:01 smithi156 systemd[1]: Started Ceph mgr.z for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:12:01.407 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:01 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:01.231+0000 7f3cc8de4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T20:12:01.407 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:01 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:01.315+0000 7f3cc8de4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T20:12:01.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:01 smithi069 ceph-mon[126399]: Deploying daemon mgr.z on smithi156 2023-07-06T20:12:01.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:01 smithi069 ceph-mon[126399]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:01.667 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:01 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:01.405+0000 7f3cc8de4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T20:12:01.668 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:01 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:01.498+0000 7f3cc8de4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T20:12:01.918 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:12:01.974 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:01 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:01.666+0000 7f3cc8de4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T20:12:02.272 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:01 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:01.973+0000 7f3cc8de4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T20:12:02.272 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:02.061+0000 7f3cc8de4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T20:12:02.272 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:02.190+0000 7f3cc8de4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T20:12:02.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:02.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:02.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:02.455 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-06T20:12:02.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-06T20:12:02.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-06T20:12:02.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:02.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:02 smithi069 ceph-mon[126399]: Deploying daemon mgr.y on smithi098 2023-07-06T20:12:02.456 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:02 smithi069 ceph-mon[126399]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:02.606 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:02.285+0000 7f3cc8de4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T20:12:02.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:02.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:02.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:02.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-06T20:12:02.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-06T20:12:02.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-06T20:12:02.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:02.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-mon[129280]: Deploying daemon mgr.y on smithi098 2023-07-06T20:12:02.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-mon[129280]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:02.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:02.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:02.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:02.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-06T20:12:02.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-06T20:12:02.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-06T20:12:02.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:02.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:02 smithi098 ceph-mon[128968]: Deploying daemon mgr.y on smithi098 2023-07-06T20:12:02.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:02 smithi098 ceph-mon[128968]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:03.106 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:02 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:02.705+0000 7f3cc8de4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T20:12:03.126 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:02 smithi098 systemd[1]: Starting Ceph mgr.y for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:12:03.461 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:03 smithi098 podman[130404]: 2023-07-06T20:12:03.461 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:03 smithi098 bash[130404]: f12ac3e248e39cc23df301f9b1db5099103f2662559263e403a7cd4c947ba6c3 2023-07-06T20:12:03.462 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:03 smithi098 ceph-mgr[130444]: -- 172.21.15.98:0/3389275061 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557f038dd1e0 con 0x557f038b2800 2023-07-06T20:12:03.606 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:03 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:03.275+0000 7f3cc8de4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T20:12:03.606 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:03 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:03.447+0000 7f3cc8de4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T20:12:03.765 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:03 smithi098 systemd[1]: Started Ceph mgr.y for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:12:03.766 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:03 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:03.495+0000 7f43f074b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T20:12:03.766 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:03 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:03.578+0000 7f43f074b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T20:12:03.766 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:03 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:03.667+0000 7f43f074b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T20:12:03.766 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:03 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:03.763+0000 7f43f074b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T20:12:03.796 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-06T20:12:03.796 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-06T20:12:03.797 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2023-07-06T20:12:03.797 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2023-07-06T20:12:03.797 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0472664 s, 222 MB/s 2023-07-06T20:12:03.797 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2023-07-06T20:12:04.125 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:03 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:03.939+0000 7f43f074b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T20:12:04.288 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch daemon add osd smithi069:vg_nvme/lv_4 2023-07-06T20:12:04.512 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:04 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:04.237+0000 7f3cc8de4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:12:04.512 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:04 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:04.334+0000 7f3cc8de4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T20:12:04.521 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:04 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:04.264+0000 7f43f074b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T20:12:04.522 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:04 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:04.359+0000 7f43f074b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T20:12:04.522 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:04 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:04.495+0000 7f43f074b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T20:12:04.809 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:04 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:04.510+0000 7f3cc8de4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:12:04.809 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:04 smithi156 ceph-mon[129280]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:04.809 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:04 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:04.809 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:04 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:04.809 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:04 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:04.810 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:04 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:04.810 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:04 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:04.875 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:04 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:04.595+0000 7f43f074b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T20:12:04.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:04 smithi098 ceph-mon[128968]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:04.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:04 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:04.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:04 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:04.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:04 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:04.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:04 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:04.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:04 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:05.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:04 smithi069 ceph-mon[126399]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:05.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:04 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:05.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:04 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:05.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:04 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:05.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:04 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:05.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:04 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:05.017 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:12:05.106 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:04 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:04.807+0000 7f3cc8de4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T20:12:05.106 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:05 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:05.049+0000 7f3cc8de4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T20:12:05.375 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:05 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:05.038+0000 7f43f074b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T20:12:05.480 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:05 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:05.216+0000 7f3cc8de4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T20:12:05.856 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:05 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:05.479+0000 7f3cc8de4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T20:12:06.125 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:05 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:05.624+0000 7f43f074b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T20:12:06.125 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:05 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:05.792+0000 7f43f074b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T20:12:06.504 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:06.184+0000 7f3cc8de4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T20:12:06.504 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:06.313+0000 7f3cc8de4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T20:12:06.505 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.505 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.505 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.505 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.505 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:06.505 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:06.505 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.506 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:06.506 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: Reconfiguring mgr.x (unknown last config time)... 2023-07-06T20:12:06.506 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-06T20:12:06.506 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-06T20:12:06.506 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:06.506 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-mon[129280]: Reconfiguring daemon mgr.x on smithi069 2023-07-06T20:12:06.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:06.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:06.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:06.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: Reconfiguring mgr.x (unknown last config time)... 2023-07-06T20:12:06.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-06T20:12:06.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-06T20:12:06.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:06.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:06 smithi069 ceph-mon[126399]: Reconfiguring daemon mgr.x on smithi069 2023-07-06T20:12:06.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.621+0000 7f9617d83700 1 -- 172.21.15.69:0/3754791451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610107b60 msgr2=0x7f9610109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:06.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.621+0000 7f9617d83700 1 --2- 172.21.15.69:0/3754791451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610107b60 0x7f9610109fe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f960800b0a0 tx=0x7f960800b3b0 comp rx=0 tx=0).stop 2023-07-06T20:12:06.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.621+0000 7f9617d83700 1 -- 172.21.15.69:0/3754791451 shutdown_connections 2023-07-06T20:12:06.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.621+0000 7f9617d83700 1 --2- 172.21.15.69:0/3754791451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610107b60 0x7f9610109fe0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:06.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.621+0000 7f9617d83700 1 --2- 172.21.15.69:0/3754791451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610074610 0x7f9610072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:06.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.621+0000 7f9617d83700 1 --2- 172.21.15.69:0/3754791451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610073040 0x7f96100734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.621+0000 7f9617d83700 1 -- 172.21.15.69:0/3754791451 >> 172.21.15.69:0/3754791451 conn(0x7f96100fb2d0 msgr2=0x7f96100fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:12:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.621+0000 7f9617d83700 1 -- 172.21.15.69:0/3754791451 shutdown_connections 2023-07-06T20:12:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.621+0000 7f9617d83700 1 -- 172.21.15.69:0/3754791451 wait complete. 2023-07-06T20:12:06.625 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:06.550+0000 7f43f074b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:12:06.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:06.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:06.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:06.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:06.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: Reconfiguring mgr.x (unknown last config time)... 2023-07-06T20:12:06.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-06T20:12:06.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-06T20:12:06.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:06.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-mon[128968]: Reconfiguring daemon mgr.x on smithi069 2023-07-06T20:12:06.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.622+0000 7f9617d83700 1 Processor -- start 2023-07-06T20:12:06.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.623+0000 7f9617d83700 1 -- start start 2023-07-06T20:12:06.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.623+0000 7f9617d83700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610073040 0x7f9610197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:12:06.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.623+0000 7f9617d83700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610074610 0x7f9610198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:12:06.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.623+0000 7f9617d83700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610107b60 0x7f961019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:12:06.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.623+0000 7f9617d83700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961019f850 con 0x7f9610107b60 2023-07-06T20:12:06.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.623+0000 7f9617d83700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961019f990 con 0x7f9610074610 2023-07-06T20:12:06.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.623+0000 7f9617d83700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961019fb50 con 0x7f9610073040 2023-07-06T20:12:06.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.623+0000 7f9615b1f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610073040 0x7f9610197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:12:06.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.623+0000 7f9615b1f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610073040 0x7f9610197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43952/0 (socket says 172.21.15.69:43952) 2023-07-06T20:12:06.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.623+0000 7f9615b1f700 1 -- 172.21.15.69:0/183803185 learned_addr learned my addr 172.21.15.69:0/183803185 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:12:06.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.623+0000 7f961531e700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610074610 0x7f9610198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:12:06.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.623+0000 7f9616320700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610107b60 0x7f961019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:12:06.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.624+0000 7f9615b1f700 1 -- 172.21.15.69:0/183803185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610074610 msgr2=0x7f9610198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:06.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.624+0000 7f9615b1f700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610074610 0x7f9610198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:06.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.624+0000 7f9615b1f700 1 -- 172.21.15.69:0/183803185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610107b60 msgr2=0x7f961019f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:06.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.624+0000 7f9615b1f700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610107b60 0x7f961019f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:06.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.624+0000 7f9615b1f700 1 -- 172.21.15.69:0/183803185 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9608009d00 con 0x7f9610073040 2023-07-06T20:12:06.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.624+0000 7f9615b1f700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610073040 0x7f9610197d00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f960c00e9a0 tx=0x7f960c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:12:06.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.624+0000 7f9606ffd700 1 -- 172.21.15.69:0/183803185 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c009790 con 0x7f9610073040 2023-07-06T20:12:06.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.624+0000 7f9606ffd700 1 -- 172.21.15.69:0/183803185 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f960c010430 con 0x7f9610073040 2023-07-06T20:12:06.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.624+0000 7f9617d83700 1 -- 172.21.15.69:0/183803185 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961019fe30 con 0x7f9610073040 2023-07-06T20:12:06.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.624+0000 7f9606ffd700 1 -- 172.21.15.69:0/183803185 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c018680 con 0x7f9610073040 2023-07-06T20:12:06.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.624+0000 7f9617d83700 1 -- 172.21.15.69:0/183803185 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96101a0240 con 0x7f9610073040 2023-07-06T20:12:06.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.625+0000 7f9606ffd700 1 -- 172.21.15.69:0/183803185 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f960c0187e0 con 0x7f9610073040 2023-07-06T20:12:06.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.625+0000 7f9617d83700 1 -- 172.21.15.69:0/183803185 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96101062a0 con 0x7f9610073040 2023-07-06T20:12:06.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.626+0000 7f9606ffd700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f95fc0389e0 0x7f95fc03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:12:06.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.626+0000 7f9606ffd700 1 -- 172.21.15.69:0/183803185 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f960c014070 con 0x7f9610073040 2023-07-06T20:12:06.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.626+0000 7f961531e700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f95fc0389e0 0x7f95fc03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:12:06.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.627+0000 7f961531e700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f95fc0389e0 0x7f95fc03ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9600009c60 tx=0x7f9600009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:12:06.634 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.631+0000 7f9606ffd700 1 -- 172.21.15.69:0/183803185 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f960c01f920 con 0x7f9610073040 2023-07-06T20:12:06.834 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:06.502+0000 7f3cc8de4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T20:12:06.834 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:06.617+0000 7f3cc8de4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T20:12:06.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:06.849+0000 7f9617d83700 1 -- 172.21.15.69:0/183803185 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9610060a80 con 0x7f95fc0389e0 2023-07-06T20:12:07.088 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:06.640+0000 7f43f074b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T20:12:07.089 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:06 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:06.808+0000 7f43f074b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T20:12:07.348 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:07.086+0000 7f43f074b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T20:12:07.348 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:07.312+0000 7f43f074b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T20:12:07.606 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:06 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:06.832+0000 7f3cc8de4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T20:12:07.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:07 smithi156 ceph-mon[129280]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:12:07.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:12:07.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:12:07.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:07.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:07.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:07.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:07.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:07.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:07.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:07.625 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:07.459+0000 7f43f074b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T20:12:07.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-mon[128968]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:12:07.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:12:07.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:12:07.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:07.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:07.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:07.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:07.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:07.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:07.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:07.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:07 smithi069 ceph-mon[126399]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:12:07.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:12:07.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:12:07.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:07.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:07.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:07.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:07.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:07.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:07.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:08.106 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:12:07 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z[131283]: 2023-07-06T20:12:07.692+0000 7f3cc8de4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T20:12:08.125 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:07 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:07.720+0000 7f43f074b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T20:12:08.511 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:08.509+0000 7f9606ffd700 1 -- 172.21.15.69:0/183803185 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7f960c0105a0 con 0x7f9610073040 2023-07-06T20:12:08.737 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:08 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:08.419+0000 7f43f074b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T20:12:08.737 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:08 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:08.548+0000 7f43f074b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T20:12:08.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:08 smithi098 ceph-mon[128968]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:08.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:08.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:08 smithi098 ceph-mon[128968]: Standby manager daemon z started 2023-07-06T20:12:08.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:08 smithi098 ceph-mon[128968]: from='mgr.? 172.21.15.156:0/979822633' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-06T20:12:08.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:08 smithi098 ceph-mon[128968]: from='mgr.? 172.21.15.156:0/979822633' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-06T20:12:08.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:08 smithi098 ceph-mon[128968]: from='mgr.? 172.21.15.156:0/979822633' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-06T20:12:08.739 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:08 smithi098 ceph-mon[128968]: from='mgr.? 172.21.15.156:0/979822633' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-06T20:12:08.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:08 smithi069 ceph-mon[126399]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:08.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:08.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:08 smithi069 ceph-mon[126399]: Standby manager daemon z started 2023-07-06T20:12:08.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:08 smithi069 ceph-mon[126399]: from='mgr.? 172.21.15.156:0/979822633' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-06T20:12:08.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:08 smithi069 ceph-mon[126399]: from='mgr.? 172.21.15.156:0/979822633' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-06T20:12:08.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:08 smithi069 ceph-mon[126399]: from='mgr.? 172.21.15.156:0/979822633' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-06T20:12:08.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:08 smithi069 ceph-mon[126399]: from='mgr.? 172.21.15.156:0/979822633' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-06T20:12:08.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:08 smithi156 ceph-mon[129280]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:08.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:08.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:08 smithi156 ceph-mon[129280]: Standby manager daemon z started 2023-07-06T20:12:08.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:08 smithi156 ceph-mon[129280]: from='mgr.? 172.21.15.156:0/979822633' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-06T20:12:08.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:08 smithi156 ceph-mon[129280]: from='mgr.? 172.21.15.156:0/979822633' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-06T20:12:08.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:08 smithi156 ceph-mon[129280]: from='mgr.? 172.21.15.156:0/979822633' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-06T20:12:08.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:08 smithi156 ceph-mon[129280]: from='mgr.? 172.21.15.156:0/979822633' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-06T20:12:09.065 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:08 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:08.734+0000 7f43f074b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T20:12:09.066 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:08 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:08.847+0000 7f43f074b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T20:12:09.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:09 smithi156 ceph-mon[129280]: mgrmap e14: x(active, since 71s), standbys: z 2023-07-06T20:12:09.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:09 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-06T20:12:09.875 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:09 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:09.063+0000 7f43f074b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T20:12:09.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:09 smithi098 ceph-mon[128968]: mgrmap e14: x(active, since 71s), standbys: z 2023-07-06T20:12:09.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:09 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-06T20:12:10.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:09 smithi069 ceph-mon[126399]: mgrmap e14: x(active, since 71s), standbys: z 2023-07-06T20:12:10.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:09 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-06T20:12:10.375 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:12:09 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y[130420]: 2023-07-06T20:12:09.935+0000 7f43f074b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T20:12:10.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:10.817+0000 7f9606ffd700 1 -- 172.21.15.69:0/183803185 <== mon.1 v2:172.21.15.156:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f960c010880 con 0x7f9610073040 2023-07-06T20:12:10.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:10 smithi156 ceph-mon[129280]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:10.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:10 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3898932466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02b7a479-153d-48fb-bac6-606262f65f1a"}]: dispatch 2023-07-06T20:12:10.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:10 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02b7a479-153d-48fb-bac6-606262f65f1a"}]: dispatch 2023-07-06T20:12:10.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:10 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02b7a479-153d-48fb-bac6-606262f65f1a"}]': finished 2023-07-06T20:12:10.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:10 smithi156 ceph-mon[129280]: osdmap e5: 1 total, 0 up, 1 in 2023-07-06T20:12:10.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:10 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:10.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:10 smithi156 ceph-mon[129280]: Standby manager daemon y started 2023-07-06T20:12:10.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:10 smithi156 ceph-mon[129280]: from='mgr.? 172.21.15.98:0/922673474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-06T20:12:10.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:10 smithi156 ceph-mon[129280]: from='mgr.? 172.21.15.98:0/922673474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-06T20:12:10.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:10 smithi156 ceph-mon[129280]: from='mgr.? 172.21.15.98:0/922673474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-06T20:12:10.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:10 smithi156 ceph-mon[129280]: from='mgr.? 172.21.15.98:0/922673474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-06T20:12:10.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:10 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2187638238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:12:10.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:10 smithi098 ceph-mon[128968]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:10.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:10 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3898932466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02b7a479-153d-48fb-bac6-606262f65f1a"}]: dispatch 2023-07-06T20:12:10.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:10 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02b7a479-153d-48fb-bac6-606262f65f1a"}]: dispatch 2023-07-06T20:12:10.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:10 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02b7a479-153d-48fb-bac6-606262f65f1a"}]': finished 2023-07-06T20:12:10.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:10 smithi098 ceph-mon[128968]: osdmap e5: 1 total, 0 up, 1 in 2023-07-06T20:12:10.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:10 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:10.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:10 smithi098 ceph-mon[128968]: Standby manager daemon y started 2023-07-06T20:12:10.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:10 smithi098 ceph-mon[128968]: from='mgr.? 172.21.15.98:0/922673474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-06T20:12:10.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:10 smithi098 ceph-mon[128968]: from='mgr.? 172.21.15.98:0/922673474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-06T20:12:10.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:10 smithi098 ceph-mon[128968]: from='mgr.? 172.21.15.98:0/922673474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-06T20:12:10.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:10 smithi098 ceph-mon[128968]: from='mgr.? 172.21.15.98:0/922673474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-06T20:12:10.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:10 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2187638238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:12:11.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:10 smithi069 ceph-mon[126399]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:11.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:10 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3898932466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02b7a479-153d-48fb-bac6-606262f65f1a"}]: dispatch 2023-07-06T20:12:11.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:10 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02b7a479-153d-48fb-bac6-606262f65f1a"}]: dispatch 2023-07-06T20:12:11.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:10 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02b7a479-153d-48fb-bac6-606262f65f1a"}]': finished 2023-07-06T20:12:11.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:10 smithi069 ceph-mon[126399]: osdmap e5: 1 total, 0 up, 1 in 2023-07-06T20:12:11.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:11.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:10 smithi069 ceph-mon[126399]: Standby manager daemon y started 2023-07-06T20:12:11.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:10 smithi069 ceph-mon[126399]: from='mgr.? 172.21.15.98:0/922673474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-06T20:12:11.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:10 smithi069 ceph-mon[126399]: from='mgr.? 172.21.15.98:0/922673474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-06T20:12:11.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:10 smithi069 ceph-mon[126399]: from='mgr.? 172.21.15.98:0/922673474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-06T20:12:11.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:10 smithi069 ceph-mon[126399]: from='mgr.? 172.21.15.98:0/922673474' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-06T20:12:11.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:10 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2187638238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:12:12.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:11 smithi156 ceph-mon[129280]: mgrmap e15: x(active, since 73s), standbys: z, y 2023-07-06T20:12:12.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-06T20:12:12.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:11 smithi098 ceph-mon[128968]: mgrmap e15: x(active, since 73s), standbys: z, y 2023-07-06T20:12:12.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-06T20:12:12.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:11 smithi069 ceph-mon[126399]: mgrmap e15: x(active, since 73s), standbys: z, y 2023-07-06T20:12:12.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-06T20:12:13.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:12 smithi156 ceph-mon[129280]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:13.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:12 smithi098 ceph-mon[128968]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:13.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:12 smithi069 ceph-mon[126399]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:15.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:14 smithi156 ceph-mon[129280]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:15.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:14 smithi098 ceph-mon[128968]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:15.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:14 smithi069 ceph-mon[126399]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:17.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:16 smithi156 ceph-mon[129280]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:17.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:16 smithi098 ceph-mon[128968]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:17.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:16 smithi069 ceph-mon[126399]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:18.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:18 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-06T20:12:18.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:18 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:18.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:18 smithi156 ceph-mon[129280]: Deploying daemon osd.0 on smithi069 2023-07-06T20:12:18.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:18 smithi156 ceph-mon[129280]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:18.369 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:18 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-06T20:12:18.369 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:18 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:18.370 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:18 smithi069 ceph-mon[126399]: Deploying daemon osd.0 on smithi069 2023-07-06T20:12:18.370 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:18 smithi069 ceph-mon[126399]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:18.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:18 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-06T20:12:18.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:18 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:18.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:18 smithi098 ceph-mon[128968]: Deploying daemon osd.0 on smithi069 2023-07-06T20:12:18.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:18 smithi098 ceph-mon[128968]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:20.574 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:20 smithi069 ceph-mon[126399]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:20.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:20 smithi156 ceph-mon[129280]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:20.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:20 smithi098 ceph-mon[128968]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:22.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:22 smithi156 ceph-mon[129280]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:22.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:22 smithi098 ceph-mon[128968]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:22.987 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:22 smithi069 ceph-mon[126399]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:24.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:24 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:24.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:24 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:24.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:24 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:24.595 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:24 smithi069 ceph-mon[126399]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:24.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:24 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:24 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:24 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:24 smithi156 ceph-mon[129280]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:24.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:24 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:24.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:24 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:24.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:24 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:24.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:24 smithi098 ceph-mon[128968]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:26.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.813+0000 7f9606ffd700 1 -- 172.21.15.69:0/183803185 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9610060a80 con 0x7f95fc0389e0 2023-07-06T20:12:26.815 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 0 on host 'smithi069' 2023-07-06T20:12:26.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.815+0000 7f9617d83700 1 -- 172.21.15.69:0/183803185 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f95fc0389e0 msgr2=0x7f95fc03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:26.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.815+0000 7f9617d83700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f95fc0389e0 0x7f95fc03ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9600009c60 tx=0x7f9600009260 comp rx=0 tx=0).stop 2023-07-06T20:12:26.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.815+0000 7f9617d83700 1 -- 172.21.15.69:0/183803185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610073040 msgr2=0x7f9610197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:26.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.815+0000 7f9617d83700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610073040 0x7f9610197d00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f960c00e9a0 tx=0x7f960c00ecb0 comp rx=0 tx=0).stop 2023-07-06T20:12:26.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.816+0000 7f9617d83700 1 -- 172.21.15.69:0/183803185 shutdown_connections 2023-07-06T20:12:26.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.816+0000 7f9617d83700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f95fc0389e0 0x7f95fc03ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:26.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.816+0000 7f9617d83700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610107b60 0x7f961019f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:26.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.816+0000 7f9617d83700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610073040 0x7f9610197d00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:26.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.816+0000 7f9617d83700 1 --2- 172.21.15.69:0/183803185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610074610 0x7f9610198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:26.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.816+0000 7f9617d83700 1 -- 172.21.15.69:0/183803185 >> 172.21.15.69:0/183803185 conn(0x7f96100fb2d0 msgr2=0x7f96100fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:12:26.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.816+0000 7f9617d83700 1 -- 172.21.15.69:0/183803185 shutdown_connections 2023-07-06T20:12:26.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:12:26.816+0000 7f9617d83700 1 -- 172.21.15.69:0/183803185 wait complete. 2023-07-06T20:12:27.234 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:27 smithi069 ceph-mon[126399]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:27.234 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.234 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.234 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:27.234 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:27.234 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.235 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.235 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.235 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:27.236 DEBUG:teuthology.orchestra.run.smithi069:osd.0> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.0.service 2023-07-06T20:12:27.240 INFO:tasks.cephadm:Deploying osd.1 on smithi098 with /dev/vg_nvme/lv_4... 2023-07-06T20:12:27.240 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-06T20:12:27.275 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:27 smithi098 ceph-mon[128968]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:27.275 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.276 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.276 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:27.276 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:27.276 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.276 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.276 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.277 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:27.315 INFO:journalctl@ceph.osd.0.smithi069.stdout:-- Logs begin at Thu 2023-07-06 19:46:03 UTC. -- 2023-07-06T20:12:27.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:27 smithi156 ceph-mon[129280]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:27.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:27.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:27.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:27.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:27.509 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:12:27 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:12:27.323+0000 7f872242c640 -1 osd.0 0 log_to_monitors true 2023-07-06T20:12:27.801 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.b/config 2023-07-06T20:12:28.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:28 smithi069 ceph-mon[126399]: from='osd.0 [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-06T20:12:28.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:28 smithi069 ceph-mon[126399]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-06T20:12:28.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:28 smithi069 ceph-mon[126399]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:28.334 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:28 smithi098 ceph-mon[128968]: from='osd.0 [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-06T20:12:28.335 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:28 smithi098 ceph-mon[128968]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-06T20:12:28.335 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:28 smithi098 ceph-mon[128968]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:28.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:28 smithi156 ceph-mon[129280]: from='osd.0 [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-06T20:12:28.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:28 smithi156 ceph-mon[129280]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-06T20:12:28.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:28 smithi156 ceph-mon[129280]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:29.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:29 smithi069 ceph-mon[126399]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-06T20:12:29.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:29 smithi069 ceph-mon[126399]: osdmap e6: 1 total, 0 up, 1 in 2023-07-06T20:12:29.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:29 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:29.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:29 smithi069 ceph-mon[126399]: from='osd.0 [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:12:29.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:29 smithi069 ceph-mon[126399]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:12:29.260 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:12:29 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:12:29.084+0000 7f87172a3700 -1 osd.0 0 waiting for initial osdmap 2023-07-06T20:12:29.260 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:12:29 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:12:29.088+0000 7f8711895700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:12:29.307 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-06T20:12:29.307 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-06T20:12:29.307 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2023-07-06T20:12:29.307 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2023-07-06T20:12:29.307 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110148 s, 952 MB/s 2023-07-06T20:12:29.307 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2023-07-06T20:12:29.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:29 smithi098 ceph-mon[128968]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-06T20:12:29.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:29 smithi098 ceph-mon[128968]: osdmap e6: 1 total, 0 up, 1 in 2023-07-06T20:12:29.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:29 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:29.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:29 smithi098 ceph-mon[128968]: from='osd.0 [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:12:29.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:29 smithi098 ceph-mon[128968]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:12:29.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:29 smithi156 ceph-mon[129280]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-06T20:12:29.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:29 smithi156 ceph-mon[129280]: osdmap e6: 1 total, 0 up, 1 in 2023-07-06T20:12:29.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:29 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:29.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:29 smithi156 ceph-mon[129280]: from='osd.0 [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:12:29.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:29 smithi156 ceph-mon[129280]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:12:29.743 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch daemon add osd smithi098:vg_nvme/lv_4 2023-07-06T20:12:30.336 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.b/config 2023-07-06T20:12:30.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:30 smithi156 ceph-mon[129280]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-07-06T20:12:30.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:30 smithi156 ceph-mon[129280]: osdmap e7: 1 total, 0 up, 1 in 2023-07-06T20:12:30.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:30 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:30.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:30 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:30.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:30 smithi156 ceph-mon[129280]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:30.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:30 smithi098 ceph-mon[128968]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-07-06T20:12:30.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:30 smithi098 ceph-mon[128968]: osdmap e7: 1 total, 0 up, 1 in 2023-07-06T20:12:30.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:30 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:30.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:30 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:30.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:30 smithi098 ceph-mon[128968]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:30.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:30 smithi069 ceph-mon[126399]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-07-06T20:12:30.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:30 smithi069 ceph-mon[126399]: osdmap e7: 1 total, 0 up, 1 in 2023-07-06T20:12:30.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:30.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:30.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:30 smithi069 ceph-mon[126399]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-06T20:12:31.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: purged_snaps scrub starts 2023-07-06T20:12:31.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: purged_snaps scrub ok 2023-07-06T20:12:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: osd.0 [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786] boot 2023-07-06T20:12:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: osdmap e8: 1 total, 1 up, 1 in 2023-07-06T20:12:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: Detected new or changed devices on smithi069 2023-07-06T20:12:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:12:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi069 to 17267M 2023-07-06T20:12:31.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:31.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:31.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:31.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:31.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: purged_snaps scrub starts 2023-07-06T20:12:31.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: purged_snaps scrub ok 2023-07-06T20:12:31.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: osd.0 [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786] boot 2023-07-06T20:12:31.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: osdmap e8: 1 total, 1 up, 1 in 2023-07-06T20:12:31.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:31.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: Detected new or changed devices on smithi069 2023-07-06T20:12:31.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:31.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:31.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:12:31.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi069 to 17267M 2023-07-06T20:12:31.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:31.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:31.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:31.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:31.509 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.506+0000 7f7aab4ab700 1 -- 172.21.15.98:0/3701566991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7aa4101880 msgr2=0x7f7aa4108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.506+0000 7f7aab4ab700 1 --2- 172.21.15.98:0/3701566991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7aa4101880 0x7f7aa4108140 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00b0a0 tx=0x7f7a9c00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:12:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.506+0000 7f7aab4ab700 1 -- 172.21.15.98:0/3701566991 shutdown_connections 2023-07-06T20:12:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.506+0000 7f7aab4ab700 1 --2- 172.21.15.98:0/3701566991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7aa4101880 0x7f7aa4108140 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.506+0000 7f7aab4ab700 1 --2- 172.21.15.98:0/3701566991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa40ffcf0 0x7f7aa4100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:31.510 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.506+0000 7f7aab4ab700 1 --2- 172.21.15.98:0/3701566991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7aa4100ef0 0x7f7aa4101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.506+0000 7f7aab4ab700 1 -- 172.21.15.98:0/3701566991 >> 172.21.15.98:0/3701566991 conn(0x7f7aa40fb360 msgr2=0x7f7aa40fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:12:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.506+0000 7f7aab4ab700 1 -- 172.21.15.98:0/3701566991 shutdown_connections 2023-07-06T20:12:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.506+0000 7f7aab4ab700 1 -- 172.21.15.98:0/3701566991 wait complete. 2023-07-06T20:12:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aab4ab700 1 Processor -- start 2023-07-06T20:12:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aab4ab700 1 -- start start 2023-07-06T20:12:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aab4ab700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa40ffcf0 0x7f7aa40743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:12:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aab4ab700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7aa4100ef0 0x7f7aa4072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:12:31.511 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aab4ab700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7aa4101880 0x7f7aa4072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:12:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aab4ab700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40736f0 con 0x7f7aa40ffcf0 2023-07-06T20:12:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aab4ab700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4073860 con 0x7f7aa4100ef0 2023-07-06T20:12:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aab4ab700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa419fb00 con 0x7f7aa4101880 2023-07-06T20:12:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aa8a46700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7aa4100ef0 0x7f7aa4072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:12:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aa8a46700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7aa4100ef0 0x7f7aa4072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:44302/0 (socket says 172.21.15.98:44302) 2023-07-06T20:12:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aa8a46700 1 -- 172.21.15.98:0/1227968524 learned_addr learned my addr 172.21.15.98:0/1227968524 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-07-06T20:12:31.512 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aa9247700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa40ffcf0 0x7f7aa40743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:12:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aa9a48700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7aa4101880 0x7f7aa4072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:12:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aa8a46700 1 -- 172.21.15.98:0/1227968524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7aa4101880 msgr2=0x7f7aa4072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.507+0000 7f7aa8a46700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7aa4101880 0x7f7aa4072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.508+0000 7f7aa8a46700 1 -- 172.21.15.98:0/1227968524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa40ffcf0 msgr2=0x7f7aa40743e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.508+0000 7f7aa8a46700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa40ffcf0 0x7f7aa40743e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.508+0000 7f7aa8a46700 1 -- 172.21.15.98:0/1227968524 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c009d00 con 0x7f7aa4100ef0 2023-07-06T20:12:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.508+0000 7f7aa8a46700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7aa4100ef0 0x7f7aa4072a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000c9b0 tx=0x7f7aa000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:12:31.513 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.508+0000 7f7a9a7fc700 1 -- 172.21.15.98:0/1227968524 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0007b00 con 0x7f7aa4100ef0 2023-07-06T20:12:31.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.508+0000 7f7aab4ab700 1 -- 172.21.15.98:0/1227968524 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa419fde0 con 0x7f7aa4100ef0 2023-07-06T20:12:31.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.508+0000 7f7a9a7fc700 1 -- 172.21.15.98:0/1227968524 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aa000ce50 con 0x7f7aa4100ef0 2023-07-06T20:12:31.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.508+0000 7f7a9a7fc700 1 -- 172.21.15.98:0/1227968524 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0018660 con 0x7f7aa4100ef0 2023-07-06T20:12:31.514 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.508+0000 7f7aab4ab700 1 -- 172.21.15.98:0/1227968524 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa41a02e0 con 0x7f7aa4100ef0 2023-07-06T20:12:31.515 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: purged_snaps scrub starts 2023-07-06T20:12:31.515 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: purged_snaps scrub ok 2023-07-06T20:12:31.515 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: osd.0 [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786] boot 2023-07-06T20:12:31.515 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: osdmap e8: 1 total, 1 up, 1 in 2023-07-06T20:12:31.515 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-06T20:12:31.515 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: Detected new or changed devices on smithi069 2023-07-06T20:12:31.515 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:31.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:31.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:12:31.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi069 to 17267M 2023-07-06T20:12:31.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:31.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:31.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:31.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.509+0000 7f7aab4ab700 1 -- 172.21.15.98:0/1227968524 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa410b550 con 0x7f7aa4100ef0 2023-07-06T20:12:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.510+0000 7f7a9a7fc700 1 -- 172.21.15.98:0/1227968524 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f7aa000f450 con 0x7f7aa4100ef0 2023-07-06T20:12:31.517 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.510+0000 7f7a9a7fc700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7a900a10c0 0x7f7a900a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:12:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.510+0000 7f7a9a7fc700 1 -- 172.21.15.98:0/1227968524 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f7aa00cae10 con 0x7f7aa4100ef0 2023-07-06T20:12:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.511+0000 7f7aa9247700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7a900a10c0 0x7f7a900a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:12:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.511+0000 7f7aa9247700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7a900a10c0 0x7f7a900a3570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4100d50 tx=0x7f7a94009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:12:31.518 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.514+0000 7f7a9a7fc700 1 -- 172.21.15.98:0/1227968524 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7aa0095fd0 con 0x7f7aa4100ef0 2023-07-06T20:12:31.711 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:31.710+0000 7f7aab4ab700 1 -- 172.21.15.98:0/1227968524 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7aa4060a80 con 0x7f7a900a10c0 2023-07-06T20:12:32.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:32 smithi069 ceph-mon[126399]: osdmap e9: 1 total, 1 up, 1 in 2023-07-06T20:12:32.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:32 smithi069 ceph-mon[126399]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:32.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:32 smithi069 ceph-mon[126399]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:12:32.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:32 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:12:32.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:32 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:12:32.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:32 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:32.540 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:32 smithi098 ceph-mon[128968]: osdmap e9: 1 total, 1 up, 1 in 2023-07-06T20:12:32.540 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:32 smithi098 ceph-mon[128968]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:32.540 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:32 smithi098 ceph-mon[128968]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:12:32.540 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:32 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:12:32.541 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:32 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:12:32.541 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:32 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:32.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:32 smithi156 ceph-mon[129280]: osdmap e9: 1 total, 1 up, 1 in 2023-07-06T20:12:32.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:32 smithi156 ceph-mon[129280]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:32.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:32 smithi156 ceph-mon[129280]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:12:32.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:32 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:12:32.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:32 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:12:32.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:32 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:34.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:34 smithi156 ceph-mon[129280]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:34.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:34 smithi156 ceph-mon[129280]: from='client.? 172.21.15.98:0/3484455483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85f52335-a4a7-42f5-8fbc-6414205fe241"}]: dispatch 2023-07-06T20:12:34.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:34 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85f52335-a4a7-42f5-8fbc-6414205fe241"}]: dispatch 2023-07-06T20:12:34.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:34 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85f52335-a4a7-42f5-8fbc-6414205fe241"}]': finished 2023-07-06T20:12:34.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:34 smithi156 ceph-mon[129280]: osdmap e10: 2 total, 1 up, 2 in 2023-07-06T20:12:34.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:34 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:34.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:34 smithi098 ceph-mon[128968]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:34.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:34 smithi098 ceph-mon[128968]: from='client.? 172.21.15.98:0/3484455483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85f52335-a4a7-42f5-8fbc-6414205fe241"}]: dispatch 2023-07-06T20:12:34.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:34 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85f52335-a4a7-42f5-8fbc-6414205fe241"}]: dispatch 2023-07-06T20:12:34.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:34 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85f52335-a4a7-42f5-8fbc-6414205fe241"}]': finished 2023-07-06T20:12:34.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:34 smithi098 ceph-mon[128968]: osdmap e10: 2 total, 1 up, 2 in 2023-07-06T20:12:34.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:34 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:34 smithi069 ceph-mon[126399]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:34 smithi069 ceph-mon[126399]: from='client.? 172.21.15.98:0/3484455483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85f52335-a4a7-42f5-8fbc-6414205fe241"}]: dispatch 2023-07-06T20:12:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:34 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85f52335-a4a7-42f5-8fbc-6414205fe241"}]: dispatch 2023-07-06T20:12:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:34 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85f52335-a4a7-42f5-8fbc-6414205fe241"}]': finished 2023-07-06T20:12:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:34 smithi069 ceph-mon[126399]: osdmap e10: 2 total, 1 up, 2 in 2023-07-06T20:12:35.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:35.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:35 smithi156 ceph-mon[129280]: from='client.? 172.21.15.98:0/3161433157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:12:35.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:35 smithi098 ceph-mon[128968]: from='client.? 172.21.15.98:0/3161433157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:12:36.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:35 smithi069 ceph-mon[126399]: from='client.? 172.21.15.98:0/3161433157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:12:36.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:36 smithi156 ceph-mon[129280]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:36.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:36 smithi098 ceph-mon[128968]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:37.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:36 smithi069 ceph-mon[126399]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:38.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:38 smithi156 ceph-mon[129280]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:38.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:38 smithi098 ceph-mon[128968]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:38.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:38 smithi069 ceph-mon[126399]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:40.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:40 smithi156 ceph-mon[129280]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:40.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:40 smithi098 ceph-mon[128968]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:41.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:40 smithi069 ceph-mon[126399]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:41.786 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:41 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-06T20:12:41.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:41 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:41.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:41 smithi098 ceph-mon[128968]: Deploying daemon osd.1 on smithi098 2023-07-06T20:12:41.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:41 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-06T20:12:41.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:41 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:41.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:41 smithi156 ceph-mon[129280]: Deploying daemon osd.1 on smithi098 2023-07-06T20:12:42.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:41 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-06T20:12:42.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:41 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:42.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:41 smithi069 ceph-mon[126399]: Deploying daemon osd.1 on smithi098 2023-07-06T20:12:42.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:42 smithi156 ceph-mon[129280]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:42.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:42 smithi098 ceph-mon[128968]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:43.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:42 smithi069 ceph-mon[126399]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:44.799 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:44 smithi098 ceph-mon[128968]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:44.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:44 smithi156 ceph-mon[129280]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:45.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:44 smithi069 ceph-mon[126399]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:46.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:46 smithi098 ceph-mon[128968]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:46.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:46.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:46.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:47.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:46 smithi069 ceph-mon[126399]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:47.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:46 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:47.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:46 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:47.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:46 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:47.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:46 smithi156 ceph-mon[129280]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:47.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:46 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:47.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:46 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:47.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:46 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:48.576 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 1 on host 'smithi098' 2023-07-06T20:12:48.576 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.574+0000 7f7a9a7fc700 1 -- 172.21.15.98:0/1227968524 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7aa4060a80 con 0x7f7a900a10c0 2023-07-06T20:12:48.577 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.576+0000 7f7aab4ab700 1 -- 172.21.15.98:0/1227968524 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7a900a10c0 msgr2=0x7f7a900a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:48.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.576+0000 7f7aab4ab700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7a900a10c0 0x7f7a900a3570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4100d50 tx=0x7f7a94009190 comp rx=0 tx=0).stop 2023-07-06T20:12:48.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.576+0000 7f7aab4ab700 1 -- 172.21.15.98:0/1227968524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7aa4100ef0 msgr2=0x7f7aa4072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:48.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.576+0000 7f7aab4ab700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7aa4100ef0 0x7f7aa4072a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000c9b0 tx=0x7f7aa000ccc0 comp rx=0 tx=0).stop 2023-07-06T20:12:48.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.576+0000 7f7aab4ab700 1 -- 172.21.15.98:0/1227968524 shutdown_connections 2023-07-06T20:12:48.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.576+0000 7f7aab4ab700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7a900a10c0 0x7f7a900a3570 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:48.578 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.576+0000 7f7aab4ab700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7aa4101880 0x7f7aa4072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:48.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.576+0000 7f7aab4ab700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa40ffcf0 0x7f7aa40743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:48.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.576+0000 7f7aab4ab700 1 --2- 172.21.15.98:0/1227968524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7aa4100ef0 0x7f7aa4072a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:48.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.576+0000 7f7aab4ab700 1 -- 172.21.15.98:0/1227968524 >> 172.21.15.98:0/1227968524 conn(0x7f7aa40fb360 msgr2=0x7f7aa40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:12:48.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.576+0000 7f7aab4ab700 1 -- 172.21.15.98:0/1227968524 shutdown_connections 2023-07-06T20:12:48.579 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:12:48.576+0000 7f7aab4ab700 1 -- 172.21.15.98:0/1227968524 wait complete. 2023-07-06T20:12:48.999 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:48 smithi098 ceph-mon[128968]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:48.999 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:48.999 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:48.999 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:49.000 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:49.000 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.000 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.000 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.000 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:49.015 DEBUG:teuthology.orchestra.run.smithi098:osd.1> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.1.service 2023-07-06T20:12:49.017 INFO:tasks.cephadm:Deploying osd.2 on smithi156 with /dev/vg_nvme/lv_4... 2023-07-06T20:12:49.018 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-06T20:12:49.047 INFO:journalctl@ceph.osd.1.smithi098.stdout:-- Logs begin at Thu 2023-07-06 19:39:25 UTC. -- 2023-07-06T20:12:49.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:48 smithi069 ceph-mon[126399]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:49.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:49.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:49.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:49.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:48 smithi156 ceph-mon[129280]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:49.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:49.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:49.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:49.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:12:49.579 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:12:49.875 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:12:49 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-1[134064]: 2023-07-06T20:12:49.629+0000 7f7d9af44640 -1 osd.1 0 log_to_monitors true 2023-07-06T20:12:50.069 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:49 smithi156 ceph-mon[129280]: from='osd.1 [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-06T20:12:50.070 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:49 smithi156 ceph-mon[129280]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-06T20:12:50.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:49 smithi069 ceph-mon[126399]: from='osd.1 [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-06T20:12:50.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:49 smithi069 ceph-mon[126399]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-06T20:12:50.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:49 smithi098 ceph-mon[128968]: from='osd.1 [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-06T20:12:50.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:49 smithi098 ceph-mon[128968]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-06T20:12:51.040 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-06T20:12:51.040 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-06T20:12:51.040 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-07-06T20:12:51.041 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-07-06T20:12:51.041 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124009 s, 846 MB/s 2023-07-06T20:12:51.041 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-07-06T20:12:51.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:50 smithi069 ceph-mon[126399]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:51.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:50 smithi069 ceph-mon[126399]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-06T20:12:51.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:50 smithi069 ceph-mon[126399]: osdmap e11: 2 total, 1 up, 2 in 2023-07-06T20:12:51.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:50 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:51.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:50 smithi069 ceph-mon[126399]: from='osd.1 [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:12:51.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:50 smithi069 ceph-mon[126399]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:12:51.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:50 smithi156 ceph-mon[129280]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:51.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:50 smithi156 ceph-mon[129280]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-06T20:12:51.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:50 smithi156 ceph-mon[129280]: osdmap e11: 2 total, 1 up, 2 in 2023-07-06T20:12:51.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:50 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:51.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:50 smithi156 ceph-mon[129280]: from='osd.1 [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:12:51.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:50 smithi156 ceph-mon[129280]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:12:51.375 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:12:50 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-1[134064]: 2023-07-06T20:12:50.941+0000 7f7d8fdbb700 -1 osd.1 0 waiting for initial osdmap 2023-07-06T20:12:51.375 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:12:50 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-1[134064]: 2023-07-06T20:12:50.944+0000 7f7d883a9700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:12:51.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:50 smithi098 ceph-mon[128968]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:51.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:50 smithi098 ceph-mon[128968]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-06T20:12:51.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:50 smithi098 ceph-mon[128968]: osdmap e11: 2 total, 1 up, 2 in 2023-07-06T20:12:51.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:50 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:51.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:50 smithi098 ceph-mon[128968]: from='osd.1 [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:12:51.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:50 smithi098 ceph-mon[128968]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:12:51.430 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch daemon add osd smithi156:vg_nvme/lv_4 2023-07-06T20:12:51.984 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:12:52.204 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:51 smithi156 ceph-mon[129280]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-07-06T20:12:52.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:51 smithi156 ceph-mon[129280]: osdmap e12: 2 total, 1 up, 2 in 2023-07-06T20:12:52.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:52.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:52.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:52.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:52.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:12:52.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:52.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:52.206 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:52.206 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:52.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:51 smithi069 ceph-mon[126399]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-07-06T20:12:52.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:51 smithi069 ceph-mon[126399]: osdmap e12: 2 total, 1 up, 2 in 2023-07-06T20:12:52.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:52.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:52.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:52.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:52.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:12:52.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:52.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:52.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:52.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:52.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:51 smithi098 ceph-mon[128968]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-07-06T20:12:52.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:51 smithi098 ceph-mon[128968]: osdmap e12: 2 total, 1 up, 2 in 2023-07-06T20:12:52.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:52.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:52.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:52.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:52.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:12:52.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:52.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:52.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:12:52.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:12:53.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.155+0000 7f3316729700 1 -- 172.21.15.156:0/2259916887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33100febd0 msgr2=0x7f33100fefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:53.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.155+0000 7f3316729700 1 --2- 172.21.15.156:0/2259916887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33100febd0 0x7f33100fefe0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3304009990 tx=0x7f3304009ca0 comp rx=0 tx=0).stop 2023-07-06T20:12:53.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.155+0000 7f3316729700 1 -- 172.21.15.156:0/2259916887 shutdown_connections 2023-07-06T20:12:53.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.155+0000 7f3316729700 1 --2- 172.21.15.156:0/2259916887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3310100820 0x7f33101070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:53.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.155+0000 7f3316729700 1 --2- 172.21.15.156:0/2259916887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33100febd0 0x7f33100fefe0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:53.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.155+0000 7f3316729700 1 --2- 172.21.15.156:0/2259916887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33100ffe70 0x7f33101002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:53.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.155+0000 7f3316729700 1 -- 172.21.15.156:0/2259916887 >> 172.21.15.156:0/2259916887 conn(0x7f33100fa240 msgr2=0x7f33100fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:12:53.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.155+0000 7f3316729700 1 -- 172.21.15.156:0/2259916887 shutdown_connections 2023-07-06T20:12:53.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.155+0000 7f3316729700 1 -- 172.21.15.156:0/2259916887 wait complete. 2023-07-06T20:12:53.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.155+0000 7f3316729700 1 Processor -- start 2023-07-06T20:12:53.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.155+0000 7f3316729700 1 -- start start 2023-07-06T20:12:53.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3316729700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33100febd0 0x7f33100695a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:12:53.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3316729700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33100ffe70 0x7f331019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:12:53.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3316729700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3310100820 0x7f331019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:12:53.161 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3316729700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331019d590 con 0x7f33100febd0 2023-07-06T20:12:53.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3316729700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331019d700 con 0x7f33100ffe70 2023-07-06T20:12:53.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3316729700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331019d8c0 con 0x7f3310100820 2023-07-06T20:12:53.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3314cc6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3310100820 0x7f331019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:12:53.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3314cc6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3310100820 0x7f331019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59712/0 (socket says 172.21.15.156:59712) 2023-07-06T20:12:53.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3314cc6700 1 -- 172.21.15.156:0/4251111983 learned_addr learned my addr 172.21.15.156:0/4251111983 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-06T20:12:53.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f330f7fe700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33100ffe70 0x7f331019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:12:53.162 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3314cc6700 1 -- 172.21.15.156:0/4251111983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33100ffe70 msgr2=0x7f331019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:53.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3314cc6700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33100ffe70 0x7f331019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:53.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3314cc6700 1 -- 172.21.15.156:0/4251111983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33100febd0 msgr2=0x7f33100695a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:12:53.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3314cc6700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33100febd0 0x7f33100695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:12:53.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3314cc6700 1 -- 172.21.15.156:0/4251111983 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33040095f0 con 0x7f3310100820 2023-07-06T20:12:53.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3314cc6700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3310100820 0x7f331019cdf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f330800ada0 tx=0x7f330800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:12:53.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f330d7fa700 1 -- 172.21.15.156:0/4251111983 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330800f070 con 0x7f3310100820 2023-07-06T20:12:53.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f330d7fa700 1 -- 172.21.15.156:0/4251111983 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3308007b30 con 0x7f3310100820 2023-07-06T20:12:53.163 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f3316729700 1 -- 172.21.15.156:0/4251111983 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331019dba0 con 0x7f3310100820 2023-07-06T20:12:53.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.156+0000 7f330d7fa700 1 -- 172.21.15.156:0/4251111983 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308005440 con 0x7f3310100820 2023-07-06T20:12:53.164 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.157+0000 7f3316729700 1 -- 172.21.15.156:0/4251111983 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331019e0a0 con 0x7f3310100820 2023-07-06T20:12:53.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.158+0000 7f3316729700 1 -- 172.21.15.156:0/4251111983 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3310106200 con 0x7f3310100820 2023-07-06T20:12:53.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.159+0000 7f330d7fa700 1 -- 172.21.15.156:0/4251111983 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f3308004ad0 con 0x7f3310100820 2023-07-06T20:12:53.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.159+0000 7f330d7fa700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f32f80a1140 0x7f32f80a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:12:53.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.159+0000 7f330d7fa700 1 -- 172.21.15.156:0/4251111983 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f33080ca690 con 0x7f3310100820 2023-07-06T20:12:53.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.159+0000 7f330ffff700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f32f80a1140 0x7f32f80a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:12:53.165 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.160+0000 7f330ffff700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f32f80a1140 0x7f32f80a35f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f33040095c0 tx=0x7f3304009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:12:53.166 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.164+0000 7f330d7fa700 1 -- 172.21.15.156:0/4251111983 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f33080956c0 con 0x7f3310100820 2023-07-06T20:12:53.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:52 smithi069 ceph-mon[126399]: purged_snaps scrub starts 2023-07-06T20:12:53.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:52 smithi069 ceph-mon[126399]: purged_snaps scrub ok 2023-07-06T20:12:53.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:52 smithi069 ceph-mon[126399]: Detected new or changed devices on smithi098 2023-07-06T20:12:53.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:52 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi098 to 17168M 2023-07-06T20:12:53.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:52 smithi069 ceph-mon[126399]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:53.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:52 smithi069 ceph-mon[126399]: osd.1 [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125] boot 2023-07-06T20:12:53.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:52 smithi069 ceph-mon[126399]: osdmap e13: 2 total, 2 up, 2 in 2023-07-06T20:12:53.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:53.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:52 smithi156 ceph-mon[129280]: purged_snaps scrub starts 2023-07-06T20:12:53.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:52 smithi156 ceph-mon[129280]: purged_snaps scrub ok 2023-07-06T20:12:53.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:52 smithi156 ceph-mon[129280]: Detected new or changed devices on smithi098 2023-07-06T20:12:53.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:52 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi098 to 17168M 2023-07-06T20:12:53.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:52 smithi156 ceph-mon[129280]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:53.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:52 smithi156 ceph-mon[129280]: osd.1 [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125] boot 2023-07-06T20:12:53.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:52 smithi156 ceph-mon[129280]: osdmap e13: 2 total, 2 up, 2 in 2023-07-06T20:12:53.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:53.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:12:53.358+0000 7f3316729700 1 -- 172.21.15.156:0/4251111983 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3310060a80 con 0x7f32f80a1140 2023-07-06T20:12:53.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:52 smithi098 ceph-mon[128968]: purged_snaps scrub starts 2023-07-06T20:12:53.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:52 smithi098 ceph-mon[128968]: purged_snaps scrub ok 2023-07-06T20:12:53.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:52 smithi098 ceph-mon[128968]: Detected new or changed devices on smithi098 2023-07-06T20:12:53.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:52 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi098 to 17168M 2023-07-06T20:12:53.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:52 smithi098 ceph-mon[128968]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-06T20:12:53.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:52 smithi098 ceph-mon[128968]: osd.1 [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125] boot 2023-07-06T20:12:53.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:52 smithi098 ceph-mon[128968]: osdmap e13: 2 total, 2 up, 2 in 2023-07-06T20:12:53.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:52 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-06T20:12:54.204 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:12:54.204 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:12:54.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:54.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:53 smithi156 ceph-mon[129280]: osdmap e14: 2 total, 2 up, 2 in 2023-07-06T20:12:54.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:12:54.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:12:54.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:54.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:53 smithi069 ceph-mon[126399]: osdmap e14: 2 total, 2 up, 2 in 2023-07-06T20:12:54.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:12:54.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:12:54.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:12:54.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:53 smithi098 ceph-mon[128968]: osdmap e14: 2 total, 2 up, 2 in 2023-07-06T20:12:55.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:54 smithi069 ceph-mon[126399]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:12:55.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:54 smithi069 ceph-mon[126399]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:12:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:54 smithi156 ceph-mon[129280]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:12:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:54 smithi156 ceph-mon[129280]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:12:55.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:54 smithi098 ceph-mon[128968]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:12:55.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:54 smithi098 ceph-mon[128968]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:12:56.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:55 smithi069 ceph-mon[126399]: from='client.? 172.21.15.156:0/2685836626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dea6f798-761a-460a-a87b-f7b2bc6e28a6"}]: dispatch 2023-07-06T20:12:56.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:55 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dea6f798-761a-460a-a87b-f7b2bc6e28a6"}]: dispatch 2023-07-06T20:12:56.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:55 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dea6f798-761a-460a-a87b-f7b2bc6e28a6"}]': finished 2023-07-06T20:12:56.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:55 smithi069 ceph-mon[126399]: osdmap e15: 3 total, 2 up, 3 in 2023-07-06T20:12:56.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:55 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:12:56.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:55 smithi156 ceph-mon[129280]: from='client.? 172.21.15.156:0/2685836626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dea6f798-761a-460a-a87b-f7b2bc6e28a6"}]: dispatch 2023-07-06T20:12:56.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:55 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dea6f798-761a-460a-a87b-f7b2bc6e28a6"}]: dispatch 2023-07-06T20:12:56.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:55 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dea6f798-761a-460a-a87b-f7b2bc6e28a6"}]': finished 2023-07-06T20:12:56.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:55 smithi156 ceph-mon[129280]: osdmap e15: 3 total, 2 up, 3 in 2023-07-06T20:12:56.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:55 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:12:56.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:55 smithi098 ceph-mon[128968]: from='client.? 172.21.15.156:0/2685836626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dea6f798-761a-460a-a87b-f7b2bc6e28a6"}]: dispatch 2023-07-06T20:12:56.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:55 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dea6f798-761a-460a-a87b-f7b2bc6e28a6"}]: dispatch 2023-07-06T20:12:56.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:55 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dea6f798-761a-460a-a87b-f7b2bc6e28a6"}]': finished 2023-07-06T20:12:56.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:55 smithi098 ceph-mon[128968]: osdmap e15: 3 total, 2 up, 3 in 2023-07-06T20:12:56.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:55 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:12:57.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:56 smithi069 ceph-mon[126399]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:12:57.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:56 smithi069 ceph-mon[126399]: from='client.? 172.21.15.156:0/608034368' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:12:57.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:56 smithi156 ceph-mon[129280]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:12:57.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:56 smithi156 ceph-mon[129280]: from='client.? 172.21.15.156:0/608034368' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:12:57.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:56 smithi098 ceph-mon[128968]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:12:57.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:56 smithi098 ceph-mon[128968]: from='client.? 172.21.15.156:0/608034368' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:12:58.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:12:58 smithi156 ceph-mon[129280]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:12:58.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:12:58 smithi098 ceph-mon[128968]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:12:58.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:12:58 smithi069 ceph-mon[126399]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:00.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:00 smithi156 ceph-mon[129280]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:00.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:00 smithi098 ceph-mon[128968]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:00 smithi069 ceph-mon[126399]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:02.848 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:02 smithi156 ceph-mon[129280]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:02.848 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-06T20:13:02.848 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:02.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:02 smithi098 ceph-mon[128968]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:02.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-06T20:13:02.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:02 smithi069 ceph-mon[126399]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-06T20:13:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:03.526 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:03 smithi156 ceph-mon[129280]: Deploying daemon osd.2 on smithi156 2023-07-06T20:13:03.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:03 smithi098 ceph-mon[128968]: Deploying daemon osd.2 on smithi156 2023-07-06T20:13:04.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:03 smithi069 ceph-mon[126399]: Deploying daemon osd.2 on smithi156 2023-07-06T20:13:04.811 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:04 smithi156 ceph-mon[129280]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:04.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:04 smithi098 ceph-mon[128968]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:05.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:04 smithi069 ceph-mon[126399]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:06.701 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:06 smithi156 ceph-mon[129280]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:06.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:06 smithi098 ceph-mon[128968]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:07.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:06 smithi069 ceph-mon[126399]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:08.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:08.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:08.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:08.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:08.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:08.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:08.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:09.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:08 smithi069 ceph-mon[126399]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:09.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:09.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:09.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:09.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:09.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:08 smithi156 ceph-mon[129280]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:09.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:09.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:09.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:09.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:09.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:08 smithi098 ceph-mon[128968]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:09.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:09.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:09.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:09.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:09.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.771+0000 7f330d7fa700 1 -- 172.21.15.156:0/4251111983 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3310060a80 con 0x7f32f80a1140 2023-07-06T20:13:09.773 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 2 on host 'smithi156' 2023-07-06T20:13:09.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.773+0000 7f3316729700 1 -- 172.21.15.156:0/4251111983 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f32f80a1140 msgr2=0x7f32f80a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:09.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.773+0000 7f3316729700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f32f80a1140 0x7f32f80a35f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f33040095c0 tx=0x7f3304009530 comp rx=0 tx=0).stop 2023-07-06T20:13:09.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.773+0000 7f3316729700 1 -- 172.21.15.156:0/4251111983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3310100820 msgr2=0x7f331019cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:09.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.773+0000 7f3316729700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3310100820 0x7f331019cdf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f330800ada0 tx=0x7f330800c330 comp rx=0 tx=0).stop 2023-07-06T20:13:09.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.773+0000 7f3316729700 1 -- 172.21.15.156:0/4251111983 shutdown_connections 2023-07-06T20:13:09.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.773+0000 7f3316729700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f32f80a1140 0x7f32f80a35f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:09.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.773+0000 7f3316729700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3310100820 0x7f331019cdf0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:09.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.773+0000 7f3316729700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33100febd0 0x7f33100695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:09.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.773+0000 7f3316729700 1 --2- 172.21.15.156:0/4251111983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33100ffe70 0x7f331019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:09.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.773+0000 7f3316729700 1 -- 172.21.15.156:0/4251111983 >> 172.21.15.156:0/4251111983 conn(0x7f33100fa240 msgr2=0x7f3310103340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:09.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.773+0000 7f3316729700 1 -- 172.21.15.156:0/4251111983 shutdown_connections 2023-07-06T20:13:09.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:13:09.773+0000 7f3316729700 1 -- 172.21.15.156:0/4251111983 wait complete. 2023-07-06T20:13:10.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:09 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:10.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:09 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:10.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:09 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:10.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:09 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:10.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:09 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:10.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:09 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:10.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:09 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:10.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:09 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:10.278 DEBUG:teuthology.orchestra.run.smithi156:osd.2> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.2.service 2023-07-06T20:13:10.280 INFO:tasks.cephadm:Deploying osd.3 on smithi069 with /dev/vg_nvme/lv_3... 2023-07-06T20:13:10.281 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-06T20:13:10.309 INFO:journalctl@ceph.osd.2.smithi156.stdout:-- Logs begin at Thu 2023-07-06 19:42:21 UTC. -- 2023-07-06T20:13:10.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:09 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:10.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:09 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:10.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:09 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:10.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:09 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:11.006 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:13:11.065 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:10 smithi156 ceph-mon[129280]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:11.066 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:10 smithi156 ceph-mon[129280]: from='osd.2 [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-06T20:13:11.066 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:10 smithi156 ceph-mon[129280]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-06T20:13:11.066 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:13:10 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-2[135046]: 2023-07-06T20:13:10.811+0000 7fd054903640 -1 osd.2 0 log_to_monitors true 2023-07-06T20:13:11.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:10 smithi069 ceph-mon[126399]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:11.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:10 smithi069 ceph-mon[126399]: from='osd.2 [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-06T20:13:11.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:10 smithi069 ceph-mon[126399]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-06T20:13:11.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:10 smithi098 ceph-mon[128968]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:11.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:10 smithi098 ceph-mon[128968]: from='osd.2 [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-06T20:13:11.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:10 smithi098 ceph-mon[128968]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-06T20:13:12.243 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:11 smithi069 ceph-mon[126399]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-06T20:13:12.244 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:11 smithi069 ceph-mon[126399]: osdmap e16: 3 total, 2 up, 3 in 2023-07-06T20:13:12.244 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:12.244 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:11 smithi069 ceph-mon[126399]: from='osd.2 [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:13:12.244 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:11 smithi069 ceph-mon[126399]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:13:12.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:11 smithi156 ceph-mon[129280]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-06T20:13:12.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:11 smithi156 ceph-mon[129280]: osdmap e16: 3 total, 2 up, 3 in 2023-07-06T20:13:12.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:12.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:11 smithi156 ceph-mon[129280]: from='osd.2 [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:13:12.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:11 smithi156 ceph-mon[129280]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:13:12.357 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:13:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-2[135046]: 2023-07-06T20:13:11.949+0000 7fd04977a700 -1 osd.2 0 waiting for initial osdmap 2023-07-06T20:13:12.357 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:13:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-2[135046]: 2023-07-06T20:13:11.953+0000 7fd041d68700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:13:12.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:11 smithi098 ceph-mon[128968]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-06T20:13:12.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:11 smithi098 ceph-mon[128968]: osdmap e16: 3 total, 2 up, 3 in 2023-07-06T20:13:12.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:12.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:11 smithi098 ceph-mon[128968]: from='osd.2 [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:13:12.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:11 smithi098 ceph-mon[128968]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:13:12.968 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-06T20:13:12.968 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-06T20:13:12.969 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2023-07-06T20:13:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2023-07-06T20:13:12.969 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0408554 s, 257 MB/s 2023-07-06T20:13:12.969 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2023-07-06T20:13:13.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:12 smithi069 ceph-mon[126399]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:13.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:12 smithi069 ceph-mon[126399]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-06T20:13:13.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:12 smithi069 ceph-mon[126399]: osdmap e17: 3 total, 2 up, 3 in 2023-07-06T20:13:13.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:13.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:13.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:13.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:13.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:13:13.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:13.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:13.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:13.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:13.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:12 smithi156 ceph-mon[129280]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:13.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:12 smithi156 ceph-mon[129280]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-06T20:13:13.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:12 smithi156 ceph-mon[129280]: osdmap e17: 3 total, 2 up, 3 in 2023-07-06T20:13:13.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:13.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:13.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:13.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:13.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:13:13.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:13.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:13.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:13.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:13.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:12 smithi098 ceph-mon[128968]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:13.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:12 smithi098 ceph-mon[128968]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-06T20:13:13.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:12 smithi098 ceph-mon[128968]: osdmap e17: 3 total, 2 up, 3 in 2023-07-06T20:13:13.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:13.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:13.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:13.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:13.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:13:13.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:13.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:13.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:13.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:13.382 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch daemon add osd smithi069:vg_nvme/lv_3 2023-07-06T20:13:14.102 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:13:14.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:14 smithi156 ceph-mon[129280]: Detected new or changed devices on smithi156 2023-07-06T20:13:14.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:14 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi156 to 17168M 2023-07-06T20:13:14.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:14 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:14.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:14 smithi156 ceph-mon[129280]: osd.2 [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946] boot 2023-07-06T20:13:14.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:14 smithi156 ceph-mon[129280]: osdmap e18: 3 total, 3 up, 3 in 2023-07-06T20:13:14.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:14 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:14.364 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:14 smithi069 ceph-mon[126399]: Detected new or changed devices on smithi156 2023-07-06T20:13:14.365 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:14 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi156 to 17168M 2023-07-06T20:13:14.365 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:14.365 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:14 smithi069 ceph-mon[126399]: osd.2 [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946] boot 2023-07-06T20:13:14.365 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:14 smithi069 ceph-mon[126399]: osdmap e18: 3 total, 3 up, 3 in 2023-07-06T20:13:14.365 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:14.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:14 smithi098 ceph-mon[128968]: Detected new or changed devices on smithi156 2023-07-06T20:13:14.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:14 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi156 to 17168M 2023-07-06T20:13:14.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:14 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:14.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:14 smithi098 ceph-mon[128968]: osd.2 [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946] boot 2023-07-06T20:13:14.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:14 smithi098 ceph-mon[128968]: osdmap e18: 3 total, 3 up, 3 in 2023-07-06T20:13:14.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:14 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-06T20:13:15.241 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:15 smithi069 ceph-mon[126399]: purged_snaps scrub starts 2023-07-06T20:13:15.242 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:15 smithi069 ceph-mon[126399]: purged_snaps scrub ok 2023-07-06T20:13:15.242 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:15 smithi069 ceph-mon[126399]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:15.242 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:15 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-06T20:13:15.242 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:15 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-06T20:13:15.242 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:15 smithi069 ceph-mon[126399]: osdmap e19: 3 total, 3 up, 3 in 2023-07-06T20:13:15.242 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:15 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-06T20:13:15.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:15 smithi156 ceph-mon[129280]: purged_snaps scrub starts 2023-07-06T20:13:15.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:15 smithi156 ceph-mon[129280]: purged_snaps scrub ok 2023-07-06T20:13:15.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:15 smithi156 ceph-mon[129280]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:15.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:15 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-06T20:13:15.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:15 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-06T20:13:15.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:15 smithi156 ceph-mon[129280]: osdmap e19: 3 total, 3 up, 3 in 2023-07-06T20:13:15.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:15 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-06T20:13:15.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:15 smithi098 ceph-mon[128968]: purged_snaps scrub starts 2023-07-06T20:13:15.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:15 smithi098 ceph-mon[128968]: purged_snaps scrub ok 2023-07-06T20:13:15.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:15 smithi098 ceph-mon[128968]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-06T20:13:15.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:15 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-06T20:13:15.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:15 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-06T20:13:15.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:15 smithi098 ceph-mon[128968]: osdmap e19: 3 total, 3 up, 3 in 2023-07-06T20:13:15.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:15 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-06T20:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.496+0000 7fa069669700 1 -- 172.21.15.69:0/3974887868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa064074670 msgr2=0x7fa064072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.496+0000 7fa069669700 1 --2- 172.21.15.69:0/3974887868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa064074670 0x7fa064072ad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa054009990 tx=0x7fa054009ca0 comp rx=0 tx=0).stop 2023-07-06T20:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.496+0000 7fa069669700 1 -- 172.21.15.69:0/3974887868 shutdown_connections 2023-07-06T20:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.496+0000 7fa069669700 1 --2- 172.21.15.69:0/3974887868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa064107b90 0x7fa064109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.496+0000 7fa069669700 1 --2- 172.21.15.69:0/3974887868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa064074670 0x7fa064072ad0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.496+0000 7fa069669700 1 --2- 172.21.15.69:0/3974887868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0640730a0 0x7fa064073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.496+0000 7fa069669700 1 -- 172.21.15.69:0/3974887868 >> 172.21.15.69:0/3974887868 conn(0x7fa0640fb2d0 msgr2=0x7fa0640fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.496+0000 7fa069669700 1 -- 172.21.15.69:0/3974887868 shutdown_connections 2023-07-06T20:13:15.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.496+0000 7fa069669700 1 -- 172.21.15.69:0/3974887868 wait complete. 2023-07-06T20:13:15.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.497+0000 7fa069669700 1 Processor -- start 2023-07-06T20:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.498+0000 7fa069669700 1 -- start start 2023-07-06T20:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.498+0000 7fa069669700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0640730a0 0x7fa064197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.498+0000 7fa069669700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa064074670 0x7fa064198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.498+0000 7fa069669700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa064107b90 0x7fa06419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.498+0000 7fa069669700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06419f770 con 0x7fa064107b90 2023-07-06T20:13:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.498+0000 7fa069669700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06419f8b0 con 0x7fa0640730a0 2023-07-06T20:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.498+0000 7fa069669700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06419fa70 con 0x7fa064074670 2023-07-06T20:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.498+0000 7fa062ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0640730a0 0x7fa064197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.498+0000 7fa0637fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa064107b90 0x7fa06419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.498+0000 7fa062ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0640730a0 0x7fa064197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:52220/0 (socket says 172.21.15.69:52220) 2023-07-06T20:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.498+0000 7fa062ffd700 1 -- 172.21.15.69:0/1845129634 learned_addr learned my addr 172.21.15.69:0/1845129634 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.498+0000 7fa0627fc700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa064074670 0x7fa064198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.499+0000 7fa0627fc700 1 -- 172.21.15.69:0/1845129634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0640730a0 msgr2=0x7fa064197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:15.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.499+0000 7fa0627fc700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0640730a0 0x7fa064197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:15.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.499+0000 7fa0627fc700 1 -- 172.21.15.69:0/1845129634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa064107b90 msgr2=0x7fa06419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:15.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.499+0000 7fa0627fc700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa064107b90 0x7fa06419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:15.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.499+0000 7fa0627fc700 1 -- 172.21.15.69:0/1845129634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0540095f0 con 0x7fa064074670 2023-07-06T20:13:15.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.499+0000 7fa062ffd700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0640730a0 0x7fa064197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:13:15.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.499+0000 7fa0627fc700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa064074670 0x7fa064198160 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa05800e9f0 tx=0x7fa05800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:15.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.499+0000 7fa04bfff700 1 -- 172.21.15.69:0/1845129634 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05800c9c0 con 0x7fa064074670 2023-07-06T20:13:15.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.499+0000 7fa069669700 1 -- 172.21.15.69:0/1845129634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06419fd50 con 0x7fa064074670 2023-07-06T20:13:15.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.499+0000 7fa04bfff700 1 -- 172.21.15.69:0/1845129634 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa058010430 con 0x7fa064074670 2023-07-06T20:13:15.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.499+0000 7fa04bfff700 1 -- 172.21.15.69:0/1845129634 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058018680 con 0x7fa064074670 2023-07-06T20:13:15.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.499+0000 7fa069669700 1 -- 172.21.15.69:0/1845129634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0641a0190 con 0x7fa064074670 2023-07-06T20:13:15.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.500+0000 7fa069669700 1 -- 172.21.15.69:0/1845129634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa064106240 con 0x7fa064074670 2023-07-06T20:13:15.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.502+0000 7fa04bfff700 1 -- 172.21.15.69:0/1845129634 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fa058010860 con 0x7fa064074670 2023-07-06T20:13:15.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.504+0000 7fa04bfff700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa04c0a10c0 0x7fa04c0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:15.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.504+0000 7fa04bfff700 1 -- 172.21.15.69:0/1845129634 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7fa058014070 con 0x7fa064074670 2023-07-06T20:13:15.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.504+0000 7fa062ffd700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa04c0a10c0 0x7fa04c0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:15.507 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.504+0000 7fa062ffd700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa04c0a10c0 0x7fa04c0a3570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa064199100 tx=0x7fa05400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:15.507 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.506+0000 7fa04bfff700 1 -- 172.21.15.69:0/1845129634 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa058096310 con 0x7fa064074670 2023-07-06T20:13:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:15.709+0000 7fa069669700 1 -- 172.21.15.69:0/1845129634 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa064060a80 con 0x7fa04c0a10c0 2023-07-06T20:13:16.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:16.089+0000 7fa04bfff700 1 -- 172.21.15.69:0/1845129634 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa0580966e0 con 0x7fa064074670 2023-07-06T20:13:16.258 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:13:15 smithi069 sudo[142534]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-06T20:13:16.259 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:13:15 smithi069 sudo[142534]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-06T20:13:16.259 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:13:15 smithi069 sudo[142534]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-06T20:13:16.259 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:13:15 smithi069 sudo[142534]: pam_unix(sudo:session): session closed for user root 2023-07-06T20:13:16.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-06T20:13:16.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:16 smithi069 ceph-mon[126399]: osdmap e20: 3 total, 3 up, 3 in 2023-07-06T20:13:16.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:16 smithi069 ceph-mon[126399]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:16.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:16 smithi069 ceph-mon[126399]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:13:16.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:13:16.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:13:16.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:16.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-06T20:13:16.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:16 smithi156 ceph-mon[129280]: osdmap e20: 3 total, 3 up, 3 in 2023-07-06T20:13:16.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:16 smithi156 ceph-mon[129280]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:16.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:16 smithi156 ceph-mon[129280]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:13:16.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:13:16.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:13:16.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:16.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-06T20:13:16.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:16 smithi098 ceph-mon[128968]: osdmap e20: 3 total, 3 up, 3 in 2023-07-06T20:13:16.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:16 smithi098 ceph-mon[128968]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:16.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:16 smithi098 ceph-mon[128968]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:13:16.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:13:16.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:13:16.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:16.758 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:13:16 smithi069 sudo[142581]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-06T20:13:16.759 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:13:16 smithi069 sudo[142581]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-06T20:13:16.759 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:13:16 smithi069 sudo[142581]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-06T20:13:16.759 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:13:16 smithi069 sudo[142581]: pam_unix(sudo:session): session closed for user root 2023-07-06T20:13:17.032 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:13:16 smithi098 sudo[136389]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-06T20:13:17.033 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:13:16 smithi098 sudo[136389]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-06T20:13:17.033 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:13:16 smithi098 sudo[136389]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-06T20:13:17.033 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:13:16 smithi098 sudo[136389]: pam_unix(sudo:session): session closed for user root 2023-07-06T20:13:17.334 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:17 smithi069 ceph-mon[126399]: osdmap e21: 3 total, 3 up, 3 in 2023-07-06T20:13:17.334 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:17 smithi069 ceph-mon[126399]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-06T20:13:17.334 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:17 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:17.334 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:17 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:17.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:17 smithi156 ceph-mon[129280]: osdmap e21: 3 total, 3 up, 3 in 2023-07-06T20:13:17.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:17 smithi156 ceph-mon[129280]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-06T20:13:17.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:17 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:17.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:17 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:17.357 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:13:17 smithi156 sudo[137311]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-06T20:13:17.357 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:13:17 smithi156 sudo[137311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-06T20:13:17.357 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:13:17 smithi156 sudo[137311]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-06T20:13:17.357 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:13:17 smithi156 sudo[137311]: pam_unix(sudo:session): session closed for user root 2023-07-06T20:13:17.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:17 smithi098 ceph-mon[128968]: osdmap e21: 3 total, 3 up, 3 in 2023-07-06T20:13:17.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:17 smithi098 ceph-mon[128968]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-06T20:13:17.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:17 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:17.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:17 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:17.376 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:13:17 smithi098 sudo[136392]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-06T20:13:17.376 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:13:17 smithi098 sudo[136392]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-06T20:13:17.376 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:13:17 smithi098 sudo[136392]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-06T20:13:17.376 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:13:17 smithi098 sudo[136392]: pam_unix(sudo:session): session closed for user root 2023-07-06T20:13:17.856 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:13:17 smithi156 sudo[137314]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-06T20:13:17.857 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:13:17 smithi156 sudo[137314]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-06T20:13:17.857 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:13:17 smithi156 sudo[137314]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-06T20:13:17.857 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:13:17 smithi156 sudo[137314]: pam_unix(sudo:session): session closed for user root 2023-07-06T20:13:18.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:18 smithi069 sudo[142704]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-06T20:13:18.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:18 smithi069 sudo[142704]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-06T20:13:18.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:18 smithi069 sudo[142704]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-06T20:13:18.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:18 smithi069 sudo[142704]: pam_unix(sudo:session): session closed for user root 2023-07-06T20:13:18.944 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:18 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:18.945 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:18 smithi156 ceph-mon[129280]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-06T20:13:18.945 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:18 smithi156 sudo[137317]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-06T20:13:18.945 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:18 smithi156 sudo[137317]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-06T20:13:18.945 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:18 smithi156 sudo[137317]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-06T20:13:18.945 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:18 smithi156 sudo[137317]: pam_unix(sudo:session): session closed for user root 2023-07-06T20:13:19.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:18 smithi069 sudo[142737]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-06T20:13:19.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:18 smithi069 sudo[142737]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-06T20:13:19.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:18 smithi069 sudo[142737]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-06T20:13:19.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:18 smithi069 sudo[142737]: pam_unix(sudo:session): session closed for user root 2023-07-06T20:13:19.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:18 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:19.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:18 smithi069 ceph-mon[126399]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-06T20:13:19.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:18 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:19.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:18 smithi098 ceph-mon[128968]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-06T20:13:19.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 sudo[137320]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-06T20:13:19.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 sudo[137320]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-06T20:13:19.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 sudo[137320]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-06T20:13:19.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 sudo[137320]: pam_unix(sudo:session): session closed for user root 2023-07-06T20:13:19.682 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 sudo[136395]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-06T20:13:19.682 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 sudo[136395]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-06T20:13:19.682 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 sudo[136395]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-06T20:13:19.682 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 sudo[136395]: pam_unix(sudo:session): session closed for user root 2023-07-06T20:13:20.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:20.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1473418233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cca1e09-50d7-4a90-bc9e-a62e092c5d55"}]: dispatch 2023-07-06T20:13:20.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-06T20:13:20.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:13:20.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:13:20.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cca1e09-50d7-4a90-bc9e-a62e092c5d55"}]: dispatch 2023-07-06T20:13:20.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:13:20.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-06T20:13:20.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:13:20.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:13:20.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:13:20.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-06T20:13:20.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cca1e09-50d7-4a90-bc9e-a62e092c5d55"}]': finished 2023-07-06T20:13:20.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: osdmap e22: 4 total, 3 up, 4 in 2023-07-06T20:13:20.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:20.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-06T20:13:20.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:13:20.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:13:20.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:13:20.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:19 smithi069 ceph-mon[126399]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-06T20:13:20.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:20.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1473418233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cca1e09-50d7-4a90-bc9e-a62e092c5d55"}]: dispatch 2023-07-06T20:13:20.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-06T20:13:20.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:13:20.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:13:20.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cca1e09-50d7-4a90-bc9e-a62e092c5d55"}]: dispatch 2023-07-06T20:13:20.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:13:20.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-06T20:13:20.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:13:20.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:13:20.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:13:20.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-06T20:13:20.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cca1e09-50d7-4a90-bc9e-a62e092c5d55"}]': finished 2023-07-06T20:13:20.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: osdmap e22: 4 total, 3 up, 4 in 2023-07-06T20:13:20.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:20.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-06T20:13:20.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:13:20.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:13:20.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:13:20.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:19 smithi156 ceph-mon[129280]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-06T20:13:20.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 sudo[136398]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-06T20:13:20.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 sudo[136398]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-06T20:13:20.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 sudo[136398]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-06T20:13:20.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 sudo[136398]: pam_unix(sudo:session): session closed for user root 2023-07-06T20:13:20.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:20.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1473418233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cca1e09-50d7-4a90-bc9e-a62e092c5d55"}]: dispatch 2023-07-06T20:13:20.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-06T20:13:20.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:13:20.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:13:20.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cca1e09-50d7-4a90-bc9e-a62e092c5d55"}]: dispatch 2023-07-06T20:13:20.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:13:20.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-06T20:13:20.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:13:20.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:13:20.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:13:20.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-06T20:13:20.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cca1e09-50d7-4a90-bc9e-a62e092c5d55"}]': finished 2023-07-06T20:13:20.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: osdmap e22: 4 total, 3 up, 4 in 2023-07-06T20:13:20.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:20.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-06T20:13:20.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-06T20:13:20.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-06T20:13:20.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-06T20:13:20.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:19 smithi098 ceph-mon[128968]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-06T20:13:21.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:20 smithi069 ceph-mon[126399]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:21.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:20 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/296074775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:13:21.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:20 smithi156 ceph-mon[129280]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:21.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:20 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/296074775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:13:21.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:20 smithi098 ceph-mon[128968]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:21.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:20 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/296074775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:13:23.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:22 smithi069 ceph-mon[126399]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:23.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:22 smithi156 ceph-mon[129280]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:23.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:22 smithi098 ceph-mon[128968]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:25.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:24 smithi069 ceph-mon[126399]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 61 KiB/s wr, 5 op/s 2023-07-06T20:13:25.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:24 smithi156 ceph-mon[129280]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 61 KiB/s wr, 5 op/s 2023-07-06T20:13:25.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:24 smithi098 ceph-mon[128968]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 61 KiB/s wr, 5 op/s 2023-07-06T20:13:27.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:26 smithi069 ceph-mon[126399]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 42 KiB/s wr, 3 op/s 2023-07-06T20:13:27.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:26 smithi156 ceph-mon[129280]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 42 KiB/s wr, 3 op/s 2023-07-06T20:13:27.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:26 smithi098 ceph-mon[128968]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 42 KiB/s wr, 3 op/s 2023-07-06T20:13:27.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-06T20:13:27.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:27.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:27 smithi069 ceph-mon[126399]: Deploying daemon osd.3 on smithi069 2023-07-06T20:13:28.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-06T20:13:28.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:28.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:27 smithi156 ceph-mon[129280]: Deploying daemon osd.3 on smithi069 2023-07-06T20:13:28.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-06T20:13:28.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:28.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:27 smithi098 ceph-mon[128968]: Deploying daemon osd.3 on smithi069 2023-07-06T20:13:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:28 smithi069 ceph-mon[126399]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 32 KiB/s wr, 2 op/s 2023-07-06T20:13:29.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:28 smithi156 ceph-mon[129280]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 32 KiB/s wr, 2 op/s 2023-07-06T20:13:29.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:28 smithi098 ceph-mon[128968]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 32 KiB/s wr, 2 op/s 2023-07-06T20:13:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:30 smithi069 ceph-mon[126399]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 896 B/s rd, 25 KiB/s wr, 2 op/s 2023-07-06T20:13:31.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:30 smithi156 ceph-mon[129280]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 896 B/s rd, 25 KiB/s wr, 2 op/s 2023-07-06T20:13:31.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:30 smithi098 ceph-mon[128968]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 896 B/s rd, 25 KiB/s wr, 2 op/s 2023-07-06T20:13:33.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:32 smithi069 ceph-mon[126399]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-06T20:13:33.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:32 smithi156 ceph-mon[129280]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-06T20:13:33.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:32 smithi098 ceph-mon[128968]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-06T20:13:34.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:33 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:34.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:33 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:34.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:33 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:34.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:33 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:34.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:33 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:34.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:33 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:34.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:33 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:34.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:33 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:34.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:33 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:34 smithi069 ceph-mon[126399]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-06T20:13:35.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:34 smithi156 ceph-mon[129280]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-06T20:13:35.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:34 smithi098 ceph-mon[128968]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-06T20:13:36.371 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 3 on host 'smithi069' 2023-07-06T20:13:36.371 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.368+0000 7fa04bfff700 1 -- 172.21.15.69:0/1845129634 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa064060a80 con 0x7fa04c0a10c0 2023-07-06T20:13:36.373 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.371+0000 7fa069669700 1 -- 172.21.15.69:0/1845129634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa04c0a10c0 msgr2=0x7fa04c0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:36.373 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.371+0000 7fa069669700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa04c0a10c0 0x7fa04c0a3570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa064199100 tx=0x7fa05400b260 comp rx=0 tx=0).stop 2023-07-06T20:13:36.373 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.371+0000 7fa069669700 1 -- 172.21.15.69:0/1845129634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa064074670 msgr2=0x7fa064198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:36.373 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.371+0000 7fa069669700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa064074670 0x7fa064198160 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa05800e9f0 tx=0x7fa05800ed00 comp rx=0 tx=0).stop 2023-07-06T20:13:36.373 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.371+0000 7fa069669700 1 -- 172.21.15.69:0/1845129634 shutdown_connections 2023-07-06T20:13:36.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.371+0000 7fa069669700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa04c0a10c0 0x7fa04c0a3570 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:36.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.371+0000 7fa069669700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa064107b90 0x7fa06419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:36.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.371+0000 7fa069669700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0640730a0 0x7fa064197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:36.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.371+0000 7fa069669700 1 --2- 172.21.15.69:0/1845129634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa064074670 0x7fa064198160 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:36.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.371+0000 7fa069669700 1 -- 172.21.15.69:0/1845129634 >> 172.21.15.69:0/1845129634 conn(0x7fa0640fb2d0 msgr2=0x7fa0640fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:36.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.371+0000 7fa069669700 1 -- 172.21.15.69:0/1845129634 shutdown_connections 2023-07-06T20:13:36.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:13:36.371+0000 7fa069669700 1 -- 172.21.15.69:0/1845129634 wait complete. 2023-07-06T20:13:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:36 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:36 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:36 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:36 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:36 smithi069 ceph-mon[126399]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:36.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:36 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:36 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:36 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:36 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:36.836 DEBUG:teuthology.orchestra.run.smithi069:osd.3> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.3.service 2023-07-06T20:13:36.840 INFO:tasks.cephadm:Deploying osd.4 on smithi098 with /dev/vg_nvme/lv_3... 2023-07-06T20:13:36.841 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-06T20:13:36.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:36 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:36 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:36 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:36.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:36 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:36.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:36 smithi156 ceph-mon[129280]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:36.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:36 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:36 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:36 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:36 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:36.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:36 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:36 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:36 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:36.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:36 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:36.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:36 smithi098 ceph-mon[128968]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:36.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:36 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:36 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:36 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:36.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:36 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:36.935 INFO:journalctl@ceph.osd.3.smithi069.stdout:-- Logs begin at Thu 2023-07-06 19:46:03 UTC. -- 2023-07-06T20:13:37.411 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.b/config 2023-07-06T20:13:37.721 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:37 smithi098 ceph-mon[128968]: from='osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-06T20:13:37.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:37 smithi069 ceph-mon[126399]: from='osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-06T20:13:37.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:37 smithi156 ceph-mon[129280]: from='osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-06T20:13:38.759 INFO:journalctl@ceph.osd.3.smithi069.stdout:Jul 06 20:13:38 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-3[145011]: 2023-07-06T20:13:38.484+0000 7f96f9dd8700 -1 osd.3 0 waiting for initial osdmap 2023-07-06T20:13:38.759 INFO:journalctl@ceph.osd.3.smithi069.stdout:Jul 06 20:13:38 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-3[145011]: 2023-07-06T20:13:38.492+0000 7f96f43ca700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:13:38.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:38 smithi069 ceph-mon[126399]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:38.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:38 smithi069 ceph-mon[126399]: from='osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-06T20:13:38.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:38 smithi069 ceph-mon[126399]: osdmap e23: 4 total, 3 up, 4 in 2023-07-06T20:13:38.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:38 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:38.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:38 smithi069 ceph-mon[126399]: from='osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:13:38.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:38 smithi156 ceph-mon[129280]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:38.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:38 smithi156 ceph-mon[129280]: from='osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-06T20:13:38.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:38 smithi156 ceph-mon[129280]: osdmap e23: 4 total, 3 up, 4 in 2023-07-06T20:13:38.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:38.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:38 smithi156 ceph-mon[129280]: from='osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:13:38.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:38 smithi098 ceph-mon[128968]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:38.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:38 smithi098 ceph-mon[128968]: from='osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-06T20:13:38.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:38 smithi098 ceph-mon[128968]: osdmap e23: 4 total, 3 up, 4 in 2023-07-06T20:13:38.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:38 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:38.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:38 smithi098 ceph-mon[128968]: from='osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:13:39.000 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-06T20:13:39.000 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-06T20:13:39.000 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2023-07-06T20:13:39.000 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2023-07-06T20:13:39.001 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011006 s, 953 MB/s 2023-07-06T20:13:39.001 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 2023-07-06T20:13:39.001 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2023-07-06T20:13:39.452 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch daemon add osd smithi098:vg_nvme/lv_3 2023-07-06T20:13:39.714 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:39 smithi069 ceph-mon[126399]: from='osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-07-06T20:13:39.715 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:39 smithi069 ceph-mon[126399]: osdmap e24: 4 total, 3 up, 4 in 2023-07-06T20:13:39.715 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:39 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:39.715 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:39 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:39.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:39 smithi156 ceph-mon[129280]: from='osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-07-06T20:13:39.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:39 smithi156 ceph-mon[129280]: osdmap e24: 4 total, 3 up, 4 in 2023-07-06T20:13:39.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:39 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:39.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:39 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:39.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:39 smithi098 ceph-mon[128968]: from='osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-07-06T20:13:39.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:39 smithi098 ceph-mon[128968]: osdmap e24: 4 total, 3 up, 4 in 2023-07-06T20:13:39.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:39 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:39.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:39 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.b/config 2023-07-06T20:13:40.624 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: purged_snaps scrub starts 2023-07-06T20:13:40.624 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: purged_snaps scrub ok 2023-07-06T20:13:40.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:40.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:40.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200] boot 2023-07-06T20:13:40.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: osdmap e25: 4 total, 4 up, 4 in 2023-07-06T20:13:40.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:40.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: Detected new or changed devices on smithi069 2023-07-06T20:13:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:13:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:13:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi069 to 8633M 2023-07-06T20:13:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:40.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:40.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:40.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: purged_snaps scrub starts 2023-07-06T20:13:40.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: purged_snaps scrub ok 2023-07-06T20:13:40.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:40.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:40.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200] boot 2023-07-06T20:13:40.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: osdmap e25: 4 total, 4 up, 4 in 2023-07-06T20:13:40.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:40.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: Detected new or changed devices on smithi069 2023-07-06T20:13:40.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:40.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:40.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:13:40.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:13:40.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi069 to 8633M 2023-07-06T20:13:40.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:40.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:40.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:40.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:40.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: purged_snaps scrub starts 2023-07-06T20:13:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: purged_snaps scrub ok 2023-07-06T20:13:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-06T20:13:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: osd.3 [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200] boot 2023-07-06T20:13:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: osdmap e25: 4 total, 4 up, 4 in 2023-07-06T20:13:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-06T20:13:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: Detected new or changed devices on smithi069 2023-07-06T20:13:40.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:40.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:40.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:13:40.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:13:40.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi069 to 8633M 2023-07-06T20:13:40.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:40.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:40.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:40.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:41.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.336+0000 7f0368308700 1 -- 172.21.15.98:0/92191852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03600febb0 msgr2=0x7f03600fefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:41.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.336+0000 7f0368308700 1 --2- 172.21.15.98:0/92191852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03600febb0 0x7f03600fefc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f035800b0a0 tx=0x7f035800b3b0 comp rx=0 tx=0).stop 2023-07-06T20:13:41.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.336+0000 7f0368308700 1 -- 172.21.15.98:0/92191852 shutdown_connections 2023-07-06T20:13:41.339 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.336+0000 7f0368308700 1 --2- 172.21.15.98:0/92191852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0360100740 0x7f0360107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:41.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.336+0000 7f0368308700 1 --2- 172.21.15.98:0/92191852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03600febb0 0x7f03600fefc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:41.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.336+0000 7f0368308700 1 --2- 172.21.15.98:0/92191852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03600ffdb0 0x7f0360100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:41.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.336+0000 7f0368308700 1 -- 172.21.15.98:0/92191852 >> 172.21.15.98:0/92191852 conn(0x7f03600fa1e0 msgr2=0x7f03600fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:41.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.336+0000 7f0368308700 1 -- 172.21.15.98:0/92191852 shutdown_connections 2023-07-06T20:13:41.340 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.336+0000 7f0368308700 1 -- 172.21.15.98:0/92191852 wait complete. 2023-07-06T20:13:41.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f0368308700 1 Processor -- start 2023-07-06T20:13:41.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f0368308700 1 -- start start 2023-07-06T20:13:41.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f0368308700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03600febb0 0x7f0360068c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:41.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f0368308700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03600ffdb0 0x7f036019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:41.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f0368308700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0360100740 0x7f036019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:41.342 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f0368308700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036019d690 con 0x7f03600febb0 2023-07-06T20:13:41.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f0368308700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036019d800 con 0x7f0360100740 2023-07-06T20:13:41.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f0368308700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036019d9c0 con 0x7f03600ffdb0 2023-07-06T20:13:41.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f03668a5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0360100740 0x7f036019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:41.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f03668a5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0360100740 0x7f036019cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:44378/0 (socket says 172.21.15.98:44378) 2023-07-06T20:13:41.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f03668a5700 1 -- 172.21.15.98:0/2655185496 learned_addr learned my addr 172.21.15.98:0/2655185496 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-07-06T20:13:41.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f03658a3700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03600ffdb0 0x7f036019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:41.343 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.337+0000 7f03660a4700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03600febb0 0x7f0360068c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:41.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.338+0000 7f03668a5700 1 -- 172.21.15.98:0/2655185496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03600ffdb0 msgr2=0x7f036019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:41.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.338+0000 7f03668a5700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03600ffdb0 0x7f036019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:41.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.338+0000 7f03668a5700 1 -- 172.21.15.98:0/2655185496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03600febb0 msgr2=0x7f0360068c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:41.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.338+0000 7f03668a5700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03600febb0 0x7f0360068c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:41.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.338+0000 7f03668a5700 1 -- 172.21.15.98:0/2655185496 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0358009d00 con 0x7f0360100740 2023-07-06T20:13:41.344 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.338+0000 7f03668a5700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0360100740 0x7f036019cef0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f035000c900 tx=0x7f035000cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:41.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.338+0000 7f035f7fe700 1 -- 172.21.15.98:0/2655185496 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350007b00 con 0x7f0360100740 2023-07-06T20:13:41.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.338+0000 7f035f7fe700 1 -- 172.21.15.98:0/2655185496 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f035000ce50 con 0x7f0360100740 2023-07-06T20:13:41.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.338+0000 7f0368308700 1 -- 172.21.15.98:0/2655185496 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036019dca0 con 0x7f0360100740 2023-07-06T20:13:41.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.338+0000 7f035f7fe700 1 -- 172.21.15.98:0/2655185496 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350018660 con 0x7f0360100740 2023-07-06T20:13:41.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.338+0000 7f0368308700 1 -- 172.21.15.98:0/2655185496 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036019e1d0 con 0x7f0360100740 2023-07-06T20:13:41.345 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.339+0000 7f0368308700 1 -- 172.21.15.98:0/2655185496 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03601061b0 con 0x7f0360100740 2023-07-06T20:13:41.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.340+0000 7f035f7fe700 1 -- 172.21.15.98:0/2655185496 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f035000f450 con 0x7f0360100740 2023-07-06T20:13:41.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.341+0000 7f035f7fe700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f03480a1150 0x7f03480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:13:41.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.341+0000 7f035f7fe700 1 -- 172.21.15.98:0/2655185496 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f03500cb6d0 con 0x7f0360100740 2023-07-06T20:13:41.346 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.341+0000 7f03660a4700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f03480a1150 0x7f03480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:13:41.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.342+0000 7f03660a4700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f03480a1150 0x7f03480a3600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0358007ab0 tx=0x7f0358006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:13:41.347 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.345+0000 7f035f7fe700 1 -- 172.21.15.98:0/2655185496 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0350096020 con 0x7f0360100740 2023-07-06T20:13:41.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:41.540+0000 7f0368308700 1 -- 172.21.15.98:0/2655185496 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0360060a80 con 0x7f03480a1150 2023-07-06T20:13:41.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:41 smithi069 ceph-mon[126399]: osdmap e26: 4 total, 4 up, 4 in 2023-07-06T20:13:41.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:41 smithi156 ceph-mon[129280]: osdmap e26: 4 total, 4 up, 4 in 2023-07-06T20:13:41.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:41 smithi098 ceph-mon[128968]: osdmap e26: 4 total, 4 up, 4 in 2023-07-06T20:13:42.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:42 smithi098 ceph-mon[128968]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:42.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:42 smithi098 ceph-mon[128968]: osdmap e27: 4 total, 4 up, 4 in 2023-07-06T20:13:42.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:42 smithi098 ceph-mon[128968]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:13:42.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:42 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:13:42.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:42 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:13:42.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:42 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:42.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:42 smithi098 ceph-mon[128968]: osdmap e28: 4 total, 4 up, 4 in 2023-07-06T20:13:42.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:42 smithi069 ceph-mon[126399]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:42.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:42 smithi069 ceph-mon[126399]: osdmap e27: 4 total, 4 up, 4 in 2023-07-06T20:13:42.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:42 smithi069 ceph-mon[126399]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:13:42.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:42 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:13:42.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:42 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:13:42.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:42 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:42.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:42 smithi069 ceph-mon[126399]: osdmap e28: 4 total, 4 up, 4 in 2023-07-06T20:13:42.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:42 smithi156 ceph-mon[129280]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:42.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:42 smithi156 ceph-mon[129280]: osdmap e27: 4 total, 4 up, 4 in 2023-07-06T20:13:42.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:42 smithi156 ceph-mon[129280]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:13:42.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:42 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:13:42.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:42 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:13:42.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:42 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:42.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:42 smithi156 ceph-mon[129280]: osdmap e28: 4 total, 4 up, 4 in 2023-07-06T20:13:44.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:44 smithi156 ceph-mon[129280]: osdmap e29: 4 total, 4 up, 4 in 2023-07-06T20:13:44.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:44 smithi156 ceph-mon[129280]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 145 KiB/s, 0 objects/s recovering 2023-07-06T20:13:44.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:44 smithi156 ceph-mon[129280]: from='client.? 172.21.15.98:0/2064789655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66a50ae1-cca5-474e-a2f0-ace23bbc0ed8"}]: dispatch 2023-07-06T20:13:44.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:44 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66a50ae1-cca5-474e-a2f0-ace23bbc0ed8"}]: dispatch 2023-07-06T20:13:44.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:44 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66a50ae1-cca5-474e-a2f0-ace23bbc0ed8"}]': finished 2023-07-06T20:13:44.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:44 smithi156 ceph-mon[129280]: osdmap e30: 5 total, 4 up, 5 in 2023-07-06T20:13:44.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:44 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:13:44.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:44 smithi098 ceph-mon[128968]: osdmap e29: 4 total, 4 up, 4 in 2023-07-06T20:13:44.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:44 smithi098 ceph-mon[128968]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 145 KiB/s, 0 objects/s recovering 2023-07-06T20:13:44.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:44 smithi098 ceph-mon[128968]: from='client.? 172.21.15.98:0/2064789655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66a50ae1-cca5-474e-a2f0-ace23bbc0ed8"}]: dispatch 2023-07-06T20:13:44.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:44 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66a50ae1-cca5-474e-a2f0-ace23bbc0ed8"}]: dispatch 2023-07-06T20:13:44.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:44 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66a50ae1-cca5-474e-a2f0-ace23bbc0ed8"}]': finished 2023-07-06T20:13:44.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:44 smithi098 ceph-mon[128968]: osdmap e30: 5 total, 4 up, 5 in 2023-07-06T20:13:44.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:44 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:13:44.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:44 smithi069 ceph-mon[126399]: osdmap e29: 4 total, 4 up, 4 in 2023-07-06T20:13:44.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:44 smithi069 ceph-mon[126399]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 145 KiB/s, 0 objects/s recovering 2023-07-06T20:13:44.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:44 smithi069 ceph-mon[126399]: from='client.? 172.21.15.98:0/2064789655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66a50ae1-cca5-474e-a2f0-ace23bbc0ed8"}]: dispatch 2023-07-06T20:13:44.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:44 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66a50ae1-cca5-474e-a2f0-ace23bbc0ed8"}]: dispatch 2023-07-06T20:13:44.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:44 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66a50ae1-cca5-474e-a2f0-ace23bbc0ed8"}]': finished 2023-07-06T20:13:44.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:44 smithi069 ceph-mon[126399]: osdmap e30: 5 total, 4 up, 5 in 2023-07-06T20:13:44.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:44 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:13:45.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:45 smithi098 ceph-mon[128968]: from='client.? 172.21.15.98:0/697635204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:13:45.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:45 smithi069 ceph-mon[126399]: from='client.? 172.21.15.98:0/697635204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:13:45.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:45 smithi156 ceph-mon[129280]: from='client.? 172.21.15.98:0/697635204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:13:46.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:46 smithi098 ceph-mon[128968]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-06T20:13:46.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:46 smithi069 ceph-mon[126399]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-06T20:13:46.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:46 smithi156 ceph-mon[129280]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-06T20:13:48.307 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:48 smithi098 ceph-mon[128968]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-06T20:13:48.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:48 smithi156 ceph-mon[129280]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-06T20:13:48.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:48 smithi069 ceph-mon[126399]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-06T20:13:50.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:50 smithi098 ceph-mon[128968]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-06T20:13:50.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:50 smithi156 ceph-mon[129280]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-06T20:13:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:50 smithi069 ceph-mon[126399]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-06T20:13:51.623 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-06T20:13:51.623 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:51.623 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:51 smithi098 ceph-mon[128968]: Deploying daemon osd.4 on smithi098 2023-07-06T20:13:51.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-06T20:13:51.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:51.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:51 smithi156 ceph-mon[129280]: Deploying daemon osd.4 on smithi098 2023-07-06T20:13:52.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-06T20:13:52.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:52.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:51 smithi069 ceph-mon[126399]: Deploying daemon osd.4 on smithi098 2023-07-06T20:13:52.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:52 smithi156 ceph-mon[129280]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:52.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:52 smithi098 ceph-mon[128968]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:52 smithi069 ceph-mon[126399]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:54.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:54 smithi156 ceph-mon[129280]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:54.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:54 smithi098 ceph-mon[128968]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:55.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:54 smithi069 ceph-mon[126399]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:56.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:56.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:56.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:56.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:56 smithi098 ceph-mon[128968]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:56.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:56.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:56.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:56.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:56 smithi069 ceph-mon[126399]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:56.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:56.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:56.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:56.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:56 smithi156 ceph-mon[129280]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:57.963 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.961+0000 7f035f7fe700 1 -- 172.21.15.98:0/2655185496 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0360060a80 con 0x7f03480a1150 2023-07-06T20:13:57.964 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 4 on host 'smithi098' 2023-07-06T20:13:57.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.964+0000 7f0368308700 1 -- 172.21.15.98:0/2655185496 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f03480a1150 msgr2=0x7f03480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:57.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.964+0000 7f0368308700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f03480a1150 0x7f03480a3600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0358007ab0 tx=0x7f0358006210 comp rx=0 tx=0).stop 2023-07-06T20:13:57.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.964+0000 7f0368308700 1 -- 172.21.15.98:0/2655185496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0360100740 msgr2=0x7f036019cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:13:57.965 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.964+0000 7f0368308700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0360100740 0x7f036019cef0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f035000c900 tx=0x7f035000cc10 comp rx=0 tx=0).stop 2023-07-06T20:13:57.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.964+0000 7f0368308700 1 -- 172.21.15.98:0/2655185496 shutdown_connections 2023-07-06T20:13:57.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.964+0000 7f0368308700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f03480a1150 0x7f03480a3600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:57.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.964+0000 7f0368308700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0360100740 0x7f036019cef0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:57.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.964+0000 7f0368308700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03600febb0 0x7f0360068c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:57.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.964+0000 7f0368308700 1 --2- 172.21.15.98:0/2655185496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03600ffdb0 0x7f036019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:13:57.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.964+0000 7f0368308700 1 -- 172.21.15.98:0/2655185496 >> 172.21.15.98:0/2655185496 conn(0x7f03600fa1e0 msgr2=0x7f03601032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:13:57.966 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.964+0000 7f0368308700 1 -- 172.21.15.98:0/2655185496 shutdown_connections 2023-07-06T20:13:57.967 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:13:57.964+0000 7f0368308700 1 -- 172.21.15.98:0/2655185496 wait complete. 2023-07-06T20:13:58.429 DEBUG:teuthology.orchestra.run.smithi098:osd.4> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.4.service 2023-07-06T20:13:58.431 INFO:tasks.cephadm:Deploying osd.5 on smithi156 with /dev/vg_nvme/lv_3... 2023-07-06T20:13:58.431 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-06T20:13:58.473 INFO:journalctl@ceph.osd.4.smithi098.stdout:-- Logs begin at Thu 2023-07-06 19:39:25 UTC. -- 2023-07-06T20:13:58.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:58 smithi156 ceph-mon[129280]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:58.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:58.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:58 smithi098 ceph-mon[128968]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:58.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:58.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:58.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:58.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:58.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:58.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:58.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:58.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:59.007 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:13:59.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:58 smithi069 ceph-mon[126399]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:13:59.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:59.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:59.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:13:59.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:13:59.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:59.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:59.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:13:59.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:13:59.322 INFO:journalctl@ceph.osd.4.smithi098.stdout:Jul 06 20:13:59 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-4[139599]: 2023-07-06T20:13:59.188+0000 7fd0805b7640 -1 osd.4 0 log_to_monitors true 2023-07-06T20:13:59.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:59 smithi098 ceph-mon[128968]: from='osd.4 [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-06T20:13:59.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:13:59 smithi098 ceph-mon[128968]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-06T20:13:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:59 smithi156 ceph-mon[129280]: from='osd.4 [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-06T20:13:59.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:13:59 smithi156 ceph-mon[129280]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-06T20:14:00.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:59 smithi069 ceph-mon[126399]: from='osd.4 [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-06T20:14:00.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:13:59 smithi069 ceph-mon[126399]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-06T20:14:00.513 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-06T20:14:00.513 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-06T20:14:00.513 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-07-06T20:14:00.514 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-07-06T20:14:00.514 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00846947 s, 1.2 GB/s 2023-07-06T20:14:00.514 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-07-06T20:14:00.625 INFO:journalctl@ceph.osd.4.smithi098.stdout:Jul 06 20:14:00 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-4[139599]: 2023-07-06T20:14:00.536+0000 7fd07542e700 -1 osd.4 0 waiting for initial osdmap 2023-07-06T20:14:00.626 INFO:journalctl@ceph.osd.4.smithi098.stdout:Jul 06 20:14:00 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-4[139599]: 2023-07-06T20:14:00.543+0000 7fd071a24700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:14:00.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:00 smithi098 ceph-mon[128968]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:14:00.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:00 smithi098 ceph-mon[128968]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-06T20:14:00.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:00 smithi098 ceph-mon[128968]: osdmap e31: 5 total, 4 up, 5 in 2023-07-06T20:14:00.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:00.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:00 smithi098 ceph-mon[128968]: from='osd.4 [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:14:00.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:00 smithi098 ceph-mon[128968]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:14:00.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:00 smithi156 ceph-mon[129280]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:14:00.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:00 smithi156 ceph-mon[129280]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-06T20:14:00.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:00 smithi156 ceph-mon[129280]: osdmap e31: 5 total, 4 up, 5 in 2023-07-06T20:14:00.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:00.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:00 smithi156 ceph-mon[129280]: from='osd.4 [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:14:00.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:00 smithi156 ceph-mon[129280]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:14:00.960 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch daemon add osd smithi156:vg_nvme/lv_3 2023-07-06T20:14:01.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:00 smithi069 ceph-mon[126399]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:14:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:00 smithi069 ceph-mon[126399]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-06T20:14:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:00 smithi069 ceph-mon[126399]: osdmap e31: 5 total, 4 up, 5 in 2023-07-06T20:14:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:00 smithi069 ceph-mon[126399]: from='osd.4 [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:14:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:00 smithi069 ceph-mon[126399]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:14:01.516 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:14:01.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-07-06T20:14:01.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: osdmap e32: 5 total, 4 up, 5 in 2023-07-06T20:14:01.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:01.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:01.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: Detected new or changed devices on smithi098 2023-07-06T20:14:01.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:01.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:01.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:01.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:01.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi098 to 8584M 2023-07-06T20:14:01.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:01.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:01.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:01.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:01 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:01.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-07-06T20:14:01.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: osdmap e32: 5 total, 4 up, 5 in 2023-07-06T20:14:01.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:01.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:01.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: Detected new or changed devices on smithi098 2023-07-06T20:14:01.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:01.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:01.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:01.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:01.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi098 to 8584M 2023-07-06T20:14:01.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:01.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:01.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:01.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:01 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:02.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-07-06T20:14:02.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: osdmap e32: 5 total, 4 up, 5 in 2023-07-06T20:14:02.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:02.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:02.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: Detected new or changed devices on smithi098 2023-07-06T20:14:02.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:02.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:02.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:02.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:02.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi098 to 8584M 2023-07-06T20:14:02.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:02.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:02.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:02.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:02.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:02 smithi156 ceph-mon[129280]: purged_snaps scrub starts 2023-07-06T20:14:02.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:02 smithi156 ceph-mon[129280]: purged_snaps scrub ok 2023-07-06T20:14:02.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:02 smithi156 ceph-mon[129280]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:14:02.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:02.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:02 smithi156 ceph-mon[129280]: osd.4 [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162] boot 2023-07-06T20:14:02.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:02 smithi156 ceph-mon[129280]: osdmap e33: 5 total, 5 up, 5 in 2023-07-06T20:14:02.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:02.795 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.793+0000 7f35a8836700 1 -- 172.21.15.156:0/191278144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a4100740 msgr2=0x7f35a4107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.793+0000 7f35a8836700 1 --2- 172.21.15.156:0/191278144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a4100740 0x7f35a4107000 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f359c00b0a0 tx=0x7f359c00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.793+0000 7f35a8836700 1 -- 172.21.15.156:0/191278144 shutdown_connections 2023-07-06T20:14:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.793+0000 7f35a8836700 1 --2- 172.21.15.156:0/191278144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a4100740 0x7f35a4107000 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.793+0000 7f35a8836700 1 --2- 172.21.15.156:0/191278144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35a40febb0 0x7f35a40fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.793+0000 7f35a8836700 1 --2- 172.21.15.156:0/191278144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35a40ffdb0 0x7f35a4100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.793+0000 7f35a8836700 1 -- 172.21.15.156:0/191278144 >> 172.21.15.156:0/191278144 conn(0x7f35a40fa1e0 msgr2=0x7f35a40fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.793+0000 7f35a8836700 1 -- 172.21.15.156:0/191278144 shutdown_connections 2023-07-06T20:14:02.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.793+0000 7f35a8836700 1 -- 172.21.15.156:0/191278144 wait complete. 2023-07-06T20:14:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a8836700 1 Processor -- start 2023-07-06T20:14:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a8836700 1 -- start start 2023-07-06T20:14:02.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a8836700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a40febb0 0x7f35a4068c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a8836700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35a40ffdb0 0x7f35a419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a8836700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35a4100740 0x7f35a419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a8836700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a419d680 con 0x7f35a4100740 2023-07-06T20:14:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a8836700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a419d7f0 con 0x7f35a40ffdb0 2023-07-06T20:14:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a8836700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a419d9b0 con 0x7f35a40febb0 2023-07-06T20:14:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a40febb0 0x7f35a4068c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:02.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a40febb0 0x7f35a4068c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59790/0 (socket says 172.21.15.156:59790) 2023-07-06T20:14:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a259c700 1 -- 172.21.15.156:0/1927293063 learned_addr learned my addr 172.21.15.156:0/1927293063 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-06T20:14:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a1d9b700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35a40ffdb0 0x7f35a419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a259c700 1 -- 172.21.15.156:0/1927293063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35a40ffdb0 msgr2=0x7f35a419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a259c700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35a40ffdb0 0x7f35a419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a259c700 1 -- 172.21.15.156:0/1927293063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35a4100740 msgr2=0x7f35a419cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a259c700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35a4100740 0x7f35a419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a259c700 1 -- 172.21.15.156:0/1927293063 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c009d00 con 0x7f35a40febb0 2023-07-06T20:14:02.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.794+0000 7f35a259c700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a40febb0 0x7f35a4068c60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f359800b500 tx=0x7f359800b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:02.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.795+0000 7f35937fe700 1 -- 172.21.15.156:0/1927293063 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598011830 con 0x7f35a40febb0 2023-07-06T20:14:02.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.795+0000 7f35937fe700 1 -- 172.21.15.156:0/1927293063 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3598011e50 con 0x7f35a40febb0 2023-07-06T20:14:02.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.795+0000 7f35a8836700 1 -- 172.21.15.156:0/1927293063 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a419dc90 con 0x7f35a40febb0 2023-07-06T20:14:02.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.795+0000 7f35937fe700 1 -- 172.21.15.156:0/1927293063 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359800f2e0 con 0x7f35a40febb0 2023-07-06T20:14:02.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.795+0000 7f35a8836700 1 -- 172.21.15.156:0/1927293063 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a419e210 con 0x7f35a40febb0 2023-07-06T20:14:02.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.796+0000 7f35a8836700 1 -- 172.21.15.156:0/1927293063 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a41060f0 con 0x7f35a40febb0 2023-07-06T20:14:02.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.797+0000 7f35937fe700 1 -- 172.21.15.156:0/1927293063 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3598010580 con 0x7f35a40febb0 2023-07-06T20:14:02.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.797+0000 7f35937fe700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f358c0a11d0 0x7f358c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:02.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.798+0000 7f35937fe700 1 -- 172.21.15.156:0/1927293063 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f35980cb890 con 0x7f35a40febb0 2023-07-06T20:14:02.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.798+0000 7f35a1d9b700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f358c0a11d0 0x7f358c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:02.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.798+0000 7f35a1d9b700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f358c0a11d0 0x7f358c0a3680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3594005fd0 tx=0x7f359400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:02.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.801+0000 7f35937fe700 1 -- 172.21.15.156:0/1927293063 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3598096050 con 0x7f35a40febb0 2023-07-06T20:14:02.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:02 smithi098 ceph-mon[128968]: purged_snaps scrub starts 2023-07-06T20:14:02.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:02 smithi098 ceph-mon[128968]: purged_snaps scrub ok 2023-07-06T20:14:02.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:02 smithi098 ceph-mon[128968]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:14:02.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:02.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:02 smithi098 ceph-mon[128968]: osd.4 [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162] boot 2023-07-06T20:14:02.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:02 smithi098 ceph-mon[128968]: osdmap e33: 5 total, 5 up, 5 in 2023-07-06T20:14:02.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:02.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:02.992+0000 7f35a8836700 1 -- 172.21.15.156:0/1927293063 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f35a4060a80 con 0x7f358c0a11d0 2023-07-06T20:14:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:02 smithi069 ceph-mon[126399]: purged_snaps scrub starts 2023-07-06T20:14:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:02 smithi069 ceph-mon[126399]: purged_snaps scrub ok 2023-07-06T20:14:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:02 smithi069 ceph-mon[126399]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-06T20:14:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:02 smithi069 ceph-mon[126399]: osd.4 [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162] boot 2023-07-06T20:14:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:02 smithi069 ceph-mon[126399]: osdmap e33: 5 total, 5 up, 5 in 2023-07-06T20:14:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-06T20:14:03.810 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:03 smithi156 ceph-mon[129280]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:14:03.810 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:03 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:14:03.810 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:03 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:14:03.810 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:03 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:03.811 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:03 smithi156 ceph-mon[129280]: osdmap e34: 5 total, 5 up, 5 in 2023-07-06T20:14:03.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:03 smithi098 ceph-mon[128968]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:14:03.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:03 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:14:03.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:03 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:14:03.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:03 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:03.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:03 smithi098 ceph-mon[128968]: osdmap e34: 5 total, 5 up, 5 in 2023-07-06T20:14:04.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:03 smithi069 ceph-mon[126399]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:14:04.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:03 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:14:04.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:03 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:14:04.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:03 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:04.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:03 smithi069 ceph-mon[126399]: osdmap e34: 5 total, 5 up, 5 in 2023-07-06T20:14:04.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:04 smithi156 ceph-mon[129280]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:04.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:04 smithi098 ceph-mon[128968]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:05.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:04 smithi069 ceph-mon[126399]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:05.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:05 smithi156 ceph-mon[129280]: from='client.? 172.21.15.156:0/709371494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f62add7f-8f7d-46a4-b28b-e26110346d85"}]: dispatch 2023-07-06T20:14:05.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:05 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f62add7f-8f7d-46a4-b28b-e26110346d85"}]: dispatch 2023-07-06T20:14:05.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:05 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f62add7f-8f7d-46a4-b28b-e26110346d85"}]': finished 2023-07-06T20:14:05.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:05 smithi156 ceph-mon[129280]: osdmap e35: 6 total, 5 up, 6 in 2023-07-06T20:14:05.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:05 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:05.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:05 smithi098 ceph-mon[128968]: from='client.? 172.21.15.156:0/709371494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f62add7f-8f7d-46a4-b28b-e26110346d85"}]: dispatch 2023-07-06T20:14:05.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:05 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f62add7f-8f7d-46a4-b28b-e26110346d85"}]: dispatch 2023-07-06T20:14:05.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:05 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f62add7f-8f7d-46a4-b28b-e26110346d85"}]': finished 2023-07-06T20:14:05.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:05 smithi098 ceph-mon[128968]: osdmap e35: 6 total, 5 up, 6 in 2023-07-06T20:14:05.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:05 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:06.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:05 smithi069 ceph-mon[126399]: from='client.? 172.21.15.156:0/709371494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f62add7f-8f7d-46a4-b28b-e26110346d85"}]: dispatch 2023-07-06T20:14:06.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:05 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f62add7f-8f7d-46a4-b28b-e26110346d85"}]: dispatch 2023-07-06T20:14:06.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:05 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f62add7f-8f7d-46a4-b28b-e26110346d85"}]': finished 2023-07-06T20:14:06.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:05 smithi069 ceph-mon[126399]: osdmap e35: 6 total, 5 up, 6 in 2023-07-06T20:14:06.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:05 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:06.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:06 smithi156 ceph-mon[129280]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:06.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:06 smithi156 ceph-mon[129280]: from='client.? 172.21.15.156:0/3219493945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:14:06.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:06 smithi098 ceph-mon[128968]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:06.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:06 smithi098 ceph-mon[128968]: from='client.? 172.21.15.156:0/3219493945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:14:07.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:06 smithi069 ceph-mon[126399]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:07.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:06 smithi069 ceph-mon[126399]: from='client.? 172.21.15.156:0/3219493945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:14:08.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:08 smithi156 ceph-mon[129280]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:08.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:08 smithi098 ceph-mon[128968]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:08.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:08 smithi069 ceph-mon[126399]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:10.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:10 smithi156 ceph-mon[129280]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:10.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:10 smithi098 ceph-mon[128968]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:11.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:10 smithi069 ceph-mon[126399]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:12.698 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:12 smithi156 ceph-mon[129280]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:12.698 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-06T20:14:12.698 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:12.698 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:12 smithi156 ceph-mon[129280]: Deploying daemon osd.5 on smithi156 2023-07-06T20:14:12.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:12 smithi098 ceph-mon[128968]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:12.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-06T20:14:12.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:12.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:12 smithi098 ceph-mon[128968]: Deploying daemon osd.5 on smithi156 2023-07-06T20:14:13.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:12 smithi069 ceph-mon[126399]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:13.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-06T20:14:13.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:13.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:12 smithi069 ceph-mon[126399]: Deploying daemon osd.5 on smithi156 2023-07-06T20:14:14.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:14 smithi156 ceph-mon[129280]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:14.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:14 smithi098 ceph-mon[128968]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:15.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:14 smithi069 ceph-mon[126399]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:16.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:16 smithi156 ceph-mon[129280]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:16.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:16.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:16.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:14:16.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:16 smithi098 ceph-mon[128968]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:16.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:16.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:16.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:14:17.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:16 smithi069 ceph-mon[126399]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:17.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:17.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:17.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:14:18.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:18 smithi156 ceph-mon[129280]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:18.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:18 smithi098 ceph-mon[128968]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:18.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:18 smithi069 ceph-mon[126399]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:19.061 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.059+0000 7f35937fe700 1 -- 172.21.15.156:0/1927293063 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f35a4060a80 con 0x7f358c0a11d0 2023-07-06T20:14:19.061 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 5 on host 'smithi156' 2023-07-06T20:14:19.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.063+0000 7f35a8836700 1 -- 172.21.15.156:0/1927293063 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f358c0a11d0 msgr2=0x7f358c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.064 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.063+0000 7f35a8836700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f358c0a11d0 0x7f358c0a3680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3594005fd0 tx=0x7f359400a560 comp rx=0 tx=0).stop 2023-07-06T20:14:19.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.063+0000 7f35a8836700 1 -- 172.21.15.156:0/1927293063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a40febb0 msgr2=0x7f35a4068c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:19.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.063+0000 7f35a8836700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a40febb0 0x7f35a4068c60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f359800b500 tx=0x7f359800b8c0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.063+0000 7f35a8836700 1 -- 172.21.15.156:0/1927293063 shutdown_connections 2023-07-06T20:14:19.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.063+0000 7f35a8836700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f358c0a11d0 0x7f358c0a3680 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.063+0000 7f35a8836700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35a4100740 0x7f35a419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.063+0000 7f35a8836700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a40febb0 0x7f35a4068c60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.065 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.063+0000 7f35a8836700 1 --2- 172.21.15.156:0/1927293063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35a40ffdb0 0x7f35a419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:19.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.063+0000 7f35a8836700 1 -- 172.21.15.156:0/1927293063 >> 172.21.15.156:0/1927293063 conn(0x7f35a40fa1e0 msgr2=0x7f35a4103230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:19.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.063+0000 7f35a8836700 1 -- 172.21.15.156:0/1927293063 shutdown_connections 2023-07-06T20:14:19.066 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:14:19.063+0000 7f35a8836700 1 -- 172.21.15.156:0/1927293063 wait complete. 2023-07-06T20:14:19.497 DEBUG:teuthology.orchestra.run.smithi156:osd.5> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.5.service 2023-07-06T20:14:19.499 INFO:tasks.cephadm:Deploying osd.6 on smithi069 with /dev/vg_nvme/lv_2... 2023-07-06T20:14:19.499 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-06T20:14:19.529 INFO:journalctl@ceph.osd.5.smithi156.stdout:-- Logs begin at Thu 2023-07-06 19:42:21 UTC. -- 2023-07-06T20:14:19.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:19.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:19.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:14:19.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:19.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:19.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:14:19.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:19.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:19.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:19.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:14:20.226 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:14:20.247 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:14:20 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:14:20.126+0000 7f0ab4add640 -1 osd.5 0 log_to_monitors true 2023-07-06T20:14:20.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:20 smithi156 ceph-mon[129280]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:20.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:20 smithi156 ceph-mon[129280]: from='osd.5 [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-06T20:14:20.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:20 smithi156 ceph-mon[129280]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-06T20:14:20.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:20 smithi069 ceph-mon[126399]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:20.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:20 smithi069 ceph-mon[126399]: from='osd.5 [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-06T20:14:20.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:20 smithi069 ceph-mon[126399]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-06T20:14:20.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:20 smithi098 ceph-mon[128968]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:20.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:20 smithi098 ceph-mon[128968]: from='osd.5 [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-06T20:14:20.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:20 smithi098 ceph-mon[128968]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-06T20:14:21.606 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:14:21 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:14:21.413+0000 7f0aab157700 -1 osd.5 0 waiting for initial osdmap 2023-07-06T20:14:21.607 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:14:21 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:14:21.420+0000 7f0aa5f4a700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:14:21.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:21 smithi156 ceph-mon[129280]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-06T20:14:21.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:21 smithi156 ceph-mon[129280]: osdmap e36: 6 total, 5 up, 6 in 2023-07-06T20:14:21.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:21 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:21.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:21 smithi156 ceph-mon[129280]: from='osd.5 [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:14:21.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:21 smithi156 ceph-mon[129280]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:14:21.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:21 smithi069 ceph-mon[126399]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-06T20:14:21.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:21 smithi069 ceph-mon[126399]: osdmap e36: 6 total, 5 up, 6 in 2023-07-06T20:14:21.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:21 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:21.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:21 smithi069 ceph-mon[126399]: from='osd.5 [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:14:21.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:21 smithi069 ceph-mon[126399]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:14:21.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:21 smithi098 ceph-mon[128968]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-06T20:14:21.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:21 smithi098 ceph-mon[128968]: osdmap e36: 6 total, 5 up, 6 in 2023-07-06T20:14:21.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:21 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:21.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:21 smithi098 ceph-mon[128968]: from='osd.5 [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:14:21.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:21 smithi098 ceph-mon[128968]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:14:22.204 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-06T20:14:22.204 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-06T20:14:22.204 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2023-07-06T20:14:22.204 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2023-07-06T20:14:22.204 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0507919 s, 206 MB/s 2023-07-06T20:14:22.204 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2023-07-06T20:14:22.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-06T20:14:22.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: osdmap e37: 6 total, 5 up, 6 in 2023-07-06T20:14:22.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:22.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:22.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:22.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: Detected new or changed devices on smithi156 2023-07-06T20:14:22.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:22.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:22.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:22.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:22.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi156 to 8584M 2023-07-06T20:14:22.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:22.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:22.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:22.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:22.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: osd.5 [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634] boot 2023-07-06T20:14:22.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: osdmap e38: 6 total, 6 up, 6 in 2023-07-06T20:14:22.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:22.701 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch daemon add osd smithi069:vg_nvme/lv_2 2023-07-06T20:14:22.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-06T20:14:22.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: osdmap e37: 6 total, 5 up, 6 in 2023-07-06T20:14:22.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:22.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:22.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:22.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: Detected new or changed devices on smithi156 2023-07-06T20:14:22.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:22.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:22.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:22.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:22.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi156 to 8584M 2023-07-06T20:14:22.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:22.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:22.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:22.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:22.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: osd.5 [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634] boot 2023-07-06T20:14:22.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: osdmap e38: 6 total, 6 up, 6 in 2023-07-06T20:14:22.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:22.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-06T20:14:22.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: osdmap e37: 6 total, 5 up, 6 in 2023-07-06T20:14:22.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:22.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:22.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-06T20:14:22.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: Detected new or changed devices on smithi156 2023-07-06T20:14:22.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:22.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:22.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:22.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:22.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi156 to 8584M 2023-07-06T20:14:22.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:22.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:22.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:22.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:22.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: osd.5 [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634] boot 2023-07-06T20:14:22.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: osdmap e38: 6 total, 6 up, 6 in 2023-07-06T20:14:22.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-06T20:14:23.422 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:14:24.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:24 smithi098 ceph-mon[128968]: purged_snaps scrub starts 2023-07-06T20:14:24.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:24 smithi098 ceph-mon[128968]: purged_snaps scrub ok 2023-07-06T20:14:24.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:24 smithi098 ceph-mon[128968]: osdmap e39: 6 total, 6 up, 6 in 2023-07-06T20:14:24.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:24 smithi098 ceph-mon[128968]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:24 smithi069 ceph-mon[126399]: purged_snaps scrub starts 2023-07-06T20:14:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:24 smithi069 ceph-mon[126399]: purged_snaps scrub ok 2023-07-06T20:14:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:24 smithi069 ceph-mon[126399]: osdmap e39: 6 total, 6 up, 6 in 2023-07-06T20:14:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:24 smithi069 ceph-mon[126399]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:24.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:24 smithi156 ceph-mon[129280]: purged_snaps scrub starts 2023-07-06T20:14:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:24 smithi156 ceph-mon[129280]: purged_snaps scrub ok 2023-07-06T20:14:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:24 smithi156 ceph-mon[129280]: osdmap e39: 6 total, 6 up, 6 in 2023-07-06T20:14:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:24 smithi156 ceph-mon[129280]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:24.885 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.883+0000 7f95982f6700 1 -- 172.21.15.69:0/3309043008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95900ffcc0 msgr2=0x7f95901000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:24.885 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.883+0000 7f95982f6700 1 --2- 172.21.15.69:0/3309043008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95900ffcc0 0x7f95901000d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f958c009990 tx=0x7f958c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:14:24.885 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.883+0000 7f95982f6700 1 -- 172.21.15.69:0/3309043008 shutdown_connections 2023-07-06T20:14:24.886 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.883+0000 7f95982f6700 1 --2- 172.21.15.69:0/3309043008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9590101850 0x7f9590108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:24.886 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.883+0000 7f95982f6700 1 --2- 172.21.15.69:0/3309043008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95900ffcc0 0x7f95901000d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:24.886 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.883+0000 7f95982f6700 1 --2- 172.21.15.69:0/3309043008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590100ec0 0x7f9590101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:24.886 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.883+0000 7f95982f6700 1 -- 172.21.15.69:0/3309043008 >> 172.21.15.69:0/3309043008 conn(0x7f95900fb2d0 msgr2=0x7f95900fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:24.886 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.883+0000 7f95982f6700 1 -- 172.21.15.69:0/3309043008 shutdown_connections 2023-07-06T20:14:24.886 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.883+0000 7f95982f6700 1 -- 172.21.15.69:0/3309043008 wait complete. 2023-07-06T20:14:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.884+0000 7f95982f6700 1 Processor -- start 2023-07-06T20:14:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.884+0000 7f95982f6700 1 -- start start 2023-07-06T20:14:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.884+0000 7f95982f6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95900ffcc0 0x7f95900743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.884+0000 7f95982f6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590100ec0 0x7f9590072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.884+0000 7f95982f6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9590101850 0x7f9590072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.884+0000 7f95982f6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95901998a0 con 0x7f95900ffcc0 2023-07-06T20:14:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.884+0000 7f95982f6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95900734b0 con 0x7f9590100ec0 2023-07-06T20:14:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.884+0000 7f95982f6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9590073670 con 0x7f9590101850 2023-07-06T20:14:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.884+0000 7f9596092700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95900ffcc0 0x7f95900743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.884+0000 7f9596092700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95900ffcc0 0x7f95900743e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45150/0 (socket says 172.21.15.69:45150) 2023-07-06T20:14:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.885+0000 7f9596092700 1 -- 172.21.15.69:0/3518839661 learned_addr learned my addr 172.21.15.69:0/3518839661 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:14:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.885+0000 7f9595891700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590100ec0 0x7f9590072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.885+0000 7f9596893700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9590101850 0x7f9590072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:24.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.885+0000 7f9596092700 1 -- 172.21.15.69:0/3518839661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9590101850 msgr2=0x7f9590072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:24.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.885+0000 7f9596092700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9590101850 0x7f9590072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:24.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.885+0000 7f9596092700 1 -- 172.21.15.69:0/3518839661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590100ec0 msgr2=0x7f9590072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:24.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.885+0000 7f9596092700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590100ec0 0x7f9590072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:24.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.885+0000 7f9596092700 1 -- 172.21.15.69:0/3518839661 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c0095f0 con 0x7f95900ffcc0 2023-07-06T20:14:24.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.885+0000 7f9595891700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590100ec0 0x7f9590072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:14:24.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.885+0000 7f9596092700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95900ffcc0 0x7f95900743e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b680 tx=0x7f958c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.885+0000 7f95877fe700 1 -- 172.21.15.69:0/3518839661 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c01d070 con 0x7f95900ffcc0 2023-07-06T20:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.886+0000 7f95877fe700 1 -- 172.21.15.69:0/3518839661 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f958c021410 con 0x7f95900ffcc0 2023-07-06T20:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.886+0000 7f95982f6700 1 -- 172.21.15.69:0/3518839661 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95901a4270 con 0x7f95900ffcc0 2023-07-06T20:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.886+0000 7f95877fe700 1 -- 172.21.15.69:0/3518839661 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c017650 con 0x7f95900ffcc0 2023-07-06T20:14:24.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.886+0000 7f95982f6700 1 -- 172.21.15.69:0/3518839661 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95901a45d0 con 0x7f95900ffcc0 2023-07-06T20:14:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.887+0000 7f95982f6700 1 -- 172.21.15.69:0/3518839661 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9590107300 con 0x7f95900ffcc0 2023-07-06T20:14:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.889+0000 7f95877fe700 1 -- 172.21.15.69:0/3518839661 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f958c021580 con 0x7f95900ffcc0 2023-07-06T20:14:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.890+0000 7f95877fe700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f957c0a11d0 0x7f957c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.890+0000 7f95877fe700 1 -- 172.21.15.69:0/3518839661 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f958c0cd5c0 con 0x7f95900ffcc0 2023-07-06T20:14:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.890+0000 7f9595891700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f957c0a11d0 0x7f957c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.891+0000 7f9595891700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f957c0a11d0 0x7f957c0a3680 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9580005950 tx=0x7f958000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:24.894 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:24.893+0000 7f95877fe700 1 -- 172.21.15.69:0/3518839661 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f958c097ba0 con 0x7f95900ffcc0 2023-07-06T20:14:25.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:25.095+0000 7f95982f6700 1 -- 172.21.15.69:0/3518839661 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9590060a80 con 0x7f957c0a11d0 2023-07-06T20:14:25.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:25 smithi098 ceph-mon[128968]: osdmap e40: 6 total, 6 up, 6 in 2023-07-06T20:14:25.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:25 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:14:25.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:25 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:14:25.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:25 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:25.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:25 smithi069 ceph-mon[126399]: osdmap e40: 6 total, 6 up, 6 in 2023-07-06T20:14:25.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:25 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:14:25.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:25 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:14:25.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:25 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:25.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:25 smithi156 ceph-mon[129280]: osdmap e40: 6 total, 6 up, 6 in 2023-07-06T20:14:25.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:25 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:14:25.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:25 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:14:25.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:25 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:26.382 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:26 smithi069 ceph-mon[126399]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:14:26.382 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:26 smithi069 ceph-mon[126399]: osdmap e41: 6 total, 6 up, 6 in 2023-07-06T20:14:26.383 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:26 smithi069 ceph-mon[126399]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:26.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:26 smithi156 ceph-mon[129280]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:14:26.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:26 smithi156 ceph-mon[129280]: osdmap e41: 6 total, 6 up, 6 in 2023-07-06T20:14:26.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:26 smithi156 ceph-mon[129280]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:26.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:26 smithi098 ceph-mon[128968]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:14:26.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:26 smithi098 ceph-mon[128968]: osdmap e41: 6 total, 6 up, 6 in 2023-07-06T20:14:26.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:26 smithi098 ceph-mon[128968]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:27.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:27 smithi069 ceph-mon[126399]: osdmap e42: 6 total, 6 up, 6 in 2023-07-06T20:14:27.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:27 smithi156 ceph-mon[129280]: osdmap e42: 6 total, 6 up, 6 in 2023-07-06T20:14:27.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:27 smithi098 ceph-mon[128968]: osdmap e42: 6 total, 6 up, 6 in 2023-07-06T20:14:28.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:28 smithi069 ceph-mon[126399]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:28.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:28 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1245715731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fdcdab4-b7d2-4e02-908a-f7ce47c8c735"}]: dispatch 2023-07-06T20:14:28.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:28 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1245715731' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fdcdab4-b7d2-4e02-908a-f7ce47c8c735"}]': finished 2023-07-06T20:14:28.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:28 smithi069 ceph-mon[126399]: osdmap e43: 7 total, 6 up, 7 in 2023-07-06T20:14:28.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:28 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:28.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:28 smithi156 ceph-mon[129280]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:28.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:28 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1245715731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fdcdab4-b7d2-4e02-908a-f7ce47c8c735"}]: dispatch 2023-07-06T20:14:28.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:28 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1245715731' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fdcdab4-b7d2-4e02-908a-f7ce47c8c735"}]': finished 2023-07-06T20:14:28.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:28 smithi156 ceph-mon[129280]: osdmap e43: 7 total, 6 up, 7 in 2023-07-06T20:14:28.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:28 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:28.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:28 smithi098 ceph-mon[128968]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:28.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:28 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1245715731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fdcdab4-b7d2-4e02-908a-f7ce47c8c735"}]: dispatch 2023-07-06T20:14:28.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:28 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1245715731' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fdcdab4-b7d2-4e02-908a-f7ce47c8c735"}]': finished 2023-07-06T20:14:28.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:28 smithi098 ceph-mon[128968]: osdmap e43: 7 total, 6 up, 7 in 2023-07-06T20:14:28.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:28 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:29.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:29 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2016688836' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:14:29.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:29 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2016688836' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:14:29.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:29 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2016688836' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:14:30.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:30 smithi069 ceph-mon[126399]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-07-06T20:14:30.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:30 smithi156 ceph-mon[129280]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-07-06T20:14:30.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:30 smithi098 ceph-mon[128968]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-07-06T20:14:32.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:32 smithi156 ceph-mon[129280]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-07-06T20:14:32.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:32 smithi098 ceph-mon[128968]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-07-06T20:14:33.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:32 smithi069 ceph-mon[126399]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-07-06T20:14:34.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:34 smithi156 ceph-mon[129280]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-06T20:14:34.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:34 smithi098 ceph-mon[128968]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-06T20:14:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:34 smithi069 ceph-mon[126399]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-06T20:14:35.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:35 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-06T20:14:35.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:35 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:35.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:35 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-06T20:14:35.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:35 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:36.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:35 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-06T20:14:36.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:35 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:36.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:36 smithi069 ceph-mon[126399]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-07-06T20:14:36.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:36 smithi069 ceph-mon[126399]: Deploying daemon osd.6 on smithi069 2023-07-06T20:14:36.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:36 smithi156 ceph-mon[129280]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-07-06T20:14:36.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:36 smithi156 ceph-mon[129280]: Deploying daemon osd.6 on smithi069 2023-07-06T20:14:36.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:36 smithi098 ceph-mon[128968]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-07-06T20:14:36.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:36 smithi098 ceph-mon[128968]: Deploying daemon osd.6 on smithi069 2023-07-06T20:14:38.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:38 smithi069 ceph-mon[126399]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-06T20:14:38.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:38 smithi098 ceph-mon[128968]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-06T20:14:38.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:38 smithi156 ceph-mon[129280]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-06T20:14:40.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:40 smithi069 ceph-mon[126399]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:40.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:40 smithi156 ceph-mon[129280]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:40.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:40 smithi098 ceph-mon[128968]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:42.931 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:42 smithi069 ceph-mon[126399]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:42.931 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:42 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:42.931 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:42 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:42.931 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:42 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:14:43.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:42 smithi156 ceph-mon[129280]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:43.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:42 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:43.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:42 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:43.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:42 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:14:43.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:42 smithi098 ceph-mon[128968]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:43.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:42 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:43.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:42 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:43.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:42 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:14:44.971 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:44 smithi069 ceph-mon[126399]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:44.972 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:44 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:44.972 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:44 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:44.972 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:44 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:44.972 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:44 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:44.972 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:44 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:45.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:44 smithi156 ceph-mon[129280]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:45.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:44 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:45.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:44 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:45.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:44 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:45.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:44 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:45.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:44 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:45.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:44 smithi098 ceph-mon[128968]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:45.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:44 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:45.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:44 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:45.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:44 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:45.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:44 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:45.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:44 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:45.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.302+0000 7f95877fe700 1 -- 172.21.15.69:0/3518839661 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9590060a80 con 0x7f957c0a11d0 2023-07-06T20:14:45.305 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 6 on host 'smithi069' 2023-07-06T20:14:45.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.305+0000 7f95982f6700 1 -- 172.21.15.69:0/3518839661 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f957c0a11d0 msgr2=0x7f957c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.305+0000 7f95982f6700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f957c0a11d0 0x7f957c0a3680 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9580005950 tx=0x7f958000b410 comp rx=0 tx=0).stop 2023-07-06T20:14:45.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.305+0000 7f95982f6700 1 -- 172.21.15.69:0/3518839661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95900ffcc0 msgr2=0x7f95900743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:45.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.305+0000 7f95982f6700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95900ffcc0 0x7f95900743e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b680 tx=0x7f958c005740 comp rx=0 tx=0).stop 2023-07-06T20:14:45.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.305+0000 7f95982f6700 1 -- 172.21.15.69:0/3518839661 shutdown_connections 2023-07-06T20:14:45.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.305+0000 7f95982f6700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f957c0a11d0 0x7f957c0a3680 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.305+0000 7f95982f6700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9590101850 0x7f9590072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.305+0000 7f95982f6700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95900ffcc0 0x7f95900743e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.306+0000 7f95982f6700 1 --2- 172.21.15.69:0/3518839661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590100ec0 0x7f9590072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:45.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.306+0000 7f95982f6700 1 -- 172.21.15.69:0/3518839661 >> 172.21.15.69:0/3518839661 conn(0x7f95900fb2d0 msgr2=0x7f9590104440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:45.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.306+0000 7f95982f6700 1 -- 172.21.15.69:0/3518839661 shutdown_connections 2023-07-06T20:14:45.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:14:45.306+0000 7f95982f6700 1 -- 172.21.15.69:0/3518839661 wait complete. 2023-07-06T20:14:45.840 DEBUG:teuthology.orchestra.run.smithi069:osd.6> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.6.service 2023-07-06T20:14:45.844 INFO:tasks.cephadm:Deploying osd.7 on smithi098 with /dev/vg_nvme/lv_2... 2023-07-06T20:14:45.845 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-06T20:14:45.916 INFO:journalctl@ceph.osd.6.smithi069.stdout:-- Logs begin at Thu 2023-07-06 19:46:03 UTC. -- 2023-07-06T20:14:46.406 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.b/config 2023-07-06T20:14:46.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:46 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:46.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:46 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:46.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:46 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:14:46.545 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:46 smithi069 ceph-mon[126399]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:46.545 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:46 smithi069 ceph-mon[126399]: from='osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-06T20:14:46.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:46 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:46.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:46 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:46.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:46 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:14:46.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:46 smithi156 ceph-mon[129280]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:46.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:46 smithi156 ceph-mon[129280]: from='osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-06T20:14:46.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:46.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:46.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:46 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:14:46.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:46 smithi098 ceph-mon[128968]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:46.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:46 smithi098 ceph-mon[128968]: from='osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-06T20:14:47.513 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:47 smithi098 ceph-mon[128968]: from='osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-06T20:14:47.514 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:47 smithi098 ceph-mon[128968]: osdmap e44: 7 total, 6 up, 7 in 2023-07-06T20:14:47.514 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:47 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:47.514 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:47 smithi098 ceph-mon[128968]: from='osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:14:47.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:47 smithi156 ceph-mon[129280]: from='osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-06T20:14:47.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:47 smithi156 ceph-mon[129280]: osdmap e44: 7 total, 6 up, 7 in 2023-07-06T20:14:47.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:47 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:47.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:47 smithi156 ceph-mon[129280]: from='osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:14:47.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:47 smithi069 ceph-mon[126399]: from='osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-06T20:14:47.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:47 smithi069 ceph-mon[126399]: osdmap e44: 7 total, 6 up, 7 in 2023-07-06T20:14:47.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:47 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:47.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:47 smithi069 ceph-mon[126399]: from='osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:14:47.760 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:14:47 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:14:47.300+0000 7f787fee0700 -1 osd.6 0 waiting for initial osdmap 2023-07-06T20:14:47.760 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:14:47 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:14:47.311+0000 7f787b4d4700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:14:47.995 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-06T20:14:47.996 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-06T20:14:47.996 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2023-07-06T20:14:47.996 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2023-07-06T20:14:47.996 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109967 s, 954 MB/s 2023-07-06T20:14:47.996 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2023-07-06T20:14:48.461 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch daemon add osd smithi098:vg_nvme/lv_2 2023-07-06T20:14:48.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:48 smithi069 ceph-mon[126399]: from='osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-07-06T20:14:48.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:48 smithi069 ceph-mon[126399]: osdmap e45: 7 total, 6 up, 7 in 2023-07-06T20:14:48.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:48.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:48.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:48 smithi069 ceph-mon[126399]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:48.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:48 smithi156 ceph-mon[129280]: from='osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-07-06T20:14:48.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:48 smithi156 ceph-mon[129280]: osdmap e45: 7 total, 6 up, 7 in 2023-07-06T20:14:48.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:48.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:48.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:48 smithi156 ceph-mon[129280]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:48.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:48 smithi098 ceph-mon[128968]: from='osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-07-06T20:14:48.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:48 smithi098 ceph-mon[128968]: osdmap e45: 7 total, 6 up, 7 in 2023-07-06T20:14:48.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:48.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:48.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:48 smithi098 ceph-mon[128968]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-06T20:14:49.010 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.b/config 2023-07-06T20:14:49.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: purged_snaps scrub starts 2023-07-06T20:14:49.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: purged_snaps scrub ok 2023-07-06T20:14:49.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:49.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544] boot 2023-07-06T20:14:49.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: osdmap e46: 7 total, 7 up, 7 in 2023-07-06T20:14:49.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:49.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: Detected new or changed devices on smithi069 2023-07-06T20:14:49.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:49.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:49.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:49.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:49.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:49.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi069 to 5755M 2023-07-06T20:14:49.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:49.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:49.510 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:49.510 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:49.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: purged_snaps scrub starts 2023-07-06T20:14:49.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: purged_snaps scrub ok 2023-07-06T20:14:49.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:49.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544] boot 2023-07-06T20:14:49.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: osdmap e46: 7 total, 7 up, 7 in 2023-07-06T20:14:49.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:49.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: Detected new or changed devices on smithi069 2023-07-06T20:14:49.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:49.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:49.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:49.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:49.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:49.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi069 to 5755M 2023-07-06T20:14:49.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:49.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:49.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:49.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:49.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: purged_snaps scrub starts 2023-07-06T20:14:49.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: purged_snaps scrub ok 2023-07-06T20:14:49.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: osd.6 [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544] boot 2023-07-06T20:14:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: osdmap e46: 7 total, 7 up, 7 in 2023-07-06T20:14:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-06T20:14:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: Detected new or changed devices on smithi069 2023-07-06T20:14:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:49.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:14:49.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi069 to 5755M 2023-07-06T20:14:49.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:49.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:49.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:14:49.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:14:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.204+0000 7f4fead99700 1 -- 172.21.15.98:0/4172093112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fe40ff7c0 msgr2=0x7f4fe4107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.204+0000 7f4fead99700 1 --2- 172.21.15.98:0/4172093112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fe40ff7c0 0x7f4fe4107070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00b0d0 tx=0x7f4fdc00b3e0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.204+0000 7f4fead99700 1 -- 172.21.15.98:0/4172093112 shutdown_connections 2023-07-06T20:14:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.204+0000 7f4fead99700 1 --2- 172.21.15.98:0/4172093112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fe40ff7c0 0x7f4fe4107070 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.204+0000 7f4fead99700 1 --2- 172.21.15.98:0/4172093112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe41001b0 0x7f4fe41005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.204+0000 7f4fead99700 1 --2- 172.21.15.98:0/4172093112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe40fee10 0x7f4fe40ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.204+0000 7f4fead99700 1 -- 172.21.15.98:0/4172093112 >> 172.21.15.98:0/4172093112 conn(0x7f4fe40f99b0 msgr2=0x7f4fe40fbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:14:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.204+0000 7f4fead99700 1 -- 172.21.15.98:0/4172093112 shutdown_connections 2023-07-06T20:14:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.204+0000 7f4fead99700 1 -- 172.21.15.98:0/4172093112 wait complete. 2023-07-06T20:14:50.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fead99700 1 Processor -- start 2023-07-06T20:14:50.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fead99700 1 -- start start 2023-07-06T20:14:50.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fead99700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe40fee10 0x7f4fe4195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fead99700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fe40ff7c0 0x7f4fe4195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.210 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fead99700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe41001b0 0x7f4fe419d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fead99700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe419d640 con 0x7f4fe41001b0 2023-07-06T20:14:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fead99700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe419d780 con 0x7f4fe40ff7c0 2023-07-06T20:14:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fead99700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe419d910 con 0x7f4fe40fee10 2023-07-06T20:14:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fe40ff7c0 0x7f4fe4195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fe40ff7c0 0x7f4fe4195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:44488/0 (socket says 172.21.15.98:44488) 2023-07-06T20:14:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe3fff700 1 -- 172.21.15.98:0/114192408 learned_addr learned my addr 172.21.15.98:0/114192408 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-07-06T20:14:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe8b35700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe40fee10 0x7f4fe4195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe9336700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe41001b0 0x7f4fe419d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe3fff700 1 -- 172.21.15.98:0/114192408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe40fee10 msgr2=0x7f4fe4195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe3fff700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe40fee10 0x7f4fe4195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe3fff700 1 -- 172.21.15.98:0/114192408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe41001b0 msgr2=0x7f4fe419d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:14:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe3fff700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe41001b0 0x7f4fe419d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:14:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe3fff700 1 -- 172.21.15.98:0/114192408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc009d00 con 0x7f4fe40ff7c0 2023-07-06T20:14:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe8b35700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe40fee10 0x7f4fe4195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:14:50.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe9336700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe41001b0 0x7f4fe419d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:14:50.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.205+0000 7f4fe3fff700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fe40ff7c0 0x7f4fe4195ff0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000d950 tx=0x7f4fd000dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:50.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.206+0000 7f4fe1ffb700 1 -- 172.21.15.98:0/114192408 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0009b40 con 0x7f4fe40ff7c0 2023-07-06T20:14:50.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.206+0000 7f4fead99700 1 -- 172.21.15.98:0/114192408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe419dbf0 con 0x7f4fe40ff7c0 2023-07-06T20:14:50.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.206+0000 7f4fe1ffb700 1 -- 172.21.15.98:0/114192408 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fd0009ca0 con 0x7f4fe40ff7c0 2023-07-06T20:14:50.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.206+0000 7f4fe1ffb700 1 -- 172.21.15.98:0/114192408 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd000f620 con 0x7f4fe40ff7c0 2023-07-06T20:14:50.213 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.206+0000 7f4fead99700 1 -- 172.21.15.98:0/114192408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe419e0f0 con 0x7f4fe40ff7c0 2023-07-06T20:14:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.207+0000 7f4fead99700 1 -- 172.21.15.98:0/114192408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe410b500 con 0x7f4fe40ff7c0 2023-07-06T20:14:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.208+0000 7f4fe1ffb700 1 -- 172.21.15.98:0/114192408 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4fd0010460 con 0x7f4fe40ff7c0 2023-07-06T20:14:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.208+0000 7f4fe1ffb700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4fd40a1150 0x7f4fd40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:14:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.208+0000 7f4fe1ffb700 1 -- 172.21.15.98:0/114192408 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f4fd00cbee0 con 0x7f4fe40ff7c0 2023-07-06T20:14:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.208+0000 7f4fe8b35700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4fd40a1150 0x7f4fd40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:14:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.209+0000 7f4fe8b35700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4fd40a1150 0x7f4fd40a3600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8005950 tx=0x7f4fd8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:14:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.212+0000 7f4fe1ffb700 1 -- 172.21.15.98:0/114192408 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4fd0096340 con 0x7f4fe40ff7c0 2023-07-06T20:14:50.409 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:14:50.406+0000 7f4fead99700 1 -- 172.21.15.98:0/114192408 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fe4060a80 con 0x7f4fd40a1150 2023-07-06T20:14:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:50 smithi069 ceph-mon[126399]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:50 smithi069 ceph-mon[126399]: osdmap e47: 7 total, 7 up, 7 in 2023-07-06T20:14:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:50 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:14:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:50 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:14:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:50 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:51.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:50 smithi156 ceph-mon[129280]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:51.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:50 smithi156 ceph-mon[129280]: osdmap e47: 7 total, 7 up, 7 in 2023-07-06T20:14:51.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:50 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:14:51.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:50 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:14:51.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:50 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:51.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:50 smithi098 ceph-mon[128968]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:51.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:50 smithi098 ceph-mon[128968]: osdmap e47: 7 total, 7 up, 7 in 2023-07-06T20:14:51.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:50 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:14:51.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:50 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:14:51.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:50 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:14:51.786 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:51 smithi098 ceph-mon[128968]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:14:52.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:51 smithi069 ceph-mon[126399]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:14:52.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:51 smithi156 ceph-mon[129280]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:14:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:52 smithi069 ceph-mon[126399]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:53.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:52 smithi156 ceph-mon[129280]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:53.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:52 smithi098 ceph-mon[128968]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:54.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:53 smithi069 ceph-mon[126399]: from='client.? 172.21.15.98:0/2611111157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce45b288-d0c7-4ce3-9b99-0b48a168cd74"}]: dispatch 2023-07-06T20:14:54.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:53 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce45b288-d0c7-4ce3-9b99-0b48a168cd74"}]: dispatch 2023-07-06T20:14:54.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:53 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce45b288-d0c7-4ce3-9b99-0b48a168cd74"}]': finished 2023-07-06T20:14:54.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:53 smithi069 ceph-mon[126399]: osdmap e48: 8 total, 7 up, 8 in 2023-07-06T20:14:54.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:14:54.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:53 smithi069 ceph-mon[126399]: from='client.? 172.21.15.98:0/479312784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:14:54.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:53 smithi156 ceph-mon[129280]: from='client.? 172.21.15.98:0/2611111157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce45b288-d0c7-4ce3-9b99-0b48a168cd74"}]: dispatch 2023-07-06T20:14:54.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:53 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce45b288-d0c7-4ce3-9b99-0b48a168cd74"}]: dispatch 2023-07-06T20:14:54.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:53 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce45b288-d0c7-4ce3-9b99-0b48a168cd74"}]': finished 2023-07-06T20:14:54.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:53 smithi156 ceph-mon[129280]: osdmap e48: 8 total, 7 up, 8 in 2023-07-06T20:14:54.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:14:54.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:53 smithi156 ceph-mon[129280]: from='client.? 172.21.15.98:0/479312784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:14:54.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:53 smithi098 ceph-mon[128968]: from='client.? 172.21.15.98:0/2611111157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce45b288-d0c7-4ce3-9b99-0b48a168cd74"}]: dispatch 2023-07-06T20:14:54.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:53 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce45b288-d0c7-4ce3-9b99-0b48a168cd74"}]: dispatch 2023-07-06T20:14:54.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:53 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce45b288-d0c7-4ce3-9b99-0b48a168cd74"}]': finished 2023-07-06T20:14:54.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:53 smithi098 ceph-mon[128968]: osdmap e48: 8 total, 7 up, 8 in 2023-07-06T20:14:54.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:14:54.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:53 smithi098 ceph-mon[128968]: from='client.? 172.21.15.98:0/479312784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:14:55.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:54 smithi069 ceph-mon[126399]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:55.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:54 smithi156 ceph-mon[129280]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:55.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:54 smithi098 ceph-mon[128968]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:57.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:56 smithi098 ceph-mon[128968]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:57.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:56 smithi069 ceph-mon[126399]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:57.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:56 smithi156 ceph-mon[129280]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:58.323 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:14:58 smithi098 ceph-mon[128968]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:58.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:14:58 smithi069 ceph-mon[126399]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:14:58.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:14:58 smithi156 ceph-mon[129280]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:00.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:00 smithi098 ceph-mon[128968]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:00.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-06T20:15:00.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:00.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:00 smithi098 ceph-mon[128968]: Deploying daemon osd.7 on smithi098 2023-07-06T20:15:00.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:00 smithi156 ceph-mon[129280]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:00.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-06T20:15:00.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:00.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:00 smithi156 ceph-mon[129280]: Deploying daemon osd.7 on smithi098 2023-07-06T20:15:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:00 smithi069 ceph-mon[126399]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-06T20:15:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:00 smithi069 ceph-mon[126399]: Deploying daemon osd.7 on smithi098 2023-07-06T20:15:02.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:02 smithi156 ceph-mon[129280]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:02.868 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:02 smithi098 ceph-mon[128968]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:02 smithi069 ceph-mon[126399]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:04.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:04 smithi098 ceph-mon[128968]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:04.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:04 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:04.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:04 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:04.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:04 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:04.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:04 smithi156 ceph-mon[129280]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:04.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:04 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:04.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:04 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:04.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:04 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:05.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:04 smithi069 ceph-mon[126399]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:05.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:04 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:05.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:04 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:05.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:04 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:06.829 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:06 smithi098 ceph-mon[128968]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:06.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:06 smithi156 ceph-mon[129280]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:07.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:06 smithi069 ceph-mon[126399]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:07.275 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 7 on host 'smithi098' 2023-07-06T20:15:07.275 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.272+0000 7f4fe1ffb700 1 -- 172.21.15.98:0/114192408 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4fe4060a80 con 0x7f4fd40a1150 2023-07-06T20:15:07.277 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.275+0000 7f4fead99700 1 -- 172.21.15.98:0/114192408 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4fd40a1150 msgr2=0x7f4fd40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:07.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.275+0000 7f4fead99700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4fd40a1150 0x7f4fd40a3600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8005950 tx=0x7f4fd8005f50 comp rx=0 tx=0).stop 2023-07-06T20:15:07.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.275+0000 7f4fead99700 1 -- 172.21.15.98:0/114192408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fe40ff7c0 msgr2=0x7f4fe4195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:07.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.275+0000 7f4fead99700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fe40ff7c0 0x7f4fe4195ff0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000d950 tx=0x7f4fd000dd10 comp rx=0 tx=0).stop 2023-07-06T20:15:07.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.275+0000 7f4fead99700 1 -- 172.21.15.98:0/114192408 shutdown_connections 2023-07-06T20:15:07.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.275+0000 7f4fead99700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4fd40a1150 0x7f4fd40a3600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:07.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.275+0000 7f4fead99700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe41001b0 0x7f4fe419d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:07.278 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.275+0000 7f4fead99700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fe40fee10 0x7f4fe4195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:07.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.275+0000 7f4fead99700 1 --2- 172.21.15.98:0/114192408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fe40ff7c0 0x7f4fe4195ff0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:07.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.275+0000 7f4fead99700 1 -- 172.21.15.98:0/114192408 >> 172.21.15.98:0/114192408 conn(0x7f4fe40f99b0 msgr2=0x7f4fe40fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:07.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.275+0000 7f4fead99700 1 -- 172.21.15.98:0/114192408 shutdown_connections 2023-07-06T20:15:07.279 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:07.275+0000 7f4fead99700 1 -- 172.21.15.98:0/114192408 wait complete. 2023-07-06T20:15:07.686 DEBUG:teuthology.orchestra.run.smithi098:osd.7> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.7.service 2023-07-06T20:15:07.688 INFO:tasks.cephadm:Deploying osd.8 on smithi156 with /dev/vg_nvme/lv_2... 2023-07-06T20:15:07.688 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-06T20:15:07.718 INFO:journalctl@ceph.osd.7.smithi098.stdout:-- Logs begin at Thu 2023-07-06 19:39:25 UTC. -- 2023-07-06T20:15:08.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:08.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:08.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:08.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:08.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:08.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:08.252 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:15:08.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:08.625 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:15:08 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:15:08.322+0000 7ff7cae10640 -1 osd.7 0 log_to_monitors true 2023-07-06T20:15:09.025 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:08 smithi156 ceph-mon[129280]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:09.025 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:08 smithi156 ceph-mon[129280]: from='osd.7 [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-06T20:15:09.025 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:08 smithi156 ceph-mon[129280]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-06T20:15:09.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:08 smithi098 ceph-mon[128968]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:09.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:08 smithi098 ceph-mon[128968]: from='osd.7 [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-06T20:15:09.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:08 smithi098 ceph-mon[128968]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-06T20:15:09.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:08 smithi069 ceph-mon[126399]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:09.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:08 smithi069 ceph-mon[126399]: from='osd.7 [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-06T20:15:09.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:08 smithi069 ceph-mon[126399]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-06T20:15:09.760 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-06T20:15:09.760 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-06T20:15:09.760 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-07-06T20:15:09.760 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-07-06T20:15:09.761 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00808381 s, 1.3 GB/s 2023-07-06T20:15:09.761 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 2023-07-06T20:15:09.761 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-07-06T20:15:10.197 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch daemon add osd smithi156:vg_nvme/lv_2 2023-07-06T20:15:10.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:10 smithi156 ceph-mon[129280]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-06T20:15:10.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:10 smithi156 ceph-mon[129280]: osdmap e49: 8 total, 7 up, 8 in 2023-07-06T20:15:10.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:10 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:15:10.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:10 smithi156 ceph-mon[129280]: from='osd.7 [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:15:10.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:10 smithi156 ceph-mon[129280]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:15:10.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:10 smithi156 ceph-mon[129280]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:10.625 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:15:10 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:15:10.272+0000 7ff7bfc87700 -1 osd.7 0 waiting for initial osdmap 2023-07-06T20:15:10.625 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:15:10 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:15:10.284+0000 7ff7b8a76700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:15:10.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:10 smithi098 ceph-mon[128968]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-06T20:15:10.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:10 smithi098 ceph-mon[128968]: osdmap e49: 8 total, 7 up, 8 in 2023-07-06T20:15:10.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:10 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:15:10.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:10 smithi098 ceph-mon[128968]: from='osd.7 [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:15:10.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:10 smithi098 ceph-mon[128968]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:15:10.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:10 smithi098 ceph-mon[128968]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:10.752 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:15:10.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:10 smithi069 ceph-mon[126399]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-06T20:15:10.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:10 smithi069 ceph-mon[126399]: osdmap e49: 8 total, 7 up, 8 in 2023-07-06T20:15:10.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:15:10.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:10 smithi069 ceph-mon[126399]: from='osd.7 [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:15:10.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:10 smithi069 ceph-mon[126399]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:15:10.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:10 smithi069 ceph-mon[126399]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-06T20:15:11.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-07-06T20:15:11.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: osdmap e50: 8 total, 7 up, 8 in 2023-07-06T20:15:11.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:15:11.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:15:11.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: Detected new or changed devices on smithi098 2023-07-06T20:15:11.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:11.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:11.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:11.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:11.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:11.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi098 to 5722M 2023-07-06T20:15:11.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:11.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:11.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:11.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:11.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-07-06T20:15:11.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: osdmap e50: 8 total, 7 up, 8 in 2023-07-06T20:15:11.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:15:11.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:15:11.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: Detected new or changed devices on smithi098 2023-07-06T20:15:11.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:11.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:11.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:11.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:11.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:11.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi098 to 5722M 2023-07-06T20:15:11.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:11.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:11.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:11.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:11.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-07-06T20:15:11.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: osdmap e50: 8 total, 7 up, 8 in 2023-07-06T20:15:11.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:15:11.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:15:11.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: Detected new or changed devices on smithi098 2023-07-06T20:15:11.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:11.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:11.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:11.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:11.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:11.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi098 to 5722M 2023-07-06T20:15:11.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:11.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:11.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:11.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:11.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.931+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3148822625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f7c0febb0 msgr2=0x7f2f7c0fefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:11.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.931+0000 7f2f80a8c700 1 --2- 172.21.15.156:0/3148822625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f7c0febb0 0x7f2f7c0fefc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009990 tx=0x7f2f70009ca0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.934 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.931+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3148822625 shutdown_connections 2023-07-06T20:15:11.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.931+0000 7f2f80a8c700 1 --2- 172.21.15.156:0/3148822625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7c100740 0x7f2f7c107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.931+0000 7f2f80a8c700 1 --2- 172.21.15.156:0/3148822625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f7c0febb0 0x7f2f7c0fefc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.931+0000 7f2f80a8c700 1 --2- 172.21.15.156:0/3148822625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f7c0ffdb0 0x7f2f7c100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.931+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3148822625 >> 172.21.15.156:0/3148822625 conn(0x7f2f7c0fa1e0 msgr2=0x7f2f7c0fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:11.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.932+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3148822625 shutdown_connections 2023-07-06T20:15:11.935 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.932+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3148822625 wait complete. 2023-07-06T20:15:11.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.932+0000 7f2f80a8c700 1 Processor -- start 2023-07-06T20:15:11.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.932+0000 7f2f80a8c700 1 -- start start 2023-07-06T20:15:11.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f80a8c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7c0febb0 0x7f2f7c069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f80a8c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f7c0ffdb0 0x7f2f7c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f80a8c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f7c100740 0x7f2f7c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.937 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f80a8c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c19d560 con 0x7f2f7c100740 2023-07-06T20:15:11.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f80a8c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c19d6d0 con 0x7f2f7c0febb0 2023-07-06T20:15:11.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f80a8c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c19d890 con 0x7f2f7c0ffdb0 2023-07-06T20:15:11.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f79d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f7c0ffdb0 0x7f2f7c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:11.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f79d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f7c0ffdb0 0x7f2f7c19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59906/0 (socket says 172.21.15.156:59906) 2023-07-06T20:15:11.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f79d9b700 1 -- 172.21.15.156:0/3407894730 learned_addr learned my addr 172.21.15.156:0/3407894730 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-06T20:15:11.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f7a59c700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7c0febb0 0x7f2f7c069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:11.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f79d9b700 1 -- 172.21.15.156:0/3407894730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7c0febb0 msgr2=0x7f2f7c069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:11.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f79d9b700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7c0febb0 0x7f2f7c069550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f79d9b700 1 -- 172.21.15.156:0/3407894730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f7c100740 msgr2=0x7f2f7c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:11.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f79d9b700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f7c100740 0x7f2f7c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:11.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f79d9b700 1 -- 172.21.15.156:0/3407894730 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f700095f0 con 0x7f2f7c0ffdb0 2023-07-06T20:15:11.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f79d9b700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f7c0ffdb0 0x7f2f7c19a730 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00c920 tx=0x7f2f6c00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:11.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f6b7fe700 1 -- 172.21.15.156:0/3407894730 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c007b00 con 0x7f2f7c0ffdb0 2023-07-06T20:15:11.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f6b7fe700 1 -- 172.21.15.156:0/3407894730 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f6c00ce70 con 0x7f2f7c0ffdb0 2023-07-06T20:15:11.939 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3407894730 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7c19db70 con 0x7f2f7c0ffdb0 2023-07-06T20:15:11.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f6b7fe700 1 -- 172.21.15.156:0/3407894730 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0054e0 con 0x7f2f7c0ffdb0 2023-07-06T20:15:11.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.933+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3407894730 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c19e070 con 0x7f2f7c0ffdb0 2023-07-06T20:15:11.940 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.934+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3407894730 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7c10a3f0 con 0x7f2f7c0ffdb0 2023-07-06T20:15:11.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.936+0000 7f2f6b7fe700 1 -- 172.21.15.156:0/3407894730 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2f6c004750 con 0x7f2f7c0ffdb0 2023-07-06T20:15:11.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.936+0000 7f2f6b7fe700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f2f640a1150 0x7f2f640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:11.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.937+0000 7f2f6b7fe700 1 -- 172.21.15.156:0/3407894730 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5430+0+0 (secure 0 0 0) 0x7f2f6c0cbd20 con 0x7f2f7c0ffdb0 2023-07-06T20:15:11.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.937+0000 7f2f7a59c700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f2f640a1150 0x7f2f640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:11.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.937+0000 7f2f7a59c700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f2f640a1150 0x7f2f640a3600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000b5c0 tx=0x7f2f70005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:11.942 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:11.940+0000 7f2f6b7fe700 1 -- 172.21.15.156:0/3407894730 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2f6c095fb0 con 0x7f2f7c0ffdb0 2023-07-06T20:15:12.136 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:12.134+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3407894730 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f7c060a80 con 0x7f2f640a1150 2023-07-06T20:15:12.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:12 smithi156 ceph-mon[129280]: purged_snaps scrub starts 2023-07-06T20:15:12.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:12 smithi156 ceph-mon[129280]: purged_snaps scrub ok 2023-07-06T20:15:12.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:12 smithi156 ceph-mon[129280]: osd.7 [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140] boot 2023-07-06T20:15:12.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:12 smithi156 ceph-mon[129280]: osdmap e51: 8 total, 8 up, 8 in 2023-07-06T20:15:12.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:15:12.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:12 smithi156 ceph-mon[129280]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:12.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:12 smithi156 ceph-mon[129280]: osdmap e52: 8 total, 8 up, 8 in 2023-07-06T20:15:12.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:15:12.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:15:12.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:12.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:12 smithi098 ceph-mon[128968]: purged_snaps scrub starts 2023-07-06T20:15:12.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:12 smithi098 ceph-mon[128968]: purged_snaps scrub ok 2023-07-06T20:15:12.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:12 smithi098 ceph-mon[128968]: osd.7 [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140] boot 2023-07-06T20:15:12.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:12 smithi098 ceph-mon[128968]: osdmap e51: 8 total, 8 up, 8 in 2023-07-06T20:15:12.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:15:12.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:12 smithi098 ceph-mon[128968]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:12.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:12 smithi098 ceph-mon[128968]: osdmap e52: 8 total, 8 up, 8 in 2023-07-06T20:15:12.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:15:12.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:15:12.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:12.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:12 smithi069 ceph-mon[126399]: purged_snaps scrub starts 2023-07-06T20:15:12.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:12 smithi069 ceph-mon[126399]: purged_snaps scrub ok 2023-07-06T20:15:12.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:12 smithi069 ceph-mon[126399]: osd.7 [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140] boot 2023-07-06T20:15:12.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:12 smithi069 ceph-mon[126399]: osdmap e51: 8 total, 8 up, 8 in 2023-07-06T20:15:12.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-06T20:15:12.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:12 smithi069 ceph-mon[126399]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:12.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:12 smithi069 ceph-mon[126399]: osdmap e52: 8 total, 8 up, 8 in 2023-07-06T20:15:12.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:15:12.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:15:12.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:13.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:13 smithi156 ceph-mon[129280]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:15:13.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:13 smithi156 ceph-mon[129280]: osdmap e53: 8 total, 8 up, 8 in 2023-07-06T20:15:13.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:13 smithi098 ceph-mon[128968]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:15:13.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:13 smithi098 ceph-mon[128968]: osdmap e53: 8 total, 8 up, 8 in 2023-07-06T20:15:13.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:13 smithi069 ceph-mon[126399]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:15:13.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:13 smithi069 ceph-mon[126399]: osdmap e53: 8 total, 8 up, 8 in 2023-07-06T20:15:14.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:14 smithi156 ceph-mon[129280]: pgmap v171: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:14.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:14 smithi098 ceph-mon[128968]: pgmap v171: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:14.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:14 smithi069 ceph-mon[126399]: pgmap v171: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:15.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:15 smithi156 ceph-mon[129280]: osdmap e54: 8 total, 8 up, 8 in 2023-07-06T20:15:15.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:15 smithi156 ceph-mon[129280]: from='client.? 172.21.15.156:0/3610662427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8288c36a-d97e-4279-95ac-a215817b4ff7"}]: dispatch 2023-07-06T20:15:15.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:15 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8288c36a-d97e-4279-95ac-a215817b4ff7"}]: dispatch 2023-07-06T20:15:15.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:15 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8288c36a-d97e-4279-95ac-a215817b4ff7"}]': finished 2023-07-06T20:15:15.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:15 smithi156 ceph-mon[129280]: osdmap e55: 9 total, 8 up, 9 in 2023-07-06T20:15:15.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:15 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:15.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:15 smithi156 ceph-mon[129280]: from='client.? 172.21.15.156:0/676094845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:15:15.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:15 smithi098 ceph-mon[128968]: osdmap e54: 8 total, 8 up, 8 in 2023-07-06T20:15:15.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:15 smithi098 ceph-mon[128968]: from='client.? 172.21.15.156:0/3610662427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8288c36a-d97e-4279-95ac-a215817b4ff7"}]: dispatch 2023-07-06T20:15:15.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:15 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8288c36a-d97e-4279-95ac-a215817b4ff7"}]: dispatch 2023-07-06T20:15:15.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:15 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8288c36a-d97e-4279-95ac-a215817b4ff7"}]': finished 2023-07-06T20:15:15.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:15 smithi098 ceph-mon[128968]: osdmap e55: 9 total, 8 up, 9 in 2023-07-06T20:15:15.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:15 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:15.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:15 smithi098 ceph-mon[128968]: from='client.? 172.21.15.156:0/676094845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:15:15.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:15 smithi069 ceph-mon[126399]: osdmap e54: 8 total, 8 up, 8 in 2023-07-06T20:15:15.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:15 smithi069 ceph-mon[126399]: from='client.? 172.21.15.156:0/3610662427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8288c36a-d97e-4279-95ac-a215817b4ff7"}]: dispatch 2023-07-06T20:15:15.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:15 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8288c36a-d97e-4279-95ac-a215817b4ff7"}]: dispatch 2023-07-06T20:15:15.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:15 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8288c36a-d97e-4279-95ac-a215817b4ff7"}]': finished 2023-07-06T20:15:15.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:15 smithi069 ceph-mon[126399]: osdmap e55: 9 total, 8 up, 9 in 2023-07-06T20:15:15.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:15 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:15.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:15 smithi069 ceph-mon[126399]: from='client.? 172.21.15.156:0/676094845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:15:16.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:16 smithi156 ceph-mon[129280]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:16.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:16 smithi098 ceph-mon[128968]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:16.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:16 smithi069 ceph-mon[126399]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:18.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:18 smithi098 ceph-mon[128968]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:18.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:18 smithi069 ceph-mon[126399]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:18.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:18 smithi156 ceph-mon[129280]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:20.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:20 smithi156 ceph-mon[129280]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:20.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:20 smithi098 ceph-mon[128968]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:21.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:20 smithi069 ceph-mon[126399]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:21.829 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:21 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-06T20:15:21.829 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:21 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:21.829 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:21 smithi156 ceph-mon[129280]: Deploying daemon osd.8 on smithi156 2023-07-06T20:15:21.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:21 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-06T20:15:21.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:21 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:21.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:21 smithi098 ceph-mon[128968]: Deploying daemon osd.8 on smithi156 2023-07-06T20:15:22.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:21 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-06T20:15:22.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:21 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:22.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:21 smithi069 ceph-mon[126399]: Deploying daemon osd.8 on smithi156 2023-07-06T20:15:22.744 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:22 smithi156 ceph-mon[129280]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:22.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:22 smithi098 ceph-mon[128968]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:23.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:22 smithi069 ceph-mon[126399]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:24.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:24 smithi156 ceph-mon[129280]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:24.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:24 smithi098 ceph-mon[128968]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:25.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:24 smithi069 ceph-mon[126399]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:26.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:26 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:26.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:26 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:26.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:26 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:26.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:26 smithi156 ceph-mon[129280]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:26.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:26 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:26.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:26 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:26.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:26 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:26.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:26 smithi098 ceph-mon[128968]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:26.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:26 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:26.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:26 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:26.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:26 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:26.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:26 smithi069 ceph-mon[126399]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:28.302 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 8 on host 'smithi156' 2023-07-06T20:15:28.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.300+0000 7f2f6b7fe700 1 -- 172.21.15.156:0/3407894730 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2f7c060a80 con 0x7f2f640a1150 2023-07-06T20:15:28.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.302+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3407894730 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f2f640a1150 msgr2=0x7f2f640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.302+0000 7f2f80a8c700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f2f640a1150 0x7f2f640a3600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000b5c0 tx=0x7f2f70005fd0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.302+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3407894730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f7c0ffdb0 msgr2=0x7f2f7c19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:28.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.302+0000 7f2f80a8c700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f7c0ffdb0 0x7f2f7c19a730 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00c920 tx=0x7f2f6c00cc30 comp rx=0 tx=0).stop 2023-07-06T20:15:28.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.302+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3407894730 shutdown_connections 2023-07-06T20:15:28.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.302+0000 7f2f80a8c700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f2f640a1150 0x7f2f640a3600 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.302+0000 7f2f80a8c700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f7c100740 0x7f2f7c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.302+0000 7f2f80a8c700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7c0febb0 0x7f2f7c069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.302+0000 7f2f80a8c700 1 --2- 172.21.15.156:0/3407894730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f7c0ffdb0 0x7f2f7c19a730 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:28.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.302+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3407894730 >> 172.21.15.156:0/3407894730 conn(0x7f2f7c0fa1e0 msgr2=0x7f2f7c0fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:28.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.302+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3407894730 shutdown_connections 2023-07-06T20:15:28.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:15:28.302+0000 7f2f80a8c700 1 -- 172.21.15.156:0/3407894730 wait complete. 2023-07-06T20:15:28.742 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:28 smithi156 ceph-mon[129280]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:28.742 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:28 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:28.742 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:28 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:28.742 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:28 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:28.742 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:28 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:28.742 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:28 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:28.743 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:28 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:28.743 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:28 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:28.743 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:28 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:28.744 DEBUG:teuthology.orchestra.run.smithi156:osd.8> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.8.service 2023-07-06T20:15:28.746 INFO:tasks.cephadm:Deploying osd.9 on smithi069 with /dev/vg_nvme/lv_1... 2023-07-06T20:15:28.746 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-06T20:15:28.777 INFO:journalctl@ceph.osd.8.smithi156.stdout:-- Logs begin at Thu 2023-07-06 19:42:21 UTC. -- 2023-07-06T20:15:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:28 smithi069 ceph-mon[126399]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:28 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:28 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:29.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:28 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:29.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:28 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:29.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:28 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:29.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:28 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:29.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:28 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:29.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:28 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:29.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:28 smithi098 ceph-mon[128968]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:29.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:28 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:29.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:28 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:29.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:28 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:29.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:28 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:29.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:28 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:29.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:28 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:29.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:28 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:29.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:28 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:29.481 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:15:29.536 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:15:29 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:15:29.221+0000 7efd4402f640 -1 osd.8 0 log_to_monitors true 2023-07-06T20:15:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:29 smithi156 ceph-mon[129280]: from='osd.8 [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-06T20:15:29.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:29 smithi156 ceph-mon[129280]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-06T20:15:29.997 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:29 smithi069 ceph-mon[126399]: from='osd.8 [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-06T20:15:29.997 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:29 smithi069 ceph-mon[126399]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-06T20:15:30.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:29 smithi098 ceph-mon[128968]: from='osd.8 [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-06T20:15:30.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:29 smithi098 ceph-mon[128968]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-06T20:15:30.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:30 smithi156 ceph-mon[129280]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:30.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:30 smithi156 ceph-mon[129280]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-06T20:15:30.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:30 smithi156 ceph-mon[129280]: osdmap e56: 9 total, 8 up, 9 in 2023-07-06T20:15:30.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:30 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:30.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:30 smithi156 ceph-mon[129280]: from='osd.8 [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:15:30.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:30 smithi156 ceph-mon[129280]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:15:30.858 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:15:30 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:15:30.718+0000 7efd38ea6700 -1 osd.8 0 waiting for initial osdmap 2023-07-06T20:15:30.859 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:15:30 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:15:30.728+0000 7efd33498700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:15:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:30 smithi069 ceph-mon[126399]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:30 smithi069 ceph-mon[126399]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-06T20:15:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:30 smithi069 ceph-mon[126399]: osdmap e56: 9 total, 8 up, 9 in 2023-07-06T20:15:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:30 smithi069 ceph-mon[126399]: from='osd.8 [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:15:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:30 smithi069 ceph-mon[126399]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:15:31.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:30 smithi098 ceph-mon[128968]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:31.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:30 smithi098 ceph-mon[128968]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-06T20:15:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:30 smithi098 ceph-mon[128968]: osdmap e56: 9 total, 8 up, 9 in 2023-07-06T20:15:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:30 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:30 smithi098 ceph-mon[128968]: from='osd.8 [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:15:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:30 smithi098 ceph-mon[128968]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:15:31.382 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-06T20:15:31.382 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-06T20:15:31.382 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2023-07-06T20:15:31.382 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2023-07-06T20:15:31.383 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0475563 s, 220 MB/s 2023-07-06T20:15:31.383 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2023-07-06T20:15:31.852 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch daemon add osd smithi069:vg_nvme/lv_1 2023-07-06T20:15:32.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-06T20:15:32.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: osdmap e57: 9 total, 8 up, 9 in 2023-07-06T20:15:32.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:32.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:32.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: Detected new or changed devices on smithi156 2023-07-06T20:15:32.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:32.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:32.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:32.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:32.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:32.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi156 to 5722M 2023-07-06T20:15:32.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:32.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:32.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:32.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:31 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:32.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-06T20:15:32.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: osdmap e57: 9 total, 8 up, 9 in 2023-07-06T20:15:32.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:32.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:32.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: Detected new or changed devices on smithi156 2023-07-06T20:15:32.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:32.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:32.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:32.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:32.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:32.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi156 to 5722M 2023-07-06T20:15:32.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:32.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:32.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:32.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:31 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:32.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-06T20:15:32.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: osdmap e57: 9 total, 8 up, 9 in 2023-07-06T20:15:32.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:32.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:32.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: Detected new or changed devices on smithi156 2023-07-06T20:15:32.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:32.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:32.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:32.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:32.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:32.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi156 to 5722M 2023-07-06T20:15:32.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:32.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:32.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:32.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:31 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:32.575 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:15:33.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:32 smithi069 ceph-mon[126399]: purged_snaps scrub starts 2023-07-06T20:15:33.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:32 smithi069 ceph-mon[126399]: purged_snaps scrub ok 2023-07-06T20:15:33.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:32 smithi069 ceph-mon[126399]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:33.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:32 smithi069 ceph-mon[126399]: osd.8 [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813] boot 2023-07-06T20:15:33.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:32 smithi069 ceph-mon[126399]: osdmap e58: 9 total, 9 up, 9 in 2023-07-06T20:15:33.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:32 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:33.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:32 smithi069 ceph-mon[126399]: osdmap e59: 9 total, 9 up, 9 in 2023-07-06T20:15:33.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:32 smithi156 ceph-mon[129280]: purged_snaps scrub starts 2023-07-06T20:15:33.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:32 smithi156 ceph-mon[129280]: purged_snaps scrub ok 2023-07-06T20:15:33.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:32 smithi156 ceph-mon[129280]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:33.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:32 smithi156 ceph-mon[129280]: osd.8 [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813] boot 2023-07-06T20:15:33.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:32 smithi156 ceph-mon[129280]: osdmap e58: 9 total, 9 up, 9 in 2023-07-06T20:15:33.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:32 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:33.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:32 smithi156 ceph-mon[129280]: osdmap e59: 9 total, 9 up, 9 in 2023-07-06T20:15:33.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:32 smithi098 ceph-mon[128968]: purged_snaps scrub starts 2023-07-06T20:15:33.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:32 smithi098 ceph-mon[128968]: purged_snaps scrub ok 2023-07-06T20:15:33.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:32 smithi098 ceph-mon[128968]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-06T20:15:33.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:32 smithi098 ceph-mon[128968]: osd.8 [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813] boot 2023-07-06T20:15:33.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:32 smithi098 ceph-mon[128968]: osdmap e58: 9 total, 9 up, 9 in 2023-07-06T20:15:33.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:32 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-06T20:15:33.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:32 smithi098 ceph-mon[128968]: osdmap e59: 9 total, 9 up, 9 in 2023-07-06T20:15:34.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.006+0000 7f9485944700 1 -- 172.21.15.69:0/3792157280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94800ffcc0 msgr2=0x7f94801000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.006+0000 7f9485944700 1 --2- 172.21.15.69:0/3792157280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94800ffcc0 0x7f94801000d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9470009990 tx=0x7f9470009ca0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.006+0000 7f9485944700 1 -- 172.21.15.69:0/3792157280 shutdown_connections 2023-07-06T20:15:34.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.006+0000 7f9485944700 1 --2- 172.21.15.69:0/3792157280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9480101850 0x7f9480108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.006+0000 7f9485944700 1 --2- 172.21.15.69:0/3792157280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94800ffcc0 0x7f94801000d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.006+0000 7f9485944700 1 --2- 172.21.15.69:0/3792157280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9480100ec0 0x7f9480101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.006+0000 7f9485944700 1 -- 172.21.15.69:0/3792157280 >> 172.21.15.69:0/3792157280 conn(0x7f94800fb2d0 msgr2=0x7f94800fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:34.010 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.006+0000 7f9485944700 1 -- 172.21.15.69:0/3792157280 shutdown_connections 2023-07-06T20:15:34.010 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.006+0000 7f9485944700 1 -- 172.21.15.69:0/3792157280 wait complete. 2023-07-06T20:15:34.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.007+0000 7f9485944700 1 Processor -- start 2023-07-06T20:15:34.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f9485944700 1 -- start start 2023-07-06T20:15:34.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f9485944700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800ffcc0 0x7f9480197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f9485944700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9480100ec0 0x7f94801980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f9485944700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9480101850 0x7f948019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f9485944700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948019f740 con 0x7f9480100ec0 2023-07-06T20:15:34.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f9485944700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948019f880 con 0x7f94800ffcc0 2023-07-06T20:15:34.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f9485944700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948019fa10 con 0x7f9480101850 2023-07-06T20:15:34.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f947e7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9480100ec0 0x7f94801980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f947effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800ffcc0 0x7f9480197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f947effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800ffcc0 0x7f9480197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:55680/0 (socket says 172.21.15.69:55680) 2023-07-06T20:15:34.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f947e7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9480100ec0 0x7f94801980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45250/0 (socket says 172.21.15.69:45250) 2023-07-06T20:15:34.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f947effd700 1 -- 172.21.15.69:0/1696869264 learned_addr learned my addr 172.21.15.69:0/1696869264 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:15:34.013 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.008+0000 7f947f7fe700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9480101850 0x7f948019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.013 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.009+0000 7f947effd700 1 -- 172.21.15.69:0/1696869264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9480101850 msgr2=0x7f948019f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.013 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.009+0000 7f947effd700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9480101850 0x7f948019f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.013 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.009+0000 7f947effd700 1 -- 172.21.15.69:0/1696869264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9480100ec0 msgr2=0x7f94801980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:34.013 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.009+0000 7f947effd700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9480100ec0 0x7f94801980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:34.013 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.009+0000 7f947effd700 1 -- 172.21.15.69:0/1696869264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700095f0 con 0x7f94800ffcc0 2023-07-06T20:15:34.013 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.009+0000 7f947e7fc700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9480100ec0 0x7f94801980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:15:34.014 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.009+0000 7f947effd700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800ffcc0 0x7f9480197bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9470004710 tx=0x7f9470009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:34.014 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.009+0000 7f9484942700 1 -- 172.21.15.69:0/1696869264 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947001d070 con 0x7f94800ffcc0 2023-07-06T20:15:34.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.009+0000 7f9485944700 1 -- 172.21.15.69:0/1696869264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948019fc90 con 0x7f94800ffcc0 2023-07-06T20:15:34.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.009+0000 7f9484942700 1 -- 172.21.15.69:0/1696869264 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9470021410 con 0x7f94800ffcc0 2023-07-06T20:15:34.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.009+0000 7f9484942700 1 -- 172.21.15.69:0/1696869264 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470017650 con 0x7f94800ffcc0 2023-07-06T20:15:34.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.009+0000 7f9485944700 1 -- 172.21.15.69:0/1696869264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94801a0100 con 0x7f94800ffcc0 2023-07-06T20:15:34.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.010+0000 7f9485944700 1 -- 172.21.15.69:0/1696869264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9480192340 con 0x7f94800ffcc0 2023-07-06T20:15:34.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.013+0000 7f9484942700 1 -- 172.21.15.69:0/1696869264 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9470021580 con 0x7f94800ffcc0 2023-07-06T20:15:34.016 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.014+0000 7f9484942700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f94680a1150 0x7f94680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:34.016 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.014+0000 7f9484942700 1 -- 172.21.15.69:0/1696869264 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f94700cd6b0 con 0x7f94800ffcc0 2023-07-06T20:15:34.016 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.014+0000 7f947e7fc700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f94680a1150 0x7f94680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:34.017 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.015+0000 7f947e7fc700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f94680a1150 0x7f94680a3600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9474011490 tx=0x7f9474009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:34.018 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.016+0000 7f9484942700 1 -- 172.21.15.69:0/1696869264 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9470097780 con 0x7f94800ffcc0 2023-07-06T20:15:34.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:34.220+0000 7f9485944700 1 -- 172.21.15.69:0/1696869264 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9480060a80 con 0x7f94680a1150 2023-07-06T20:15:34.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:34 smithi069 ceph-mon[126399]: osdmap e60: 9 total, 9 up, 9 in 2023-07-06T20:15:34.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:34 smithi069 ceph-mon[126399]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:34.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:34 smithi156 ceph-mon[129280]: osdmap e60: 9 total, 9 up, 9 in 2023-07-06T20:15:34.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:34 smithi156 ceph-mon[129280]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:34.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:34 smithi098 ceph-mon[128968]: osdmap e60: 9 total, 9 up, 9 in 2023-07-06T20:15:34.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:34 smithi098 ceph-mon[128968]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:35.427 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:35 smithi069 ceph-mon[126399]: osdmap e61: 9 total, 9 up, 9 in 2023-07-06T20:15:35.427 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:35 smithi069 ceph-mon[126399]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:15:35.427 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:35 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:15:35.427 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:35 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:15:35.428 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:35 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:35.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:35 smithi156 ceph-mon[129280]: osdmap e61: 9 total, 9 up, 9 in 2023-07-06T20:15:35.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:35 smithi156 ceph-mon[129280]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:15:35.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:35 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:15:35.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:35 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:15:35.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:35 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:35.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:35 smithi098 ceph-mon[128968]: osdmap e61: 9 total, 9 up, 9 in 2023-07-06T20:15:35.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:35 smithi098 ceph-mon[128968]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:15:35.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:35 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:15:35.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:35 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:15:35.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:35 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:36.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:36 smithi069 ceph-mon[126399]: osdmap e62: 9 total, 9 up, 9 in 2023-07-06T20:15:36.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:36 smithi069 ceph-mon[126399]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:36.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:36 smithi156 ceph-mon[129280]: osdmap e62: 9 total, 9 up, 9 in 2023-07-06T20:15:36.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:36 smithi156 ceph-mon[129280]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:36.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:36 smithi098 ceph-mon[128968]: osdmap e62: 9 total, 9 up, 9 in 2023-07-06T20:15:36.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:36 smithi098 ceph-mon[128968]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:38.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:38 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2251223926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fc7fc17-09f7-4d5c-b267-46605c603c75"}]: dispatch 2023-07-06T20:15:38.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:38 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2251223926' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fc7fc17-09f7-4d5c-b267-46605c603c75"}]': finished 2023-07-06T20:15:38.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:38 smithi069 ceph-mon[126399]: osdmap e63: 10 total, 9 up, 10 in 2023-07-06T20:15:38.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:38 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:38.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:38 smithi069 ceph-mon[126399]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-07-06T20:15:38.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:38 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1292948142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:15:38.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:38 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2251223926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fc7fc17-09f7-4d5c-b267-46605c603c75"}]: dispatch 2023-07-06T20:15:38.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:38 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2251223926' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fc7fc17-09f7-4d5c-b267-46605c603c75"}]': finished 2023-07-06T20:15:38.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:38 smithi156 ceph-mon[129280]: osdmap e63: 10 total, 9 up, 10 in 2023-07-06T20:15:38.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:38.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:38 smithi156 ceph-mon[129280]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-07-06T20:15:38.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:38 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1292948142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:15:38.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:38 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2251223926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fc7fc17-09f7-4d5c-b267-46605c603c75"}]: dispatch 2023-07-06T20:15:38.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:38 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2251223926' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fc7fc17-09f7-4d5c-b267-46605c603c75"}]': finished 2023-07-06T20:15:38.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:38 smithi098 ceph-mon[128968]: osdmap e63: 10 total, 9 up, 10 in 2023-07-06T20:15:38.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:38 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:38.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:38 smithi098 ceph-mon[128968]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-07-06T20:15:38.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:38 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1292948142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:15:40.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:40 smithi156 ceph-mon[129280]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-06T20:15:40.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:40 smithi098 ceph-mon[128968]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-06T20:15:41.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:40 smithi069 ceph-mon[126399]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-06T20:15:42.826 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:42 smithi069 ceph-mon[126399]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-06T20:15:42.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:42 smithi156 ceph-mon[129280]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-06T20:15:42.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:42 smithi098 ceph-mon[128968]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-06T20:15:44.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:44 smithi156 ceph-mon[129280]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-07-06T20:15:44.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:44 smithi098 ceph-mon[128968]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-07-06T20:15:45.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:44 smithi069 ceph-mon[126399]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-07-06T20:15:45.817 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:45 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-06T20:15:45.817 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:45 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:45.817 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:45 smithi069 ceph-mon[126399]: Deploying daemon osd.9 on smithi069 2023-07-06T20:15:45.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:45 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-06T20:15:45.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:45 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:45.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:45 smithi156 ceph-mon[129280]: Deploying daemon osd.9 on smithi069 2023-07-06T20:15:45.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:45 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-06T20:15:45.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:45 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:45.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:45 smithi098 ceph-mon[128968]: Deploying daemon osd.9 on smithi069 2023-07-06T20:15:46.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:46 smithi156 ceph-mon[129280]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-06T20:15:46.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:46 smithi098 ceph-mon[128968]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-06T20:15:47.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:46 smithi069 ceph-mon[126399]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-06T20:15:48.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:48 smithi069 ceph-mon[126399]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:48.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:48 smithi156 ceph-mon[129280]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:48.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:48 smithi098 ceph-mon[128968]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:50.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:50 smithi069 ceph-mon[126399]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:50.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:50 smithi156 ceph-mon[129280]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:50.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:50 smithi098 ceph-mon[128968]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:52.176 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:52.176 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:52.177 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:52.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:52.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:52.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:52.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:52.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:52.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:53.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:52 smithi069 ceph-mon[126399]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:53.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:52 smithi156 ceph-mon[129280]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:53.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:52 smithi098 ceph-mon[128968]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:54.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.480+0000 7f9484942700 1 -- 172.21.15.69:0/1696869264 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9480060a80 con 0x7f94680a1150 2023-07-06T20:15:54.483 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 9 on host 'smithi069' 2023-07-06T20:15:54.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.483+0000 7f9485944700 1 -- 172.21.15.69:0/1696869264 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f94680a1150 msgr2=0x7f94680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:54.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.483+0000 7f9485944700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f94680a1150 0x7f94680a3600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9474011490 tx=0x7f9474009260 comp rx=0 tx=0).stop 2023-07-06T20:15:54.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.483+0000 7f9485944700 1 -- 172.21.15.69:0/1696869264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800ffcc0 msgr2=0x7f9480197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:54.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.483+0000 7f9485944700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800ffcc0 0x7f9480197bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9470004710 tx=0x7f9470009580 comp rx=0 tx=0).stop 2023-07-06T20:15:54.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.483+0000 7f9485944700 1 -- 172.21.15.69:0/1696869264 shutdown_connections 2023-07-06T20:15:54.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.483+0000 7f9485944700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f94680a1150 0x7f94680a3600 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.483+0000 7f9485944700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9480101850 0x7f948019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.483+0000 7f9485944700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800ffcc0 0x7f9480197bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.483+0000 7f9485944700 1 --2- 172.21.15.69:0/1696869264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9480100ec0 0x7f94801980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:54.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.483+0000 7f9485944700 1 -- 172.21.15.69:0/1696869264 >> 172.21.15.69:0/1696869264 conn(0x7f94800fb2d0 msgr2=0x7f9480106840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:54.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.483+0000 7f9485944700 1 -- 172.21.15.69:0/1696869264 shutdown_connections 2023-07-06T20:15:54.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:15:54.483+0000 7f9485944700 1 -- 172.21.15.69:0/1696869264 wait complete. 2023-07-06T20:15:54.995 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:54 smithi069 ceph-mon[126399]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:54.995 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:54.995 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:54.996 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:54.996 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:54.996 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:54.996 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:54.996 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:54.996 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:54.996 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:54 smithi069 ceph-mon[126399]: from='osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-06T20:15:54.997 DEBUG:teuthology.orchestra.run.smithi069:osd.9> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.9.service 2023-07-06T20:15:55.002 INFO:tasks.cephadm:Deploying osd.10 on smithi098 with /dev/vg_nvme/lv_1... 2023-07-06T20:15:55.002 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-06T20:15:55.079 INFO:journalctl@ceph.osd.9.smithi069.stdout:-- Logs begin at Thu 2023-07-06 19:46:03 UTC. -- 2023-07-06T20:15:55.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:54 smithi156 ceph-mon[129280]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:55.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:55.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:55.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:54 smithi156 ceph-mon[129280]: from='osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-06T20:15:55.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:54 smithi098 ceph-mon[128968]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:55.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:55.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:55.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:55.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:55.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:55.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:55.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:55.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:15:55.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:54 smithi098 ceph-mon[128968]: from='osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-06T20:15:55.566 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.b/config 2023-07-06T20:15:56.490 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:56 smithi069 ceph-mon[126399]: from='osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-06T20:15:56.490 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:56 smithi069 ceph-mon[126399]: osdmap e64: 10 total, 9 up, 10 in 2023-07-06T20:15:56.490 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:56.490 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:56 smithi069 ceph-mon[126399]: from='osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:15:56.490 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:56 smithi069 ceph-mon[126399]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:56.491 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:15:56 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:15:56.472+0000 7faaefcc6700 -1 osd.9 0 waiting for initial osdmap 2023-07-06T20:15:56.491 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:15:56 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:15:56.487+0000 7faaea2b8700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:15:56.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:56 smithi156 ceph-mon[129280]: from='osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-06T20:15:56.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:56 smithi156 ceph-mon[129280]: osdmap e64: 10 total, 9 up, 10 in 2023-07-06T20:15:56.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:56.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:56 smithi156 ceph-mon[129280]: from='osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:15:56.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:56 smithi156 ceph-mon[129280]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:56.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:56 smithi098 ceph-mon[128968]: from='osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-06T20:15:56.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:56 smithi098 ceph-mon[128968]: osdmap e64: 10 total, 9 up, 10 in 2023-07-06T20:15:56.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:56.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:56 smithi098 ceph-mon[128968]: from='osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-07-06T20:15:56.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:56 smithi098 ceph-mon[128968]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-06T20:15:57.022 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-06T20:15:57.022 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-06T20:15:57.022 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2023-07-06T20:15:57.023 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2023-07-06T20:15:57.023 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110264 s, 951 MB/s 2023-07-06T20:15:57.023 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2023-07-06T20:15:57.421 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch daemon add osd smithi098:vg_nvme/lv_1 2023-07-06T20:15:57.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:57 smithi069 ceph-mon[126399]: from='osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-07-06T20:15:57.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:57 smithi069 ceph-mon[126399]: osdmap e65: 10 total, 9 up, 10 in 2023-07-06T20:15:57.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:57.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:57.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:57 smithi156 ceph-mon[129280]: from='osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-07-06T20:15:57.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:57 smithi156 ceph-mon[129280]: osdmap e65: 10 total, 9 up, 10 in 2023-07-06T20:15:57.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:57.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:57.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:57 smithi098 ceph-mon[128968]: from='osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-07-06T20:15:57.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:57 smithi098 ceph-mon[128968]: osdmap e65: 10 total, 9 up, 10 in 2023-07-06T20:15:57.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:57.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:57.976 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.b/config 2023-07-06T20:15:58.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: purged_snaps scrub starts 2023-07-06T20:15:58.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: purged_snaps scrub ok 2023-07-06T20:15:58.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097] boot 2023-07-06T20:15:58.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: osdmap e66: 10 total, 10 up, 10 in 2023-07-06T20:15:58.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:58.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:15:58.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: Detected new or changed devices on smithi069 2023-07-06T20:15:58.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:58.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:58.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:58.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:58.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:58.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:58.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi069 to 4316M 2023-07-06T20:15:58.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:58.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:58.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:58.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:58.792 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: purged_snaps scrub starts 2023-07-06T20:15:58.792 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: purged_snaps scrub ok 2023-07-06T20:15:58.792 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097] boot 2023-07-06T20:15:58.792 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: osdmap e66: 10 total, 10 up, 10 in 2023-07-06T20:15:58.792 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:58.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:15:58.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: Detected new or changed devices on smithi069 2023-07-06T20:15:58.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:58.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:58.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:58.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:58.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:58.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:58.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi069 to 4316M 2023-07-06T20:15:58.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:58.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:58.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:58.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:58.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: purged_snaps scrub starts 2023-07-06T20:15:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: purged_snaps scrub ok 2023-07-06T20:15:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: osd.9 [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097] boot 2023-07-06T20:15:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: osdmap e66: 10 total, 10 up, 10 in 2023-07-06T20:15:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-06T20:15:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:15:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: Detected new or changed devices on smithi069 2023-07-06T20:15:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:58.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:58.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:58.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:58.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:58.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:15:58.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi069 to 4316M 2023-07-06T20:15:58.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:58.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:15:58.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:15:58.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:15:59.240 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.236+0000 7f78cd605700 1 -- 172.21.15.98:0/2781492804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c80ffdb0 msgr2=0x7f78c8100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:59.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.236+0000 7f78cd605700 1 --2- 172.21.15.98:0/2781492804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c80ffdb0 0x7f78c8100200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009990 tx=0x7f78bc009ca0 comp rx=0 tx=0).stop 2023-07-06T20:15:59.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.236+0000 7f78cd605700 1 -- 172.21.15.98:0/2781492804 shutdown_connections 2023-07-06T20:15:59.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.236+0000 7f78cd605700 1 --2- 172.21.15.98:0/2781492804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78c8100740 0x7f78c8107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:59.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.236+0000 7f78cd605700 1 --2- 172.21.15.98:0/2781492804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78c80febb0 0x7f78c80fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:59.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.236+0000 7f78cd605700 1 --2- 172.21.15.98:0/2781492804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c80ffdb0 0x7f78c8100200 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:59.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.236+0000 7f78cd605700 1 -- 172.21.15.98:0/2781492804 >> 172.21.15.98:0/2781492804 conn(0x7f78c80fa1e0 msgr2=0x7f78c80fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:15:59.241 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.237+0000 7f78cd605700 1 -- 172.21.15.98:0/2781492804 shutdown_connections 2023-07-06T20:15:59.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.237+0000 7f78cd605700 1 -- 172.21.15.98:0/2781492804 wait complete. 2023-07-06T20:15:59.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.237+0000 7f78cd605700 1 Processor -- start 2023-07-06T20:15:59.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.237+0000 7f78cd605700 1 -- start start 2023-07-06T20:15:59.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.237+0000 7f78cd605700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c80febb0 0x7f78c8069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:59.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.237+0000 7f78cd605700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78c80ffdb0 0x7f78c819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:59.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.237+0000 7f78cd605700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78c8100740 0x7f78c819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:59.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.237+0000 7f78cd605700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c819d560 con 0x7f78c80ffdb0 2023-07-06T20:15:59.242 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.237+0000 7f78cd605700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c819d6d0 con 0x7f78c80febb0 2023-07-06T20:15:59.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.237+0000 7f78cd605700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c819d890 con 0x7f78c8100740 2023-07-06T20:15:59.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.237+0000 7f78c6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c80febb0 0x7f78c8069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:59.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78c6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c80febb0 0x7f78c8069510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:44624/0 (socket says 172.21.15.98:44624) 2023-07-06T20:15:59.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78c6ffd700 1 -- 172.21.15.98:0/2459598713 learned_addr learned my addr 172.21.15.98:0/2459598713 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-07-06T20:15:59.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78c77fe700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78c8100740 0x7f78c819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:59.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78c6ffd700 1 -- 172.21.15.98:0/2459598713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78c8100740 msgr2=0x7f78c819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:59.243 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78c6ffd700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78c8100740 0x7f78c819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:59.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78c6ffd700 1 -- 172.21.15.98:0/2459598713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78c80ffdb0 msgr2=0x7f78c819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:15:59.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78c6ffd700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78c80ffdb0 0x7f78c819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:15:59.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78c6ffd700 1 -- 172.21.15.98:0/2459598713 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc0095f0 con 0x7f78c80febb0 2023-07-06T20:15:59.244 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78c6ffd700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c80febb0 0x7f78c8069510 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f78b800d980 tx=0x7f78b800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:59.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78affff700 1 -- 172.21.15.98:0/2459598713 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b800b950 con 0x7f78c80febb0 2023-07-06T20:15:59.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78cd605700 1 -- 172.21.15.98:0/2459598713 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c819db70 con 0x7f78c80febb0 2023-07-06T20:15:59.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78affff700 1 -- 172.21.15.98:0/2459598713 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78b8007620 con 0x7f78c80febb0 2023-07-06T20:15:59.245 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78affff700 1 -- 172.21.15.98:0/2459598713 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b800f3c0 con 0x7f78c80febb0 2023-07-06T20:15:59.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.238+0000 7f78cd605700 1 -- 172.21.15.98:0/2459598713 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c819e0c0 con 0x7f78c80febb0 2023-07-06T20:15:59.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.239+0000 7f78cd605700 1 -- 172.21.15.98:0/2459598713 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c810a3f0 con 0x7f78c80febb0 2023-07-06T20:15:59.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.241+0000 7f78affff700 1 -- 172.21.15.98:0/2459598713 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f78b8010640 con 0x7f78c80febb0 2023-07-06T20:15:59.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.241+0000 7f78affff700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f78b00a55e0 0x7f78b00a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:15:59.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.242+0000 7f78affff700 1 -- 172.21.15.98:0/2459598713 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f78b80cc880 con 0x7f78c80febb0 2023-07-06T20:15:59.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.242+0000 7f78c67fc700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f78b00a55e0 0x7f78b00a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:15:59.246 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.242+0000 7f78c67fc700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f78b00a55e0 0x7f78b00a7a90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0098e0 tx=0x7f78bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:15:59.247 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.245+0000 7f78affff700 1 -- 172.21.15.98:0/2459598713 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f78b8096730 con 0x7f78c80febb0 2023-07-06T20:15:59.439 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:15:59.437+0000 7f78cd605700 1 -- 172.21.15.98:0/2459598713 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f78c8060a80 con 0x7f78b00a55e0 2023-07-06T20:16:00.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:59 smithi156 ceph-mon[129280]: osdmap e67: 10 total, 10 up, 10 in 2023-07-06T20:16:00.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:59 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:16:00.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:59 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:16:00.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:15:59 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:00.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:59 smithi098 ceph-mon[128968]: osdmap e67: 10 total, 10 up, 10 in 2023-07-06T20:16:00.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:59 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:16:00.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:59 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:16:00.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:15:59 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:00.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:59 smithi069 ceph-mon[126399]: osdmap e67: 10 total, 10 up, 10 in 2023-07-06T20:16:00.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:59 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:16:00.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:59 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:16:00.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:15:59 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:00.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:00 smithi098 ceph-mon[128968]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:16:00.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:00 smithi098 ceph-mon[128968]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:01.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:00 smithi156 ceph-mon[129280]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:16:01.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:00 smithi156 ceph-mon[129280]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:01.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:00 smithi069 ceph-mon[126399]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:16:01.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:00 smithi069 ceph-mon[126399]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:03.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:02 smithi156 ceph-mon[129280]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:02 smithi156 ceph-mon[129280]: from='client.? 172.21.15.98:0/180339576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2f95a02-3a23-48bb-8b7e-3ecc462b3192"}]: dispatch 2023-07-06T20:16:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:02 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2f95a02-3a23-48bb-8b7e-3ecc462b3192"}]: dispatch 2023-07-06T20:16:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:02 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2f95a02-3a23-48bb-8b7e-3ecc462b3192"}]': finished 2023-07-06T20:16:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:02 smithi156 ceph-mon[129280]: osdmap e68: 11 total, 10 up, 11 in 2023-07-06T20:16:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:02 smithi156 ceph-mon[129280]: from='client.? 172.21.15.98:0/1714315446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:16:03.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:02 smithi098 ceph-mon[128968]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:03.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:02 smithi098 ceph-mon[128968]: from='client.? 172.21.15.98:0/180339576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2f95a02-3a23-48bb-8b7e-3ecc462b3192"}]: dispatch 2023-07-06T20:16:03.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:02 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2f95a02-3a23-48bb-8b7e-3ecc462b3192"}]: dispatch 2023-07-06T20:16:03.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:02 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2f95a02-3a23-48bb-8b7e-3ecc462b3192"}]': finished 2023-07-06T20:16:03.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:02 smithi098 ceph-mon[128968]: osdmap e68: 11 total, 10 up, 11 in 2023-07-06T20:16:03.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:03.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:02 smithi098 ceph-mon[128968]: from='client.? 172.21.15.98:0/1714315446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:16:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:02 smithi069 ceph-mon[126399]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:02 smithi069 ceph-mon[126399]: from='client.? 172.21.15.98:0/180339576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2f95a02-3a23-48bb-8b7e-3ecc462b3192"}]: dispatch 2023-07-06T20:16:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:02 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2f95a02-3a23-48bb-8b7e-3ecc462b3192"}]: dispatch 2023-07-06T20:16:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:02 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2f95a02-3a23-48bb-8b7e-3ecc462b3192"}]': finished 2023-07-06T20:16:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:02 smithi069 ceph-mon[126399]: osdmap e68: 11 total, 10 up, 11 in 2023-07-06T20:16:03.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:03.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:02 smithi069 ceph-mon[126399]: from='client.? 172.21.15.98:0/1714315446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:16:05.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:04 smithi156 ceph-mon[129280]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:05.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:04 smithi098 ceph-mon[128968]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:05.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:04 smithi069 ceph-mon[126399]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:07.065 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:06 smithi098 ceph-mon[128968]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:07.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:06 smithi156 ceph-mon[129280]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:07.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:06 smithi069 ceph-mon[126399]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:08.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:08 smithi069 ceph-mon[126399]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:08.563 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:08 smithi098 ceph-mon[128968]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:08.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:08 smithi156 ceph-mon[129280]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:09.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:09 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-06T20:16:09.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:09 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:09.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:09 smithi069 ceph-mon[126399]: Deploying daemon osd.10 on smithi098 2023-07-06T20:16:09.551 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:09 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-06T20:16:09.551 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:09 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:09.552 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:09 smithi098 ceph-mon[128968]: Deploying daemon osd.10 on smithi098 2023-07-06T20:16:09.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:09 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-06T20:16:09.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:09 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:09.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:09 smithi156 ceph-mon[129280]: Deploying daemon osd.10 on smithi098 2023-07-06T20:16:10.320 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:10 smithi098 ceph-mon[128968]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:10.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:10 smithi069 ceph-mon[126399]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:10.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:10 smithi156 ceph-mon[129280]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:12.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:12 smithi098 ceph-mon[128968]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:12.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:12 smithi156 ceph-mon[129280]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:13.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:12 smithi069 ceph-mon[126399]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:14.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:14 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:14.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:14 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:14.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:14 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:14.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:14 smithi156 ceph-mon[129280]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:14.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:14 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:14.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:14 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:14.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:14 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:14.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:14 smithi098 ceph-mon[128968]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:14.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:14.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:14.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:14.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:14 smithi069 ceph-mon[126399]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:16.314 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.311+0000 7f78affff700 1 -- 172.21.15.98:0/2459598713 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f78c8060a80 con 0x7f78b00a55e0 2023-07-06T20:16:16.314 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 10 on host 'smithi098' 2023-07-06T20:16:16.315 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.313+0000 7f78cd605700 1 -- 172.21.15.98:0/2459598713 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f78b00a55e0 msgr2=0x7f78b00a7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:16.315 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.313+0000 7f78cd605700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f78b00a55e0 0x7f78b00a7a90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0098e0 tx=0x7f78bc005fb0 comp rx=0 tx=0).stop 2023-07-06T20:16:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.313+0000 7f78cd605700 1 -- 172.21.15.98:0/2459598713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c80febb0 msgr2=0x7f78c8069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.313+0000 7f78cd605700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c80febb0 0x7f78c8069510 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f78b800d980 tx=0x7f78b800dc90 comp rx=0 tx=0).stop 2023-07-06T20:16:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.313+0000 7f78cd605700 1 -- 172.21.15.98:0/2459598713 shutdown_connections 2023-07-06T20:16:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.313+0000 7f78cd605700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f78b00a55e0 0x7f78b00a7a90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.313+0000 7f78cd605700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78c8100740 0x7f78c819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.313+0000 7f78cd605700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c80febb0 0x7f78c8069510 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.313+0000 7f78cd605700 1 --2- 172.21.15.98:0/2459598713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78c80ffdb0 0x7f78c819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.313+0000 7f78cd605700 1 -- 172.21.15.98:0/2459598713 >> 172.21.15.98:0/2459598713 conn(0x7f78c80fa1e0 msgr2=0x7f78c80fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.313+0000 7f78cd605700 1 -- 172.21.15.98:0/2459598713 shutdown_connections 2023-07-06T20:16:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2023-07-06T20:16:16.313+0000 7f78cd605700 1 -- 172.21.15.98:0/2459598713 wait complete. 2023-07-06T20:16:16.666 DEBUG:teuthology.orchestra.run.smithi098:osd.10> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.10.service 2023-07-06T20:16:16.668 INFO:tasks.cephadm:Deploying osd.11 on smithi156 with /dev/vg_nvme/lv_1... 2023-07-06T20:16:16.669 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-06T20:16:16.697 INFO:journalctl@ceph.osd.10.smithi098.stdout:-- Logs begin at Thu 2023-07-06 19:39:25 UTC. -- 2023-07-06T20:16:17.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:16 smithi156 ceph-mon[129280]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:17.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:17.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:17.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:17.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:16 smithi098 ceph-mon[128968]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:17.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:17.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:17.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:17.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.231 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:16:17.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:16 smithi069 ceph-mon[126399]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:17.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:17.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:17.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:17.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:17.495 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:16:17 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:16:17.228+0000 7fd7e2bd0640 -1 osd.10 0 log_to_monitors true 2023-07-06T20:16:18.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:18 smithi098 ceph-mon[128968]: from='osd.10 [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-06T20:16:18.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:18 smithi098 ceph-mon[128968]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-06T20:16:18.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:18 smithi098 ceph-mon[128968]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:18.407 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:18 smithi156 ceph-mon[129280]: from='osd.10 [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-06T20:16:18.408 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:18 smithi156 ceph-mon[129280]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-06T20:16:18.408 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:18 smithi156 ceph-mon[129280]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:18.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:18 smithi069 ceph-mon[126399]: from='osd.10 [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-06T20:16:18.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:18 smithi069 ceph-mon[126399]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-06T20:16:18.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:18 smithi069 ceph-mon[126399]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:18.830 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-06T20:16:18.831 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-06T20:16:18.831 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-07-06T20:16:18.831 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-07-06T20:16:18.831 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00825654 s, 1.3 GB/s 2023-07-06T20:16:18.831 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-07-06T20:16:19.144 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-06T20:16:19.145 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: osdmap e69: 11 total, 10 up, 11 in 2023-07-06T20:16:19.145 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:19.145 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='osd.10 [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:16:19.145 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:16:19.145 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: Detected new or changed devices on smithi098 2023-07-06T20:16:19.146 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:19.146 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:19.146 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:19.146 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:19.146 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:19.146 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:19.146 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi098 to 4292M 2023-07-06T20:16:19.147 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:19.147 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:19.147 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:19.147 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:19.276 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch daemon add osd smithi156:vg_nvme/lv_1 2023-07-06T20:16:19.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-06T20:16:19.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: osdmap e69: 11 total, 10 up, 11 in 2023-07-06T20:16:19.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:19.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='osd.10 [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:16:19.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:16:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: Detected new or changed devices on smithi098 2023-07-06T20:16:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:19.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi098 to 4292M 2023-07-06T20:16:19.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:19.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:19.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:19.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:19.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:16:19.144+0000 7fd7d7a47700 -1 osd.10 0 waiting for initial osdmap 2023-07-06T20:16:19.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:16:19.161+0000 7fd7d0035700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:16:19.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-06T20:16:19.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: osdmap e69: 11 total, 10 up, 11 in 2023-07-06T20:16:19.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:19.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='osd.10 [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:16:19.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-07-06T20:16:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: Detected new or changed devices on smithi098 2023-07-06T20:16:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi098 to 4292M 2023-07-06T20:16:19.628 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:19.628 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:19.628 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:19.628 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:19.829 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:16:20.352 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:20 smithi156 ceph-mon[129280]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-07-06T20:16:20.353 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:20 smithi156 ceph-mon[129280]: osdmap e70: 11 total, 10 up, 11 in 2023-07-06T20:16:20.353 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:20.353 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:20.353 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:20 smithi156 ceph-mon[129280]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:20.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:20 smithi069 ceph-mon[126399]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-07-06T20:16:20.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:20 smithi069 ceph-mon[126399]: osdmap e70: 11 total, 10 up, 11 in 2023-07-06T20:16:20.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:20.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:20.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:20 smithi069 ceph-mon[126399]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:20.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:20 smithi098 ceph-mon[128968]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-07-06T20:16:20.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:20 smithi098 ceph-mon[128968]: osdmap e70: 11 total, 10 up, 11 in 2023-07-06T20:16:20.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:20.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:20.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:20 smithi098 ceph-mon[128968]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-06T20:16:21.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.038+0000 7f9f09a4f700 1 -- 172.21.15.156:0/2407136733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f041001a0 msgr2=0x7f9f041005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:21.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.038+0000 7f9f09a4f700 1 --2- 172.21.15.156:0/2407136733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f041001a0 0x7f9f041005b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9ef40099c0 tx=0x7f9ef4009cd0 comp rx=0 tx=0).stop 2023-07-06T20:16:21.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.038+0000 7f9f09a4f700 1 -- 172.21.15.156:0/2407136733 shutdown_connections 2023-07-06T20:16:21.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.038+0000 7f9f09a4f700 1 --2- 172.21.15.156:0/2407136733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f040ff870 0x7f9f04107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:21.041 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.038+0000 7f9f09a4f700 1 --2- 172.21.15.156:0/2407136733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f041001a0 0x7f9f041005b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:21.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.038+0000 7f9f09a4f700 1 --2- 172.21.15.156:0/2407136733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f040feee0 0x7f9f040ff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:21.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.038+0000 7f9f09a4f700 1 -- 172.21.15.156:0/2407136733 >> 172.21.15.156:0/2407136733 conn(0x7f9f040f99c0 msgr2=0x7f9f040fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:21.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.038+0000 7f9f09a4f700 1 -- 172.21.15.156:0/2407136733 shutdown_connections 2023-07-06T20:16:21.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.038+0000 7f9f09a4f700 1 -- 172.21.15.156:0/2407136733 wait complete. 2023-07-06T20:16:21.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.038+0000 7f9f09a4f700 1 Processor -- start 2023-07-06T20:16:21.042 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f09a4f700 1 -- start start 2023-07-06T20:16:21.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f09a4f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f040feee0 0x7f9f040fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:21.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f09a4f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f040ff870 0x7f9f0419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:21.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f09a4f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f041001a0 0x7f9f0419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:21.044 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f09a4f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0419d4f0 con 0x7f9f040ff870 2023-07-06T20:16:21.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f09a4f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0419d660 con 0x7f9f040feee0 2023-07-06T20:16:21.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f09a4f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0419d820 con 0x7f9f041001a0 2023-07-06T20:16:21.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f037fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f041001a0 0x7f9f0419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:21.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f037fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f041001a0 0x7f9f0419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60058/0 (socket says 172.21.15.156:60058) 2023-07-06T20:16:21.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f037fe700 1 -- 172.21.15.156:0/943064669 learned_addr learned my addr 172.21.15.156:0/943064669 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-06T20:16:21.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f02ffd700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f040feee0 0x7f9f040fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:21.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f027fc700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f040ff870 0x7f9f0419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:21.045 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f037fe700 1 -- 172.21.15.156:0/943064669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f040feee0 msgr2=0x7f9f040fe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:21.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f037fe700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f040feee0 0x7f9f040fe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:21.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f037fe700 1 -- 172.21.15.156:0/943064669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f040ff870 msgr2=0x7f9f0419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:21.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f037fe700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f040ff870 0x7f9f0419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:21.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f037fe700 1 -- 172.21.15.156:0/943064669 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40095f0 con 0x7f9f041001a0 2023-07-06T20:16:21.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f037fe700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f041001a0 0x7f9f0419cd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9efc007d70 tx=0x7f9efc00a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:21.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f08a4d700 1 -- 172.21.15.156:0/943064669 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc00ad40 con 0x7f9f041001a0 2023-07-06T20:16:21.046 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f08a4d700 1 -- 172.21.15.156:0/943064669 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9efc0128e0 con 0x7f9f041001a0 2023-07-06T20:16:21.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.039+0000 7f9f09a4f700 1 -- 172.21.15.156:0/943064669 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0419db00 con 0x7f9f041001a0 2023-07-06T20:16:21.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.040+0000 7f9f08a4d700 1 -- 172.21.15.156:0/943064669 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0116b0 con 0x7f9f041001a0 2023-07-06T20:16:21.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.040+0000 7f9f09a4f700 1 -- 172.21.15.156:0/943064669 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0419e000 con 0x7f9f041001a0 2023-07-06T20:16:21.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.040+0000 7f9f09a4f700 1 -- 172.21.15.156:0/943064669 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0410a460 con 0x7f9f041001a0 2023-07-06T20:16:21.047 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.041+0000 7f9f08a4d700 1 -- 172.21.15.156:0/943064669 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9efc012420 con 0x7f9f041001a0 2023-07-06T20:16:21.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.042+0000 7f9f08a4d700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9eec0a11d0 0x7f9eec0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:21.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.042+0000 7f9f08a4d700 1 -- 172.21.15.156:0/943064669 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f9efc0cc4f0 con 0x7f9f041001a0 2023-07-06T20:16:21.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.042+0000 7f9f02ffd700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9eec0a11d0 0x7f9eec0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:21.048 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.043+0000 7f9f02ffd700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9eec0a11d0 0x7f9eec0a3680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009990 tx=0x7f9ef4005420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:21.049 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.047+0000 7f9f08a4d700 1 -- 172.21.15.156:0/943064669 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9efc096270 con 0x7f9f041001a0 2023-07-06T20:16:21.245 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:21.242+0000 7f9f09a4f700 1 -- 172.21.15.156:0/943064669 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f04060a80 con 0x7f9eec0a11d0 2023-07-06T20:16:21.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:21 smithi069 ceph-mon[126399]: purged_snaps scrub starts 2023-07-06T20:16:21.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:21 smithi069 ceph-mon[126399]: purged_snaps scrub ok 2023-07-06T20:16:21.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:21 smithi069 ceph-mon[126399]: osd.10 [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049] boot 2023-07-06T20:16:21.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:21 smithi069 ceph-mon[126399]: osdmap e71: 11 total, 11 up, 11 in 2023-07-06T20:16:21.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:21 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:21.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:21 smithi069 ceph-mon[126399]: osdmap e72: 11 total, 11 up, 11 in 2023-07-06T20:16:21.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:21 smithi156 ceph-mon[129280]: purged_snaps scrub starts 2023-07-06T20:16:21.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:21 smithi156 ceph-mon[129280]: purged_snaps scrub ok 2023-07-06T20:16:21.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:21 smithi156 ceph-mon[129280]: osd.10 [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049] boot 2023-07-06T20:16:21.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:21 smithi156 ceph-mon[129280]: osdmap e71: 11 total, 11 up, 11 in 2023-07-06T20:16:21.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:21 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:21.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:21 smithi156 ceph-mon[129280]: osdmap e72: 11 total, 11 up, 11 in 2023-07-06T20:16:21.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:21 smithi098 ceph-mon[128968]: purged_snaps scrub starts 2023-07-06T20:16:21.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:21 smithi098 ceph-mon[128968]: purged_snaps scrub ok 2023-07-06T20:16:21.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:21 smithi098 ceph-mon[128968]: osd.10 [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049] boot 2023-07-06T20:16:21.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:21 smithi098 ceph-mon[128968]: osdmap e71: 11 total, 11 up, 11 in 2023-07-06T20:16:21.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:21 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-06T20:16:21.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:21 smithi098 ceph-mon[128968]: osdmap e72: 11 total, 11 up, 11 in 2023-07-06T20:16:22.335 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:22 smithi156 ceph-mon[129280]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:16:22.335 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:16:22.336 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:16:22.336 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:22.336 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:22 smithi156 ceph-mon[129280]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:22.336 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:22 smithi156 ceph-mon[129280]: osdmap e73: 11 total, 11 up, 11 in 2023-07-06T20:16:22.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:22 smithi069 ceph-mon[126399]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:16:22.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:16:22.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:16:22.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:22.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:22 smithi069 ceph-mon[126399]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:22.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:22 smithi069 ceph-mon[126399]: osdmap e73: 11 total, 11 up, 11 in 2023-07-06T20:16:22.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:22 smithi098 ceph-mon[128968]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:16:22.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-06T20:16:22.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-06T20:16:22.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:22.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:22 smithi098 ceph-mon[128968]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:22.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:22 smithi098 ceph-mon[128968]: osdmap e73: 11 total, 11 up, 11 in 2023-07-06T20:16:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:24 smithi069 ceph-mon[126399]: osdmap e74: 11 total, 11 up, 11 in 2023-07-06T20:16:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:24 smithi069 ceph-mon[126399]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:24 smithi069 ceph-mon[126399]: from='client.? 172.21.15.156:0/2683794837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d0deb6c-92a0-4ae9-897e-51ddfc190bf4"}]: dispatch 2023-07-06T20:16:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:24 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d0deb6c-92a0-4ae9-897e-51ddfc190bf4"}]: dispatch 2023-07-06T20:16:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:24 smithi069 ceph-mon[126399]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d0deb6c-92a0-4ae9-897e-51ddfc190bf4"}]': finished 2023-07-06T20:16:24.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:24 smithi069 ceph-mon[126399]: osdmap e75: 12 total, 11 up, 12 in 2023-07-06T20:16:24.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:24 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:24 smithi156 ceph-mon[129280]: osdmap e74: 11 total, 11 up, 11 in 2023-07-06T20:16:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:24 smithi156 ceph-mon[129280]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:24 smithi156 ceph-mon[129280]: from='client.? 172.21.15.156:0/2683794837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d0deb6c-92a0-4ae9-897e-51ddfc190bf4"}]: dispatch 2023-07-06T20:16:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:24 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d0deb6c-92a0-4ae9-897e-51ddfc190bf4"}]: dispatch 2023-07-06T20:16:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:24 smithi156 ceph-mon[129280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d0deb6c-92a0-4ae9-897e-51ddfc190bf4"}]': finished 2023-07-06T20:16:24.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:24 smithi156 ceph-mon[129280]: osdmap e75: 12 total, 11 up, 12 in 2023-07-06T20:16:24.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:24 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:24.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:24 smithi098 ceph-mon[128968]: osdmap e74: 11 total, 11 up, 11 in 2023-07-06T20:16:24.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:24 smithi098 ceph-mon[128968]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:24.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:24 smithi098 ceph-mon[128968]: from='client.? 172.21.15.156:0/2683794837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d0deb6c-92a0-4ae9-897e-51ddfc190bf4"}]: dispatch 2023-07-06T20:16:24.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:24 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d0deb6c-92a0-4ae9-897e-51ddfc190bf4"}]: dispatch 2023-07-06T20:16:24.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:24 smithi098 ceph-mon[128968]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d0deb6c-92a0-4ae9-897e-51ddfc190bf4"}]': finished 2023-07-06T20:16:24.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:24 smithi098 ceph-mon[128968]: osdmap e75: 12 total, 11 up, 12 in 2023-07-06T20:16:24.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:24 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:25.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:25 smithi069 ceph-mon[126399]: from='client.? 172.21.15.156:0/510326183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:16:25.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:25 smithi156 ceph-mon[129280]: from='client.? 172.21.15.156:0/510326183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:16:25.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:25 smithi098 ceph-mon[128968]: from='client.? 172.21.15.156:0/510326183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-06T20:16:26.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:26 smithi069 ceph-mon[126399]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:26.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:26 smithi156 ceph-mon[129280]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:26.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:26 smithi098 ceph-mon[128968]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:28.390 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:28 smithi156 ceph-mon[129280]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-06T20:16:28.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:28 smithi069 ceph-mon[126399]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-06T20:16:28.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:28 smithi098 ceph-mon[128968]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-06T20:16:30.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:30 smithi156 ceph-mon[129280]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-06T20:16:30.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:30 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-06T20:16:30.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:30 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:30.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:30 smithi156 ceph-mon[129280]: Deploying daemon osd.11 on smithi156 2023-07-06T20:16:30.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:30 smithi098 ceph-mon[128968]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-06T20:16:30.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:30 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-06T20:16:30.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:30 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:30.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:30 smithi098 ceph-mon[128968]: Deploying daemon osd.11 on smithi156 2023-07-06T20:16:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:30 smithi069 ceph-mon[126399]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-06T20:16:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-06T20:16:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:30 smithi069 ceph-mon[126399]: Deploying daemon osd.11 on smithi156 2023-07-06T20:16:32.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:32 smithi156 ceph-mon[129280]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-07-06T20:16:32.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:32 smithi098 ceph-mon[128968]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-07-06T20:16:33.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:32 smithi069 ceph-mon[126399]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-07-06T20:16:34.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:34 smithi156 ceph-mon[129280]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-06T20:16:34.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:34 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:34.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:34 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:34.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:34 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:34.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:34 smithi098 ceph-mon[128968]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-06T20:16:34.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:34 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:34.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:34 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:34.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:34 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:34 smithi069 ceph-mon[126399]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-06T20:16:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:36.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:36 smithi156 ceph-mon[129280]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-06T20:16:36.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:36 smithi098 ceph-mon[128968]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-06T20:16:37.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:36 smithi069 ceph-mon[126399]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-06T20:16:37.357 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.355+0000 7f9f08a4d700 1 -- 172.21.15.156:0/943064669 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f9f04060a80 con 0x7f9eec0a11d0 2023-07-06T20:16:37.357 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 11 on host 'smithi156' 2023-07-06T20:16:37.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.357+0000 7f9f09a4f700 1 -- 172.21.15.156:0/943064669 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9eec0a11d0 msgr2=0x7f9eec0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:37.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.357+0000 7f9f09a4f700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9eec0a11d0 0x7f9eec0a3680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009990 tx=0x7f9ef4005420 comp rx=0 tx=0).stop 2023-07-06T20:16:37.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.357+0000 7f9f09a4f700 1 -- 172.21.15.156:0/943064669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f041001a0 msgr2=0x7f9f0419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:37.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.357+0000 7f9f09a4f700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f041001a0 0x7f9f0419cd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9efc007d70 tx=0x7f9efc00a340 comp rx=0 tx=0).stop 2023-07-06T20:16:37.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.357+0000 7f9f09a4f700 1 -- 172.21.15.156:0/943064669 shutdown_connections 2023-07-06T20:16:37.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.357+0000 7f9f09a4f700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9eec0a11d0 0x7f9eec0a3680 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:37.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.357+0000 7f9f09a4f700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f041001a0 0x7f9f0419cd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:37.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.357+0000 7f9f09a4f700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f040feee0 0x7f9f040fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:37.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.357+0000 7f9f09a4f700 1 --2- 172.21.15.156:0/943064669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f040ff870 0x7f9f0419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:37.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.357+0000 7f9f09a4f700 1 -- 172.21.15.156:0/943064669 >> 172.21.15.156:0/943064669 conn(0x7f9f040f99c0 msgr2=0x7f9f040fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:37.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.357+0000 7f9f09a4f700 1 -- 172.21.15.156:0/943064669 shutdown_connections 2023-07-06T20:16:37.360 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:37.357+0000 7f9f09a4f700 1 -- 172.21.15.156:0/943064669 wait complete. 2023-07-06T20:16:37.725 DEBUG:teuthology.orchestra.run.smithi156:osd.11> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.11.service 2023-07-06T20:16:37.727 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-06T20:16:37.728 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd stat -f json 2023-07-06T20:16:37.764 INFO:journalctl@ceph.osd.11.smithi156.stdout:-- Logs begin at Thu 2023-07-06 19:42:21 UTC. -- 2023-07-06T20:16:38.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:37 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:37 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:37 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:38.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:37 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:38.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:37 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:37 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:37 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:37 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:38.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:37 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:37 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:37 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:38.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:37 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:38.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:37 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:37 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:37 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:37 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:38.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:37 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:37 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:37 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:38.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:37 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:38.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:37 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:37 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:37 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:38.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:37 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:16:38.558 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:16:38 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:16:38.476+0000 7fab2042d640 -1 osd.11 0 log_to_monitors true 2023-07-06T20:16:38.961 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:38 smithi069 ceph-mon[126399]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-06T20:16:38.961 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:38 smithi069 ceph-mon[126399]: from='osd.11 [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-06T20:16:38.962 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:38 smithi069 ceph-mon[126399]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-06T20:16:39.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:38 smithi156 ceph-mon[129280]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-06T20:16:39.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:38 smithi156 ceph-mon[129280]: from='osd.11 [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-06T20:16:39.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:38 smithi156 ceph-mon[129280]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-06T20:16:39.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:38 smithi098 ceph-mon[128968]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-06T20:16:39.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:38 smithi098 ceph-mon[128968]: from='osd.11 [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-06T20:16:39.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:38 smithi098 ceph-mon[128968]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-06T20:16:39.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.843+0000 7fef25517700 1 -- 172.21.15.69:0/4130173809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef20100ec0 msgr2=0x7fef20101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:39.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.843+0000 7fef25517700 1 --2- 172.21.15.69:0/4130173809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef20100ec0 0x7fef20101310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef14009990 tx=0x7fef14009ca0 comp rx=0 tx=0).stop 2023-07-06T20:16:39.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.843+0000 7fef25517700 1 -- 172.21.15.69:0/4130173809 shutdown_connections 2023-07-06T20:16:39.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.843+0000 7fef25517700 1 --2- 172.21.15.69:0/4130173809 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef20101850 0x7fef20108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:39.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.843+0000 7fef25517700 1 --2- 172.21.15.69:0/4130173809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef200ffcc0 0x7fef201000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:39.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.843+0000 7fef25517700 1 --2- 172.21.15.69:0/4130173809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef20100ec0 0x7fef20101310 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:39.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.843+0000 7fef25517700 1 -- 172.21.15.69:0/4130173809 >> 172.21.15.69:0/4130173809 conn(0x7fef200fb2d0 msgr2=0x7fef200fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:39.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.843+0000 7fef25517700 1 -- 172.21.15.69:0/4130173809 shutdown_connections 2023-07-06T20:16:39.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.843+0000 7fef25517700 1 -- 172.21.15.69:0/4130173809 wait complete. 2023-07-06T20:16:39.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.844+0000 7fef25517700 1 Processor -- start 2023-07-06T20:16:39.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.844+0000 7fef25517700 1 -- start start 2023-07-06T20:16:39.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.844+0000 7fef25517700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef200ffcc0 0x7fef20195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:39.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.844+0000 7fef25517700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef20100ec0 0x7fef20195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:39.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef25517700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef20101850 0x7fef2019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:39.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef25517700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2019d5a0 con 0x7fef20101850 2023-07-06T20:16:39.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef25517700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2019d6e0 con 0x7fef200ffcc0 2023-07-06T20:16:39.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef25517700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2019d870 con 0x7fef20100ec0 2023-07-06T20:16:39.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef1effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef200ffcc0 0x7fef20195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:39.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef1effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef200ffcc0 0x7fef20195a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:50056/0 (socket says 172.21.15.69:50056) 2023-07-06T20:16:39.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef1effd700 1 -- 172.21.15.69:0/1564979684 learned_addr learned my addr 172.21.15.69:0/1564979684 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:16:39.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef1f7fe700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef20101850 0x7fef2019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:39.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef1e7fc700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef20100ec0 0x7fef20195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:39.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef1f7fe700 1 -- 172.21.15.69:0/1564979684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef20100ec0 msgr2=0x7fef20195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:39.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef1f7fe700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef20100ec0 0x7fef20195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:39.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef1f7fe700 1 -- 172.21.15.69:0/1564979684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef200ffcc0 msgr2=0x7fef20195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:39.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef1f7fe700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef200ffcc0 0x7fef20195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:39.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef1f7fe700 1 -- 172.21.15.69:0/1564979684 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef10009520 con 0x7fef20101850 2023-07-06T20:16:39.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef1effd700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef200ffcc0 0x7fef20195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:16:39.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.845+0000 7fef1f7fe700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef20101850 0x7fef2019cfd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fef18007f40 tx=0x7fef1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:39.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.846+0000 7fef07fff700 1 -- 172.21.15.69:0/1564979684 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18004db0 con 0x7fef20101850 2023-07-06T20:16:39.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.846+0000 7fef07fff700 1 -- 172.21.15.69:0/1564979684 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef1800f040 con 0x7fef20101850 2023-07-06T20:16:39.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.846+0000 7fef25517700 1 -- 172.21.15.69:0/1564979684 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef140095f0 con 0x7fef20101850 2023-07-06T20:16:39.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.846+0000 7fef07fff700 1 -- 172.21.15.69:0/1564979684 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef180059d0 con 0x7fef20101850 2023-07-06T20:16:39.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.846+0000 7fef25517700 1 -- 172.21.15.69:0/1564979684 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2019dec0 con 0x7fef20101850 2023-07-06T20:16:39.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.847+0000 7fef25517700 1 -- 172.21.15.69:0/1564979684 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2010b4d0 con 0x7fef20101850 2023-07-06T20:16:39.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.849+0000 7fef07fff700 1 -- 172.21.15.69:0/1564979684 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fef18005000 con 0x7fef20101850 2023-07-06T20:16:39.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.850+0000 7fef07fff700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fef080a11d0 0x7fef080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:39.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.850+0000 7fef07fff700 1 -- 172.21.15.69:0/1564979684 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fef180cbc30 con 0x7fef20101850 2023-07-06T20:16:39.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.850+0000 7fef1effd700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fef080a11d0 0x7fef080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:39.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.851+0000 7fef1effd700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fef080a11d0 0x7fef080a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fef10009fd0 tx=0x7fef10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:39.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:39.854+0000 7fef07fff700 1 -- 172.21.15.69:0/1564979684 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fef1804e770 con 0x7fef20101850 2023-07-06T20:16:40.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.074+0000 7fef25517700 1 -- 172.21.15.69:0/1564979684 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fef2010b690 con 0x7fef20101850 2023-07-06T20:16:40.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.074+0000 7fef07fff700 1 -- 172.21.15.69:0/1564979684 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7fef180958b0 con 0x7fef20101850 2023-07-06T20:16:40.077 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:16:40.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.077+0000 7fef25517700 1 -- 172.21.15.69:0/1564979684 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fef080a11d0 msgr2=0x7fef080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:40.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.077+0000 7fef25517700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fef080a11d0 0x7fef080a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fef10009fd0 tx=0x7fef10009260 comp rx=0 tx=0).stop 2023-07-06T20:16:40.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.077+0000 7fef25517700 1 -- 172.21.15.69:0/1564979684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef20101850 msgr2=0x7fef2019cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:40.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.077+0000 7fef25517700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef20101850 0x7fef2019cfd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fef18007f40 tx=0x7fef1800d3b0 comp rx=0 tx=0).stop 2023-07-06T20:16:40.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.077+0000 7fef25517700 1 -- 172.21.15.69:0/1564979684 shutdown_connections 2023-07-06T20:16:40.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.077+0000 7fef25517700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fef080a11d0 0x7fef080a3680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:40.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.077+0000 7fef25517700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef20101850 0x7fef2019cfd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:40.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.077+0000 7fef25517700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef200ffcc0 0x7fef20195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:40.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.077+0000 7fef25517700 1 --2- 172.21.15.69:0/1564979684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef20100ec0 0x7fef20195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:40.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.077+0000 7fef25517700 1 -- 172.21.15.69:0/1564979684 >> 172.21.15.69:0/1564979684 conn(0x7fef200fb2d0 msgr2=0x7fef200fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:40.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.077+0000 7fef25517700 1 -- 172.21.15.69:0/1564979684 shutdown_connections 2023-07-06T20:16:40.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:40.077+0000 7fef25517700 1 -- 172.21.15.69:0/1564979684 wait complete. 2023-07-06T20:16:40.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-06T20:16:40.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: osdmap e76: 12 total, 11 up, 12 in 2023-07-06T20:16:40.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:40.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='osd.11 [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:16:40.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:16:40.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:40.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: Detected new or changed devices on smithi156 2023-07-06T20:16:40.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:40.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:40.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:40.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:40.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:40.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:40.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi156 to 4292M 2023-07-06T20:16:40.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:40.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:40.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:40.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:40.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:40 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1564979684' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-06T20:16:40.534 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1688674580,"num_in_osds":12,"osd_in_since":1688674583,"num_remapped_pgs":0} 2023-07-06T20:16:40.606 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:16:40.343+0000 7fab152a4700 -1 osd.11 0 waiting for initial osdmap 2023-07-06T20:16:40.607 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:16:40.359+0000 7fab0d892700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T20:16:40.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-06T20:16:40.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: osdmap e76: 12 total, 11 up, 12 in 2023-07-06T20:16:40.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:40.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='osd.11 [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:16:40.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:16:40.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:40.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: Detected new or changed devices on smithi156 2023-07-06T20:16:40.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:40.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:40.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:40.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:40.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:40.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:40.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi156 to 4292M 2023-07-06T20:16:40.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:40.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:40.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:40.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:40.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:40 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1564979684' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-06T20:16:40.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-06T20:16:40.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: osdmap e76: 12 total, 11 up, 12 in 2023-07-06T20:16:40.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='osd.11 [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:16:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-06T20:16:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: Detected new or changed devices on smithi156 2023-07-06T20:16:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:40.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:40.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:40.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:40.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:16:40.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi156 to 4292M 2023-07-06T20:16:40.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:40.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:40.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:40.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:40.628 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:40 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1564979684' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-06T20:16:41.536 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd stat -f json 2023-07-06T20:16:41.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:41 smithi069 ceph-mon[126399]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-06T20:16:41.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:41 smithi069 ceph-mon[126399]: osdmap e77: 12 total, 11 up, 12 in 2023-07-06T20:16:41.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:41 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:41.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:41 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:41.605 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:41 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:41.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:41 smithi156 ceph-mon[129280]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-06T20:16:41.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:41 smithi156 ceph-mon[129280]: osdmap e77: 12 total, 11 up, 12 in 2023-07-06T20:16:41.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:41 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:41.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:41 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:41.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:41 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:41.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:41 smithi098 ceph-mon[128968]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-06T20:16:41.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:41 smithi098 ceph-mon[128968]: osdmap e77: 12 total, 11 up, 12 in 2023-07-06T20:16:41.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:41 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:41.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:41 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:41.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:41 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:42.268 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:16:42.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:42 smithi098 ceph-mon[128968]: purged_snaps scrub starts 2023-07-06T20:16:42.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:42 smithi098 ceph-mon[128968]: purged_snaps scrub ok 2023-07-06T20:16:42.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:42 smithi098 ceph-mon[128968]: osd.11 [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539] boot 2023-07-06T20:16:42.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:42 smithi098 ceph-mon[128968]: osdmap e78: 12 total, 12 up, 12 in 2023-07-06T20:16:42.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:42 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:42.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:42 smithi098 ceph-mon[128968]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:42.641 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:42 smithi069 ceph-mon[126399]: purged_snaps scrub starts 2023-07-06T20:16:42.642 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:42 smithi069 ceph-mon[126399]: purged_snaps scrub ok 2023-07-06T20:16:42.642 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:42 smithi069 ceph-mon[126399]: osd.11 [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539] boot 2023-07-06T20:16:42.642 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:42 smithi069 ceph-mon[126399]: osdmap e78: 12 total, 12 up, 12 in 2023-07-06T20:16:42.642 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:42 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:42.642 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:42 smithi069 ceph-mon[126399]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:42.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:42 smithi156 ceph-mon[129280]: purged_snaps scrub starts 2023-07-06T20:16:42.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:42 smithi156 ceph-mon[129280]: purged_snaps scrub ok 2023-07-06T20:16:42.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:42 smithi156 ceph-mon[129280]: osd.11 [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539] boot 2023-07-06T20:16:42.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:42 smithi156 ceph-mon[129280]: osdmap e78: 12 total, 12 up, 12 in 2023-07-06T20:16:42.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:42 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-06T20:16:42.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:42 smithi156 ceph-mon[129280]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-06T20:16:43.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:43 smithi098 ceph-mon[128968]: osdmap e79: 12 total, 12 up, 12 in 2023-07-06T20:16:43.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.668+0000 7f68204da700 1 -- 172.21.15.69:0/3461387911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818073040 msgr2=0x7f68180734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:43.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.668+0000 7f68204da700 1 --2- 172.21.15.69:0/3461387911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818073040 0x7f68180734b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6808009990 tx=0x7f6808009ca0 comp rx=0 tx=0).stop 2023-07-06T20:16:43.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.668+0000 7f68204da700 1 -- 172.21.15.69:0/3461387911 shutdown_connections 2023-07-06T20:16:43.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.668+0000 7f68204da700 1 --2- 172.21.15.69:0/3461387911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6818107b60 0x7f6818109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:43.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.668+0000 7f68204da700 1 --2- 172.21.15.69:0/3461387911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6818074610 0x7f6818072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:43.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.668+0000 7f68204da700 1 --2- 172.21.15.69:0/3461387911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818073040 0x7f68180734b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:43.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.668+0000 7f68204da700 1 -- 172.21.15.69:0/3461387911 >> 172.21.15.69:0/3461387911 conn(0x7f68180fb2d0 msgr2=0x7f68180fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:43.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.668+0000 7f68204da700 1 -- 172.21.15.69:0/3461387911 shutdown_connections 2023-07-06T20:16:43.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.668+0000 7f68204da700 1 -- 172.21.15.69:0/3461387911 wait complete. 2023-07-06T20:16:43.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.669+0000 7f68204da700 1 Processor -- start 2023-07-06T20:16:43.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.669+0000 7f68204da700 1 -- start start 2023-07-06T20:16:43.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.669+0000 7f68204da700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6818073040 0x7f6818197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:43.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.669+0000 7f68204da700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6818074610 0x7f6818198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:43.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.669+0000 7f68204da700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818107b60 0x7f681819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:43.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.669+0000 7f68204da700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681819f770 con 0x7f6818107b60 2023-07-06T20:16:43.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.669+0000 7f68204da700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681819f8b0 con 0x7f6818073040 2023-07-06T20:16:43.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.669+0000 7f68204da700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681819fa40 con 0x7f6818074610 2023-07-06T20:16:43.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.670+0000 7f681da75700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6818074610 0x7f6818198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:43.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.670+0000 7f681da75700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6818074610 0x7f6818198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:44032/0 (socket says 172.21.15.69:44032) 2023-07-06T20:16:43.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.670+0000 7f681da75700 1 -- 172.21.15.69:0/3004476019 learned_addr learned my addr 172.21.15.69:0/3004476019 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:16:43.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.670+0000 7f681ea77700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818107b60 0x7f681819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:43.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.670+0000 7f681e276700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6818073040 0x7f6818197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:43.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.670+0000 7f681da75700 1 -- 172.21.15.69:0/3004476019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6818073040 msgr2=0x7f6818197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:43.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.670+0000 7f681da75700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6818073040 0x7f6818197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:43.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.670+0000 7f681da75700 1 -- 172.21.15.69:0/3004476019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818107b60 msgr2=0x7f681819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:43.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.670+0000 7f681da75700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818107b60 0x7f681819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:43.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.670+0000 7f681da75700 1 -- 172.21.15.69:0/3004476019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68080095f0 con 0x7f6818074610 2023-07-06T20:16:43.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.670+0000 7f681da75700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6818074610 0x7f6818198120 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6808004770 tx=0x7f6808009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:43.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.670+0000 7f680f7fe700 1 -- 172.21.15.69:0/3004476019 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680801d070 con 0x7f6818074610 2023-07-06T20:16:43.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.671+0000 7f680f7fe700 1 -- 172.21.15.69:0/3004476019 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6808005030 con 0x7f6818074610 2023-07-06T20:16:43.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.671+0000 7f68204da700 1 -- 172.21.15.69:0/3004476019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681819fcc0 con 0x7f6818074610 2023-07-06T20:16:43.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.671+0000 7f680f7fe700 1 -- 172.21.15.69:0/3004476019 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808004b50 con 0x7f6818074610 2023-07-06T20:16:43.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.671+0000 7f68204da700 1 -- 172.21.15.69:0/3004476019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68181a0070 con 0x7f6818074610 2023-07-06T20:16:43.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.672+0000 7f68204da700 1 -- 172.21.15.69:0/3004476019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68181062a0 con 0x7f6818074610 2023-07-06T20:16:43.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.674+0000 7f680f7fe700 1 -- 172.21.15.69:0/3004476019 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6808004cf0 con 0x7f6818074610 2023-07-06T20:16:43.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.675+0000 7f680f7fe700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f68040a1150 0x7f68040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:43.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.675+0000 7f680f7fe700 1 -- 172.21.15.69:0/3004476019 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f68080cd660 con 0x7f6818074610 2023-07-06T20:16:43.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.675+0000 7f681e276700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f68040a1150 0x7f68040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:43.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.676+0000 7f681e276700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f68040a1150 0x7f68040a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6814009c60 tx=0x7f6814009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:43.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.677+0000 7f680f7fe700 1 -- 172.21.15.69:0/3004476019 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6808097240 con 0x7f6818074610 2023-07-06T20:16:43.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:43 smithi069 ceph-mon[126399]: osdmap e79: 12 total, 12 up, 12 in 2023-07-06T20:16:43.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:43 smithi156 ceph-mon[129280]: osdmap e79: 12 total, 12 up, 12 in 2023-07-06T20:16:43.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.874+0000 7f68204da700 1 -- 172.21.15.69:0/3004476019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f681810d740 con 0x7f6818074610 2023-07-06T20:16:43.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.875+0000 7f680f7fe700 1 -- 172.21.15.69:0/3004476019 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f680809a860 con 0x7f6818074610 2023-07-06T20:16:43.877 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:16:43.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.877+0000 7f68204da700 1 -- 172.21.15.69:0/3004476019 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f68040a1150 msgr2=0x7f68040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:43.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.877+0000 7f68204da700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f68040a1150 0x7f68040a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6814009c60 tx=0x7f6814009260 comp rx=0 tx=0).stop 2023-07-06T20:16:43.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.877+0000 7f68204da700 1 -- 172.21.15.69:0/3004476019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6818074610 msgr2=0x7f6818198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:43.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.877+0000 7f68204da700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6818074610 0x7f6818198120 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6808004770 tx=0x7f6808009580 comp rx=0 tx=0).stop 2023-07-06T20:16:43.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.877+0000 7f68204da700 1 -- 172.21.15.69:0/3004476019 shutdown_connections 2023-07-06T20:16:43.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.878+0000 7f68204da700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f68040a1150 0x7f68040a3600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:43.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.878+0000 7f68204da700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6818107b60 0x7f681819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:43.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.878+0000 7f68204da700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6818073040 0x7f6818197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:43.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.878+0000 7f68204da700 1 --2- 172.21.15.69:0/3004476019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6818074610 0x7f6818198120 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:43.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.878+0000 7f68204da700 1 -- 172.21.15.69:0/3004476019 >> 172.21.15.69:0/3004476019 conn(0x7f68180fb2d0 msgr2=0x7f68180fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:43.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.878+0000 7f68204da700 1 -- 172.21.15.69:0/3004476019 shutdown_connections 2023-07-06T20:16:43.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:43.878+0000 7f68204da700 1 -- 172.21.15.69:0/3004476019 wait complete. 2023-07-06T20:16:44.347 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1688674601,"num_in_osds":12,"osd_in_since":1688674583,"num_remapped_pgs":0} 2023-07-06T20:16:44.347 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd dump --format=json 2023-07-06T20:16:44.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:44 smithi098 ceph-mon[128968]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:44.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:44 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3004476019' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-06T20:16:44.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:44 smithi069 ceph-mon[126399]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:44.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:44 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3004476019' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-06T20:16:44.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:44 smithi156 ceph-mon[129280]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:44.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:44 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3004476019' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-06T20:16:45.067 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:16:46.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.450+0000 7fe8b689c700 1 -- 172.21.15.69:0/3328419463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0104fd0 msgr2=0x7fe8b010cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:46.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.450+0000 7fe8b689c700 1 --2- 172.21.15.69:0/3328419463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0104fd0 0x7fe8b010cb50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800b0d0 tx=0x7fe8a800b3e0 comp rx=0 tx=0).stop 2023-07-06T20:16:46.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.450+0000 7fe8b689c700 1 -- 172.21.15.69:0/3328419463 shutdown_connections 2023-07-06T20:16:46.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.450+0000 7fe8b689c700 1 --2- 172.21.15.69:0/3328419463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0104fd0 0x7fe8b010cb50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:46.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.450+0000 7fe8b689c700 1 --2- 172.21.15.69:0/3328419463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8b00fff30 0x7fe8b0100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:46.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.450+0000 7fe8b689c700 1 --2- 172.21.15.69:0/3328419463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b01008d0 0x7fe8b01048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:46.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.450+0000 7fe8b689c700 1 -- 172.21.15.69:0/3328419463 >> 172.21.15.69:0/3328419463 conn(0x7fe8b00fb790 msgr2=0x7fe8b00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:46.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.450+0000 7fe8b689c700 1 -- 172.21.15.69:0/3328419463 shutdown_connections 2023-07-06T20:16:46.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.450+0000 7fe8b689c700 1 -- 172.21.15.69:0/3328419463 wait complete. 2023-07-06T20:16:46.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.451+0000 7fe8b689c700 1 Processor -- start 2023-07-06T20:16:46.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.451+0000 7fe8b689c700 1 -- start start 2023-07-06T20:16:46.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.451+0000 7fe8b689c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b00fff30 0x7fe8b0195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:46.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b689c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8b01008d0 0x7fe8b0196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:46.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b689c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0104fd0 0x7fe8b019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:46.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b689c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b019aeb0 con 0x7fe8b0104fd0 2023-07-06T20:16:46.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b689c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b019b020 con 0x7fe8b01008d0 2023-07-06T20:16:46.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b689c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b019b1e0 con 0x7fe8b00fff30 2023-07-06T20:16:46.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b4e39700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0104fd0 0x7fe8b019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:46.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b4e39700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0104fd0 0x7fe8b019a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41800/0 (socket says 172.21.15.69:41800) 2023-07-06T20:16:46.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b4e39700 1 -- 172.21.15.69:0/1983702405 learned_addr learned my addr 172.21.15.69:0/1983702405 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:16:46.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8affff700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b00fff30 0x7fe8b0195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:46.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8af7fe700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8b01008d0 0x7fe8b0196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:46.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b4e39700 1 -- 172.21.15.69:0/1983702405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b00fff30 msgr2=0x7fe8b0195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:46.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b4e39700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b00fff30 0x7fe8b0195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:46.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b4e39700 1 -- 172.21.15.69:0/1983702405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8b01008d0 msgr2=0x7fe8b0196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:46.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b4e39700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8b01008d0 0x7fe8b0196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:46.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b4e39700 1 -- 172.21.15.69:0/1983702405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a8009d00 con 0x7fe8b0104fd0 2023-07-06T20:16:46.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.452+0000 7fe8b4e39700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0104fd0 0x7fe8b019a8a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8007870 tx=0x7fe8a80078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:46.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.453+0000 7fe8ad7fa700 1 -- 172.21.15.69:0/1983702405 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a800e040 con 0x7fe8b0104fd0 2023-07-06T20:16:46.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.453+0000 7fe8b689c700 1 -- 172.21.15.69:0/1983702405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b019b460 con 0x7fe8b0104fd0 2023-07-06T20:16:46.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.453+0000 7fe8ad7fa700 1 -- 172.21.15.69:0/1983702405 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a8003e10 con 0x7fe8b0104fd0 2023-07-06T20:16:46.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.453+0000 7fe8ad7fa700 1 -- 172.21.15.69:0/1983702405 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a801ba10 con 0x7fe8b0104fd0 2023-07-06T20:16:46.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.453+0000 7fe8b689c700 1 -- 172.21.15.69:0/1983702405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b01a2d20 con 0x7fe8b0104fd0 2023-07-06T20:16:46.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.454+0000 7fe8b689c700 1 -- 172.21.15.69:0/1983702405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b004e340 con 0x7fe8b0104fd0 2023-07-06T20:16:46.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.456+0000 7fe8ad7fa700 1 -- 172.21.15.69:0/1983702405 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe8a8024410 con 0x7fe8b0104fd0 2023-07-06T20:16:46.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.457+0000 7fe8ad7fa700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe8980a1150 0x7fe8980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:46.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.457+0000 7fe8ad7fa700 1 -- 172.21.15.69:0/1983702405 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe8a80cdb00 con 0x7fe8b0104fd0 2023-07-06T20:16:46.460 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.457+0000 7fe8affff700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe8980a1150 0x7fe8980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:46.460 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.458+0000 7fe8affff700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe8980a1150 0x7fe8980a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0005950 tx=0x7fe8a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:46.463 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.461+0000 7fe8ad7fa700 1 -- 172.21.15.69:0/1983702405 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe8a80cfe60 con 0x7fe8b0104fd0 2023-07-06T20:16:46.662 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.660+0000 7fe8b689c700 1 -- 172.21.15.69:0/1983702405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8b01a2eb0 con 0x7fe8b0104fd0 2023-07-06T20:16:46.663 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.661+0000 7fe8ad7fa700 1 -- 172.21.15.69:0/1983702405 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18368 (secure 0 0 0) 0x7fe8a80976e0 con 0x7fe8b0104fd0 2023-07-06T20:16:46.664 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:16:46.664 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":79,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:16:42.349843+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:16:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.663+0000 7fe8b689c700 1 -- 172.21.15.69:0/1983702405 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe8980a1150 msgr2=0x7fe8980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.663+0000 7fe8b689c700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe8980a1150 0x7fe8980a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0005950 tx=0x7fe8a000b410 comp rx=0 tx=0).stop 2023-07-06T20:16:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.664+0000 7fe8b689c700 1 -- 172.21.15.69:0/1983702405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0104fd0 msgr2=0x7fe8b019a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.664+0000 7fe8b689c700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0104fd0 0x7fe8b019a8a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8007870 tx=0x7fe8a80078a0 comp rx=0 tx=0).stop 2023-07-06T20:16:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.664+0000 7fe8b689c700 1 -- 172.21.15.69:0/1983702405 shutdown_connections 2023-07-06T20:16:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.664+0000 7fe8b689c700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe8980a1150 0x7fe8980a3600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.664+0000 7fe8b689c700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b0104fd0 0x7fe8b019a8a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.664+0000 7fe8b689c700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b00fff30 0x7fe8b0195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.664+0000 7fe8b689c700 1 --2- 172.21.15.69:0/1983702405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8b01008d0 0x7fe8b0196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.664+0000 7fe8b689c700 1 -- 172.21.15.69:0/1983702405 >> 172.21.15.69:0/1983702405 conn(0x7fe8b00fb790 msgr2=0x7fe8b00fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.664+0000 7fe8b689c700 1 -- 172.21.15.69:0/1983702405 shutdown_connections 2023-07-06T20:16:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:46.664+0000 7fe8b689c700 1 -- 172.21.15.69:0/1983702405 wait complete. 2023-07-06T20:16:46.825 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:46 smithi069 ceph-mon[126399]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:46.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:46 smithi156 ceph-mon[129280]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:46.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:46 smithi098 ceph-mon[128968]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:47.139 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:14.022799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:16:47.139 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-06T20:16:47.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:47 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1983702405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:16:47.860 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:16:47.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:47 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1983702405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:16:48.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:47 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1983702405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:16:48.679 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:48 smithi069 ceph-mon[126399]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:48.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:48 smithi156 ceph-mon[129280]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:48.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:48 smithi098 ceph-mon[128968]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:49.265 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.263+0000 7fcb1a08e700 1 -- 172.21.15.69:0/214825957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb140fff30 msgr2=0x7fcb14100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:49.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.263+0000 7fcb1a08e700 1 --2- 172.21.15.69:0/214825957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb140fff30 0x7fcb14100300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcb040099c0 tx=0x7fcb04009cd0 comp rx=0 tx=0).stop 2023-07-06T20:16:49.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.263+0000 7fcb1a08e700 1 -- 172.21.15.69:0/214825957 shutdown_connections 2023-07-06T20:16:49.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.263+0000 7fcb1a08e700 1 --2- 172.21.15.69:0/214825957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb14104fd0 0x7fcb1410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:49.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.263+0000 7fcb1a08e700 1 --2- 172.21.15.69:0/214825957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb140fff30 0x7fcb14100300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:49.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.263+0000 7fcb1a08e700 1 --2- 172.21.15.69:0/214825957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb141008d0 0x7fcb141048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:49.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.263+0000 7fcb1a08e700 1 -- 172.21.15.69:0/214825957 >> 172.21.15.69:0/214825957 conn(0x7fcb140fb790 msgr2=0x7fcb140fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:49.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.263+0000 7fcb1a08e700 1 -- 172.21.15.69:0/214825957 shutdown_connections 2023-07-06T20:16:49.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.263+0000 7fcb1a08e700 1 -- 172.21.15.69:0/214825957 wait complete. 2023-07-06T20:16:49.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.264+0000 7fcb1a08e700 1 Processor -- start 2023-07-06T20:16:49.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.264+0000 7fcb1a08e700 1 -- start start 2023-07-06T20:16:49.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.265+0000 7fcb1a08e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb140fff30 0x7fcb14195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:49.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.265+0000 7fcb1a08e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb141008d0 0x7fcb14196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:49.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.265+0000 7fcb1a08e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb14104fd0 0x7fcb1419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:49.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.265+0000 7fcb1a08e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1419adc0 con 0x7fcb14104fd0 2023-07-06T20:16:49.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.265+0000 7fcb1a08e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1419af30 con 0x7fcb141008d0 2023-07-06T20:16:49.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.265+0000 7fcb1a08e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1419b0f0 con 0x7fcb140fff30 2023-07-06T20:16:49.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.265+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb14104fd0 0x7fcb1419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:49.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.265+0000 7fcb137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb140fff30 0x7fcb14195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:49.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.265+0000 7fcb12ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb141008d0 0x7fcb14196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:49.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.265+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb14104fd0 0x7fcb1419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41810/0 (socket says 172.21.15.69:41810) 2023-07-06T20:16:49.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.265+0000 7fcb13fff700 1 -- 172.21.15.69:0/220220624 learned_addr learned my addr 172.21.15.69:0/220220624 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:16:49.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.266+0000 7fcb13fff700 1 -- 172.21.15.69:0/220220624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb140fff30 msgr2=0x7fcb14195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:49.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.266+0000 7fcb13fff700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb140fff30 0x7fcb14195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:49.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.266+0000 7fcb13fff700 1 -- 172.21.15.69:0/220220624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb141008d0 msgr2=0x7fcb14196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:49.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.266+0000 7fcb13fff700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb141008d0 0x7fcb14196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:49.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.266+0000 7fcb13fff700 1 -- 172.21.15.69:0/220220624 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040095f0 con 0x7fcb14104fd0 2023-07-06T20:16:49.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.266+0000 7fcb12ffd700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb141008d0 0x7fcb14196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:16:49.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.266+0000 7fcb13fff700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb14104fd0 0x7fcb1419a7b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00ef30 tx=0x7fcb0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:49.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.266+0000 7fcb10ff9700 1 -- 172.21.15.69:0/220220624 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c018070 con 0x7fcb14104fd0 2023-07-06T20:16:49.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.266+0000 7fcb1a08e700 1 -- 172.21.15.69:0/220220624 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1419b370 con 0x7fcb14104fd0 2023-07-06T20:16:49.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.266+0000 7fcb10ff9700 1 -- 172.21.15.69:0/220220624 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb0c00f040 con 0x7fcb14104fd0 2023-07-06T20:16:49.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.266+0000 7fcb10ff9700 1 -- 172.21.15.69:0/220220624 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00cd00 con 0x7fcb14104fd0 2023-07-06T20:16:49.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.266+0000 7fcb1a08e700 1 -- 172.21.15.69:0/220220624 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb141a2c30 con 0x7fcb14104fd0 2023-07-06T20:16:49.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.267+0000 7fcb1a08e700 1 -- 172.21.15.69:0/220220624 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1404e340 con 0x7fcb14104fd0 2023-07-06T20:16:49.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.269+0000 7fcb10ff9700 1 -- 172.21.15.69:0/220220624 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcb0c003a00 con 0x7fcb14104fd0 2023-07-06T20:16:49.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.271+0000 7fcb10ff9700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcafc0a1150 0x7fcafc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:49.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.271+0000 7fcb10ff9700 1 -- 172.21.15.69:0/220220624 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcb0c0cbf80 con 0x7fcb14104fd0 2023-07-06T20:16:49.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.271+0000 7fcb137fe700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcafc0a1150 0x7fcafc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:49.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.272+0000 7fcb137fe700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcafc0a1150 0x7fcafc0a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009990 tx=0x7fcb04005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:49.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.275+0000 7fcb10ff9700 1 -- 172.21.15.69:0/220220624 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcb0c095b60 con 0x7fcb14104fd0 2023-07-06T20:16:49.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.470+0000 7fcb1a08e700 1 -- 172.21.15.69:0/220220624 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb141a2ec0 con 0x7fcb14104fd0 2023-07-06T20:16:49.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.470+0000 7fcb10ff9700 1 -- 172.21.15.69:0/220220624 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb0c099180 con 0x7fcb14104fd0 2023-07-06T20:16:49.473 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2023-07-06T20:16:49.475 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.473+0000 7fcb1a08e700 1 -- 172.21.15.69:0/220220624 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcafc0a1150 msgr2=0x7fcafc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:49.475 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.473+0000 7fcb1a08e700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcafc0a1150 0x7fcafc0a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009990 tx=0x7fcb04005fb0 comp rx=0 tx=0).stop 2023-07-06T20:16:49.476 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.473+0000 7fcb1a08e700 1 -- 172.21.15.69:0/220220624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb14104fd0 msgr2=0x7fcb1419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:49.476 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.473+0000 7fcb1a08e700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb14104fd0 0x7fcb1419a7b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00ef30 tx=0x7fcb0c00c3a0 comp rx=0 tx=0).stop 2023-07-06T20:16:49.476 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.473+0000 7fcb1a08e700 1 -- 172.21.15.69:0/220220624 shutdown_connections 2023-07-06T20:16:49.476 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.473+0000 7fcb1a08e700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcafc0a1150 0x7fcafc0a3600 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:49.476 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.473+0000 7fcb1a08e700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb14104fd0 0x7fcb1419a7b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:49.476 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.473+0000 7fcb1a08e700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb140fff30 0x7fcb14195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:49.476 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.473+0000 7fcb1a08e700 1 --2- 172.21.15.69:0/220220624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb141008d0 0x7fcb14196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:49.477 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.473+0000 7fcb1a08e700 1 -- 172.21.15.69:0/220220624 >> 172.21.15.69:0/220220624 conn(0x7fcb140fb790 msgr2=0x7fcb140fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:49.477 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.473+0000 7fcb1a08e700 1 -- 172.21.15.69:0/220220624 shutdown_connections 2023-07-06T20:16:49.477 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:49.473+0000 7fcb1a08e700 1 -- 172.21.15.69:0/220220624 wait complete. 2023-07-06T20:16:49.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:49 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/220220624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:16:49.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:49 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/220220624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:16:49.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:49 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/220220624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:16:49.934 INFO:tasks.cephadm:Adding mds.a on smithi069 2023-07-06T20:16:49.934 INFO:tasks.cephadm:Adding mds.d on smithi069 2023-07-06T20:16:49.934 INFO:tasks.cephadm:Adding mds.g on smithi069 2023-07-06T20:16:49.935 INFO:tasks.cephadm:Adding mds.j on smithi069 2023-07-06T20:16:49.935 INFO:tasks.cephadm:Adding mds.b on smithi098 2023-07-06T20:16:49.935 INFO:tasks.cephadm:Adding mds.e on smithi098 2023-07-06T20:16:49.935 INFO:tasks.cephadm:Adding mds.h on smithi098 2023-07-06T20:16:49.935 INFO:tasks.cephadm:Adding mds.k on smithi098 2023-07-06T20:16:49.936 INFO:tasks.cephadm:Adding mds.c on smithi156 2023-07-06T20:16:49.936 INFO:tasks.cephadm:Adding mds.f on smithi156 2023-07-06T20:16:49.936 INFO:tasks.cephadm:Adding mds.i on smithi156 2023-07-06T20:16:49.936 INFO:tasks.cephadm:Adding mds.l on smithi156 2023-07-06T20:16:49.937 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph orch apply mds all '12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l' 2023-07-06T20:16:50.502 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:16:50.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:50 smithi156 ceph-mon[129280]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:50.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:50 smithi098 ceph-mon[128968]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:50 smithi069 ceph-mon[126399]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:51.691 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.688+0000 7fc5aef0e700 1 -- 172.21.15.156:0/1365987535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5a8101920 msgr2=0x7fc5a81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:51.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.688+0000 7fc5aef0e700 1 --2- 172.21.15.156:0/1365987535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5a8101920 0x7fc5a81081c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000b0a0 tx=0x7fc5a000b3b0 comp rx=0 tx=0).stop 2023-07-06T20:16:51.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.688+0000 7fc5aef0e700 1 -- 172.21.15.156:0/1365987535 shutdown_connections 2023-07-06T20:16:51.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.688+0000 7fc5aef0e700 1 --2- 172.21.15.156:0/1365987535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5a8101920 0x7fc5a81081c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:51.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.688+0000 7fc5aef0e700 1 --2- 172.21.15.156:0/1365987535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5a80ffcd0 0x7fc5a81000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:51.692 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.688+0000 7fc5aef0e700 1 --2- 172.21.15.156:0/1365987535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5a8100f70 0x7fc5a81013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:51.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.688+0000 7fc5aef0e700 1 -- 172.21.15.156:0/1365987535 >> 172.21.15.156:0/1365987535 conn(0x7fc5a80fb320 msgr2=0x7fc5a80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:51.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5aef0e700 1 -- 172.21.15.156:0/1365987535 shutdown_connections 2023-07-06T20:16:51.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5aef0e700 1 -- 172.21.15.156:0/1365987535 wait complete. 2023-07-06T20:16:51.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5aef0e700 1 Processor -- start 2023-07-06T20:16:51.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5aef0e700 1 -- start start 2023-07-06T20:16:51.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5aef0e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5a80ffcd0 0x7fc5a8195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:51.693 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5aef0e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5a8100f70 0x7fc5a8195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:51.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5aef0e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5a8101920 0x7fc5a819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:51.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5aef0e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a819d620 con 0x7fc5a8100f70 2023-07-06T20:16:51.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5aef0e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a819d760 con 0x7fc5a80ffcd0 2023-07-06T20:16:51.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5aef0e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a819d8f0 con 0x7fc5a8101920 2023-07-06T20:16:51.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5ad4ab700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5a8101920 0x7fc5a819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:51.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5ad4ab700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5a8101920 0x7fc5a819d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60184/0 (socket says 172.21.15.156:60184) 2023-07-06T20:16:51.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5ad4ab700 1 -- 172.21.15.156:0/2547808281 learned_addr learned my addr 172.21.15.156:0/2547808281 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-06T20:16:51.694 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.689+0000 7fc5accaa700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5a80ffcd0 0x7fc5a8195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:51.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.690+0000 7fc5a7fff700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5a8100f70 0x7fc5a8195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:51.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.690+0000 7fc5ad4ab700 1 -- 172.21.15.156:0/2547808281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5a80ffcd0 msgr2=0x7fc5a8195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:51.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.690+0000 7fc5ad4ab700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5a80ffcd0 0x7fc5a8195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:51.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.690+0000 7fc5ad4ab700 1 -- 172.21.15.156:0/2547808281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5a8100f70 msgr2=0x7fc5a8195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:51.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.690+0000 7fc5ad4ab700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5a8100f70 0x7fc5a8195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:51.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.690+0000 7fc5ad4ab700 1 -- 172.21.15.156:0/2547808281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a0009d00 con 0x7fc5a8101920 2023-07-06T20:16:51.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.690+0000 7fc5ad4ab700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5a8101920 0x7fc5a819d050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0007a60 tx=0x7fc5a000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:51.695 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.690+0000 7fc5a5ffb700 1 -- 172.21.15.156:0/2547808281 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000e040 con 0x7fc5a8101920 2023-07-06T20:16:51.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.690+0000 7fc5a5ffb700 1 -- 172.21.15.156:0/2547808281 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5a00094a0 con 0x7fc5a8101920 2023-07-06T20:16:51.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.690+0000 7fc5aef0e700 1 -- 172.21.15.156:0/2547808281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a819db70 con 0x7fc5a8101920 2023-07-06T20:16:51.696 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.690+0000 7fc5a5ffb700 1 -- 172.21.15.156:0/2547808281 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0005440 con 0x7fc5a8101920 2023-07-06T20:16:51.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.690+0000 7fc5aef0e700 1 -- 172.21.15.156:0/2547808281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a819df90 con 0x7fc5a8101920 2023-07-06T20:16:51.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.691+0000 7fc5aef0e700 1 -- 172.21.15.156:0/2547808281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a8106c10 con 0x7fc5a8101920 2023-07-06T20:16:51.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.692+0000 7fc5a5ffb700 1 -- 172.21.15.156:0/2547808281 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc5a001d470 con 0x7fc5a8101920 2023-07-06T20:16:51.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.693+0000 7fc5a5ffb700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5980a1150 0x7fc5980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:51.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.693+0000 7fc5a5ffb700 1 -- 172.21.15.156:0/2547808281 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc5a00cdca0 con 0x7fc5a8101920 2023-07-06T20:16:51.697 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.693+0000 7fc5accaa700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5980a1150 0x7fc5980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:51.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.694+0000 7fc5accaa700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5980a1150 0x7fc5980a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009c60 tx=0x7fc59c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:51.698 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.697+0000 7fc5a5ffb700 1 -- 172.21.15.156:0/2547808281 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc5a0097880 con 0x7fc5a8101920 2023-07-06T20:16:51.891 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.890+0000 7fc5aef0e700 1 -- 172.21.15.156:0/2547808281 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5a8060d40 con 0x7fc5980a1150 2023-07-06T20:16:51.901 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.900+0000 7fc5a5ffb700 1 -- 172.21.15.156:0/2547808281 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fc5a8060d40 con 0x7fc5980a1150 2023-07-06T20:16:51.902 INFO:teuthology.orchestra.run.smithi156.stdout:Scheduled mds.all update... 2023-07-06T20:16:51.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.902+0000 7fc5aef0e700 1 -- 172.21.15.156:0/2547808281 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5980a1150 msgr2=0x7fc5980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:51.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.902+0000 7fc5aef0e700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5980a1150 0x7fc5980a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009c60 tx=0x7fc59c009260 comp rx=0 tx=0).stop 2023-07-06T20:16:51.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.902+0000 7fc5aef0e700 1 -- 172.21.15.156:0/2547808281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5a8101920 msgr2=0x7fc5a819d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:51.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.902+0000 7fc5aef0e700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5a8101920 0x7fc5a819d050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0007a60 tx=0x7fc5a000bd60 comp rx=0 tx=0).stop 2023-07-06T20:16:51.904 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.902+0000 7fc5aef0e700 1 -- 172.21.15.156:0/2547808281 shutdown_connections 2023-07-06T20:16:51.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.902+0000 7fc5aef0e700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5980a1150 0x7fc5980a3600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:51.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.902+0000 7fc5aef0e700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5a8101920 0x7fc5a819d050 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:51.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.902+0000 7fc5aef0e700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5a80ffcd0 0x7fc5a8195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:51.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.902+0000 7fc5aef0e700 1 --2- 172.21.15.156:0/2547808281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5a8100f70 0x7fc5a8195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:51.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.902+0000 7fc5aef0e700 1 -- 172.21.15.156:0/2547808281 >> 172.21.15.156:0/2547808281 conn(0x7fc5a80fb320 msgr2=0x7fc5a80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:51.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.902+0000 7fc5aef0e700 1 -- 172.21.15.156:0/2547808281 shutdown_connections 2023-07-06T20:16:51.905 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-06T20:16:51.902+0000 7fc5aef0e700 1 -- 172.21.15.156:0/2547808281 wait complete. 2023-07-06T20:16:52.348 DEBUG:teuthology.orchestra.run.smithi069:mds.a> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.a.service 2023-07-06T20:16:52.353 DEBUG:teuthology.orchestra.run.smithi069:mds.d> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.d.service 2023-07-06T20:16:52.398 DEBUG:teuthology.orchestra.run.smithi069:mds.g> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.g.service 2023-07-06T20:16:52.438 INFO:journalctl@ceph.mds.a.smithi069.stdout:-- Logs begin at Thu 2023-07-06 19:46:03 UTC. -- 2023-07-06T20:16:52.441 DEBUG:teuthology.orchestra.run.smithi069:mds.j> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.j.service 2023-07-06T20:16:52.471 INFO:journalctl@ceph.mds.d.smithi069.stdout:-- Logs begin at Thu 2023-07-06 19:46:03 UTC. -- 2023-07-06T20:16:52.475 DEBUG:teuthology.orchestra.run.smithi098:mds.b> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.b.service 2023-07-06T20:16:52.477 DEBUG:teuthology.orchestra.run.smithi098:mds.e> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.e.service 2023-07-06T20:16:52.507 INFO:journalctl@ceph.mds.b.smithi098.stdout:-- Logs begin at Thu 2023-07-06 19:39:25 UTC. -- 2023-07-06T20:16:52.508 DEBUG:teuthology.orchestra.run.smithi098:mds.h> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.h.service 2023-07-06T20:16:52.514 INFO:journalctl@ceph.mds.g.smithi069.stdout:-- Logs begin at Thu 2023-07-06 19:46:03 UTC. -- 2023-07-06T20:16:52.533 INFO:journalctl@ceph.mds.e.smithi098.stdout:-- Logs begin at Thu 2023-07-06 19:39:25 UTC. -- 2023-07-06T20:16:52.533 DEBUG:teuthology.orchestra.run.smithi098:mds.k> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.k.service 2023-07-06T20:16:52.544 INFO:journalctl@ceph.mds.j.smithi069.stdout:-- Logs begin at Thu 2023-07-06 19:46:03 UTC. -- 2023-07-06T20:16:52.558 INFO:journalctl@ceph.mds.h.smithi098.stdout:-- Logs begin at Thu 2023-07-06 19:39:25 UTC. -- 2023-07-06T20:16:52.559 DEBUG:teuthology.orchestra.run.smithi156:mds.c> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.c.service 2023-07-06T20:16:52.561 DEBUG:teuthology.orchestra.run.smithi156:mds.f> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.f.service 2023-07-06T20:16:52.584 INFO:journalctl@ceph.mds.k.smithi098.stdout:-- Logs begin at Thu 2023-07-06 19:39:25 UTC. -- 2023-07-06T20:16:52.590 INFO:journalctl@ceph.mds.c.smithi156.stdout:-- Logs begin at Thu 2023-07-06 19:42:21 UTC. -- 2023-07-06T20:16:52.591 DEBUG:teuthology.orchestra.run.smithi156:mds.i> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.i.service 2023-07-06T20:16:52.617 INFO:journalctl@ceph.mds.f.smithi156.stdout:-- Logs begin at Thu 2023-07-06 19:42:21 UTC. -- 2023-07-06T20:16:52.617 DEBUG:teuthology.orchestra.run.smithi156:mds.l> sudo journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.l.service 2023-07-06T20:16:52.643 INFO:journalctl@ceph.mds.i.smithi156.stdout:-- Logs begin at Thu 2023-07-06 19:42:21 UTC. -- 2023-07-06T20:16:52.644 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-06T20:16:52.645 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:16:52.670 INFO:journalctl@ceph.mds.l.smithi156.stdout:-- Logs begin at Thu 2023-07-06 19:42:21 UTC. -- 2023-07-06T20:16:52.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.853+0000 7f7565dff700 1 -- 172.21.15.69:0/3962109465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75600ffc30 msgr2=0x7f7560100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:52.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.853+0000 7f7565dff700 1 --2- 172.21.15.69:0/3962109465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75600ffc30 0x7f7560100040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7550009990 tx=0x7f7550009ca0 comp rx=0 tx=0).stop 2023-07-06T20:16:52.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.853+0000 7f7565dff700 1 -- 172.21.15.69:0/3962109465 shutdown_connections 2023-07-06T20:16:52.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.853+0000 7f7565dff700 1 --2- 172.21.15.69:0/3962109465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75601017b0 0x7f7560108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:52.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.853+0000 7f7565dff700 1 --2- 172.21.15.69:0/3962109465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75600ffc30 0x7f7560100040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:52.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.853+0000 7f7565dff700 1 --2- 172.21.15.69:0/3962109465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7560100e20 0x7f7560101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:52.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.853+0000 7f7565dff700 1 -- 172.21.15.69:0/3962109465 >> 172.21.15.69:0/3962109465 conn(0x7f75600fb210 msgr2=0x7f75600fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:52.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.853+0000 7f7565dff700 1 -- 172.21.15.69:0/3962109465 shutdown_connections 2023-07-06T20:16:52.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.853+0000 7f7565dff700 1 -- 172.21.15.69:0/3962109465 wait complete. 2023-07-06T20:16:52.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.855+0000 7f7565dff700 1 Processor -- start 2023-07-06T20:16:52.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.855+0000 7f7565dff700 1 -- start start 2023-07-06T20:16:52.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f7565dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75600ffc30 0x7f7560195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:52.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f7565dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7560100e20 0x7f7560195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:52.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f7565dff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75601017b0 0x7f756019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:52.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f7565dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756019d480 con 0x7f75600ffc30 2023-07-06T20:16:52.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f7565dff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756019d5c0 con 0x7f75601017b0 2023-07-06T20:16:52.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f7565dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756019d750 con 0x7f7560100e20 2023-07-06T20:16:52.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f755f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75600ffc30 0x7f7560195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:52.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f755effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7560100e20 0x7f7560195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:52.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f755f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75600ffc30 0x7f7560195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59976/0 (socket says 172.21.15.69:59976) 2023-07-06T20:16:52.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f755effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7560100e20 0x7f7560195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:60916/0 (socket says 172.21.15.69:60916) 2023-07-06T20:16:52.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f755f7fe700 1 -- 172.21.15.69:0/1779132547 learned_addr learned my addr 172.21.15.69:0/1779132547 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:16:52.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f755ffff700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75601017b0 0x7f756019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:52.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f755effd700 1 -- 172.21.15.69:0/1779132547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75601017b0 msgr2=0x7f756019ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:52.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f755effd700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75601017b0 0x7f756019ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:52.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f755effd700 1 -- 172.21.15.69:0/1779132547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75600ffc30 msgr2=0x7f7560195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:52.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f755effd700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75600ffc30 0x7f7560195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:52.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f755effd700 1 -- 172.21.15.69:0/1779132547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75500095f0 con 0x7f7560100e20 2023-07-06T20:16:52.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.856+0000 7f755f7fe700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75600ffc30 0x7f7560195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:16:52.861 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.857+0000 7f755effd700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7560100e20 0x7f7560195e40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f755400e9f0 tx=0x7f755400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:52.861 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.857+0000 7f755cff9700 1 -- 172.21.15.69:0/1779132547 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755400c9c0 con 0x7f7560100e20 2023-07-06T20:16:52.862 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.857+0000 7f755cff9700 1 -- 172.21.15.69:0/1779132547 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7554010430 con 0x7f7560100e20 2023-07-06T20:16:52.862 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.857+0000 7f7565dff700 1 -- 172.21.15.69:0/1779132547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756019da30 con 0x7f7560100e20 2023-07-06T20:16:52.862 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.857+0000 7f755cff9700 1 -- 172.21.15.69:0/1779132547 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554018680 con 0x7f7560100e20 2023-07-06T20:16:52.862 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.857+0000 7f7565dff700 1 -- 172.21.15.69:0/1779132547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756019df40 con 0x7f7560100e20 2023-07-06T20:16:52.862 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.858+0000 7f7565dff700 1 -- 172.21.15.69:0/1779132547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75601071a0 con 0x7f7560100e20 2023-07-06T20:16:52.863 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.860+0000 7f755cff9700 1 -- 172.21.15.69:0/1779132547 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7554010860 con 0x7f7560100e20 2023-07-06T20:16:52.863 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.861+0000 7f755cff9700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f75480a11a0 0x7f75480a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:52.863 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.862+0000 7f755cff9700 1 -- 172.21.15.69:0/1779132547 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7554014070 con 0x7f7560100e20 2023-07-06T20:16:52.864 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.862+0000 7f755f7fe700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f75480a11a0 0x7f75480a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:52.864 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.862+0000 7f755f7fe700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f75480a11a0 0x7f75480a3650 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f755000b2e0 tx=0x7f7550005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:52.868 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:52.866+0000 7f755cff9700 1 -- 172.21.15.69:0/1779132547 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7554051770 con 0x7f7560100e20 2023-07-06T20:16:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:52 smithi069 ceph-mon[126399]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:52 smithi069 ceph-mon[126399]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:16:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:52 smithi069 ceph-mon[126399]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-07-06T20:16:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:53.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:53.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:53.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:53.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:53.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:16:53.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:16:53.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.069+0000 7f7565dff700 1 -- 172.21.15.69:0/1779132547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f756002c8e0 con 0x7f7560100e20 2023-07-06T20:16:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.071+0000 7f755cff9700 1 -- 172.21.15.69:0/1779132547 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18368 (secure 0 0 0) 0x7f7554096180 con 0x7f7560100e20 2023-07-06T20:16:53.074 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:16:53.074 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":79,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:16:42.349843+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:16:53.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.074+0000 7f7565dff700 1 -- 172.21.15.69:0/1779132547 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f75480a11a0 msgr2=0x7f75480a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:53.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.074+0000 7f7565dff700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f75480a11a0 0x7f75480a3650 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f755000b2e0 tx=0x7f7550005fb0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.074+0000 7f7565dff700 1 -- 172.21.15.69:0/1779132547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7560100e20 msgr2=0x7f7560195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:53.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.074+0000 7f7565dff700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7560100e20 0x7f7560195e40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f755400e9f0 tx=0x7f755400ed00 comp rx=0 tx=0).stop 2023-07-06T20:16:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.074+0000 7f7565dff700 1 -- 172.21.15.69:0/1779132547 shutdown_connections 2023-07-06T20:16:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.074+0000 7f7565dff700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f75480a11a0 0x7f75480a3650 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.074+0000 7f7565dff700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75601017b0 0x7f756019ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.074+0000 7f7565dff700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75600ffc30 0x7f7560195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.074+0000 7f7565dff700 1 --2- 172.21.15.69:0/1779132547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7560100e20 0x7f7560195e40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.074+0000 7f7565dff700 1 -- 172.21.15.69:0/1779132547 >> 172.21.15.69:0/1779132547 conn(0x7f75600fb210 msgr2=0x7f75601042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.074+0000 7f7565dff700 1 -- 172.21.15.69:0/1779132547 shutdown_connections 2023-07-06T20:16:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.074+0000 7f7565dff700 1 -- 172.21.15.69:0/1779132547 wait complete. 2023-07-06T20:16:53.096 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:14.022799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:16:53.097 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:16:53.152 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:52 smithi156 ceph-mon[129280]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:53.152 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:52 smithi156 ceph-mon[129280]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:16:53.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:52 smithi156 ceph-mon[129280]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-07-06T20:16:53.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:53.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:53.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:53.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:53.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:53.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:16:53.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:16:53.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:53.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.294+0000 7f992a5ff700 1 -- 172.21.15.69:0/473287862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99240ffc10 msgr2=0x7f9924100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:53.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.294+0000 7f992a5ff700 1 --2- 172.21.15.69:0/473287862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99240ffc10 0x7f9924100020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9914009990 tx=0x7f9914009ca0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.294+0000 7f992a5ff700 1 -- 172.21.15.69:0/473287862 shutdown_connections 2023-07-06T20:16:53.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.294+0000 7f992a5ff700 1 --2- 172.21.15.69:0/473287862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9924101790 0x7f9924108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.294+0000 7f992a5ff700 1 --2- 172.21.15.69:0/473287862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99240ffc10 0x7f9924100020 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.294+0000 7f992a5ff700 1 --2- 172.21.15.69:0/473287862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924100e00 0x7f9924101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.294+0000 7f992a5ff700 1 -- 172.21.15.69:0/473287862 >> 172.21.15.69:0/473287862 conn(0x7f99240fb210 msgr2=0x7f99240fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:53.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.294+0000 7f992a5ff700 1 -- 172.21.15.69:0/473287862 shutdown_connections 2023-07-06T20:16:53.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.294+0000 7f992a5ff700 1 -- 172.21.15.69:0/473287862 wait complete. 2023-07-06T20:16:53.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.295+0000 7f992a5ff700 1 Processor -- start 2023-07-06T20:16:53.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.295+0000 7f992a5ff700 1 -- start start 2023-07-06T20:16:53.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f992a5ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99240ffc10 0x7f99241959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:53.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f992a5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9924100e00 0x7f9924195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:53.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f992a5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101790 0x7f992419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:53.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f992a5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992419d540 con 0x7f9924100e00 2023-07-06T20:16:53.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f992a5ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992419d680 con 0x7f99240ffc10 2023-07-06T20:16:53.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f992a5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992419d810 con 0x7f9924101790 2023-07-06T20:16:53.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f99237fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9924100e00 0x7f9924195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:53.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f99237fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9924100e00 0x7f9924195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59982/0 (socket says 172.21.15.69:59982) 2023-07-06T20:16:53.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f99237fe700 1 -- 172.21.15.69:0/441704634 learned_addr learned my addr 172.21.15.69:0/441704634 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:16:53.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f9928b9c700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101790 0x7f992419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:53.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f9923fff700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99240ffc10 0x7f99241959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:53.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f99237fe700 1 -- 172.21.15.69:0/441704634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101790 msgr2=0x7f992419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:53.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f99237fe700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101790 0x7f992419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f99237fe700 1 -- 172.21.15.69:0/441704634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99240ffc10 msgr2=0x7f99241959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:53.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f99237fe700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99240ffc10 0x7f99241959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f99237fe700 1 -- 172.21.15.69:0/441704634 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99140095f0 con 0x7f9924100e00 2023-07-06T20:16:53.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.296+0000 7f9928b9c700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101790 0x7f992419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:16:53.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.297+0000 7f99237fe700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9924100e00 0x7f9924195f00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f991800e9a0 tx=0x7f991800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:53.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.297+0000 7f99217fa700 1 -- 172.21.15.69:0/441704634 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918009790 con 0x7f9924100e00 2023-07-06T20:16:53.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.297+0000 7f992a5ff700 1 -- 172.21.15.69:0/441704634 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992419daf0 con 0x7f9924100e00 2023-07-06T20:16:53.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.297+0000 7f99217fa700 1 -- 172.21.15.69:0/441704634 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9918010430 con 0x7f9924100e00 2023-07-06T20:16:53.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.297+0000 7f99217fa700 1 -- 172.21.15.69:0/441704634 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918018680 con 0x7f9924100e00 2023-07-06T20:16:53.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.297+0000 7f992a5ff700 1 -- 172.21.15.69:0/441704634 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992419e050 con 0x7f9924100e00 2023-07-06T20:16:53.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.298+0000 7f992a5ff700 1 -- 172.21.15.69:0/441704634 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992410b430 con 0x7f9924100e00 2023-07-06T20:16:53.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.300+0000 7f99217fa700 1 -- 172.21.15.69:0/441704634 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f99180105a0 con 0x7f9924100e00 2023-07-06T20:16:53.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.301+0000 7f99217fa700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f990c0a1150 0x7f990c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:53.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.302+0000 7f99217fa700 1 -- 172.21.15.69:0/441704634 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9918014070 con 0x7f9924100e00 2023-07-06T20:16:53.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.302+0000 7f9923fff700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f990c0a1150 0x7f990c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:53.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.302+0000 7f9923fff700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f990c0a1150 0x7f990c0a3600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f991400b2e0 tx=0x7f9914005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:53.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.303+0000 7f99217fa700 1 -- 172.21.15.69:0/441704634 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9918096400 con 0x7f9924100e00 2023-07-06T20:16:53.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:52 smithi098 ceph-mon[128968]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:53.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:52 smithi098 ceph-mon[128968]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:16:53.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:52 smithi098 ceph-mon[128968]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-07-06T20:16:53.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:52 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:53.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:52 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:16:53.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:52 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:53.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:52 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:16:53.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:52 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:53.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:52 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:16:53.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:52 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:16:53.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:52 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:53.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.499+0000 7f992a5ff700 1 -- 172.21.15.69:0/441704634 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9924106b60 con 0x7f9924100e00 2023-07-06T20:16:53.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.500+0000 7f99217fa700 1 -- 172.21.15.69:0/441704634 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9918099a20 con 0x7f9924100e00 2023-07-06T20:16:53.502 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2023-07-06T20:16:53.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.502+0000 7f992a5ff700 1 -- 172.21.15.69:0/441704634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f990c0a1150 msgr2=0x7f990c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:53.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.502+0000 7f992a5ff700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f990c0a1150 0x7f990c0a3600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f991400b2e0 tx=0x7f9914005fb0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.502+0000 7f992a5ff700 1 -- 172.21.15.69:0/441704634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9924100e00 msgr2=0x7f9924195f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:53.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.502+0000 7f992a5ff700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9924100e00 0x7f9924195f00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f991800e9a0 tx=0x7f991800ecb0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.503+0000 7f992a5ff700 1 -- 172.21.15.69:0/441704634 shutdown_connections 2023-07-06T20:16:53.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.503+0000 7f992a5ff700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f990c0a1150 0x7f990c0a3600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.503+0000 7f992a5ff700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101790 0x7f992419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.503+0000 7f992a5ff700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99240ffc10 0x7f99241959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.503+0000 7f992a5ff700 1 --2- 172.21.15.69:0/441704634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9924100e00 0x7f9924195f00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.503+0000 7f992a5ff700 1 -- 172.21.15.69:0/441704634 >> 172.21.15.69:0/441704634 conn(0x7f99240fb210 msgr2=0x7f99240fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:53.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.503+0000 7f992a5ff700 1 -- 172.21.15.69:0/441704634 shutdown_connections 2023-07-06T20:16:53.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.503+0000 7f992a5ff700 1 -- 172.21.15.69:0/441704634 wait complete. 2023-07-06T20:16:53.523 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:16:53.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.717+0000 7ffa09bff700 1 -- 172.21.15.69:0/1309119411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa040ffc30 msgr2=0x7ffa04100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:53.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.717+0000 7ffa09bff700 1 --2- 172.21.15.69:0/1309119411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa040ffc30 0x7ffa04100040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009990 tx=0x7ff9f4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.717+0000 7ffa09bff700 1 -- 172.21.15.69:0/1309119411 shutdown_connections 2023-07-06T20:16:53.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.717+0000 7ffa09bff700 1 --2- 172.21.15.69:0/1309119411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa041017b0 0x7ffa04108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.717+0000 7ffa09bff700 1 --2- 172.21.15.69:0/1309119411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa040ffc30 0x7ffa04100040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.717+0000 7ffa09bff700 1 --2- 172.21.15.69:0/1309119411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa04100e20 0x7ffa04101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.717+0000 7ffa09bff700 1 -- 172.21.15.69:0/1309119411 >> 172.21.15.69:0/1309119411 conn(0x7ffa040fb210 msgr2=0x7ffa040fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:53.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.717+0000 7ffa09bff700 1 -- 172.21.15.69:0/1309119411 shutdown_connections 2023-07-06T20:16:53.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.717+0000 7ffa09bff700 1 -- 172.21.15.69:0/1309119411 wait complete. 2023-07-06T20:16:53.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.718+0000 7ffa09bff700 1 Processor -- start 2023-07-06T20:16:53.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.718+0000 7ffa09bff700 1 -- start start 2023-07-06T20:16:53.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.718+0000 7ffa09bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa040ffc30 0x7ffa04197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:53.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.718+0000 7ffa09bff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa04100e20 0x7ffa04198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:53.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.718+0000 7ffa09bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa041017b0 0x7ffa0419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:53.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.718+0000 7ffa09bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0419f6d0 con 0x7ffa040ffc30 2023-07-06T20:16:53.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.718+0000 7ffa09bff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0419f810 con 0x7ffa04100e20 2023-07-06T20:16:53.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.718+0000 7ffa09bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0419f9a0 con 0x7ffa041017b0 2023-07-06T20:16:53.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.718+0000 7ffa037fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa040ffc30 0x7ffa04197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:53.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa037fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa040ffc30 0x7ffa04197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60002/0 (socket says 172.21.15.69:60002) 2023-07-06T20:16:53.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa037fe700 1 -- 172.21.15.69:0/1255895269 learned_addr learned my addr 172.21.15.69:0/1255895269 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:16:53.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa03fff700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa041017b0 0x7ffa0419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:53.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa02ffd700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa04100e20 0x7ffa04198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:53.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa037fe700 1 -- 172.21.15.69:0/1255895269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa041017b0 msgr2=0x7ffa0419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:53.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa037fe700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa041017b0 0x7ffa0419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa037fe700 1 -- 172.21.15.69:0/1255895269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa04100e20 msgr2=0x7ffa04198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:53.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa037fe700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa04100e20 0x7ffa04198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa037fe700 1 -- 172.21.15.69:0/1255895269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f40095f0 con 0x7ffa040ffc30 2023-07-06T20:16:53.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa037fe700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa040ffc30 0x7ffa04197b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400b620 tx=0x7ff9f4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:53.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa00ff9700 1 -- 172.21.15.69:0/1255895269 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f401d070 con 0x7ffa040ffc30 2023-07-06T20:16:53.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa00ff9700 1 -- 172.21.15.69:0/1255895269 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9f4021410 con 0x7ffa040ffc30 2023-07-06T20:16:53.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa00ff9700 1 -- 172.21.15.69:0/1255895269 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4017650 con 0x7ffa040ffc30 2023-07-06T20:16:53.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.719+0000 7ffa09bff700 1 -- 172.21.15.69:0/1255895269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0419fc20 con 0x7ffa040ffc30 2023-07-06T20:16:53.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.720+0000 7ffa09bff700 1 -- 172.21.15.69:0/1255895269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa041a0040 con 0x7ffa040ffc30 2023-07-06T20:16:53.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.721+0000 7ffa09bff700 1 -- 172.21.15.69:0/1255895269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa04192310 con 0x7ffa040ffc30 2023-07-06T20:16:53.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.722+0000 7ffa00ff9700 1 -- 172.21.15.69:0/1255895269 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff9f4021580 con 0x7ffa040ffc30 2023-07-06T20:16:53.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.724+0000 7ffa00ff9700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff9f00a1220 0x7ff9f00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:53.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.724+0000 7ffa00ff9700 1 -- 172.21.15.69:0/1255895269 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff9f40cd9b0 con 0x7ffa040ffc30 2023-07-06T20:16:53.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.724+0000 7ffa02ffd700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff9f00a1220 0x7ff9f00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:53.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.724+0000 7ffa02ffd700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff9f00a1220 0x7ff9f00a36d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009bf0 tx=0x7ff9ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.727+0000 7ffa00ff9700 1 -- 172.21.15.69:0/1255895269 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff9f4097590 con 0x7ffa040ffc30 2023-07-06T20:16:53.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.972+0000 7ffa09bff700 1 -- 172.21.15.69:0/1255895269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa0402c8e0 con 0x7ffa040ffc30 2023-07-06T20:16:53.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.972+0000 7ffa00ff9700 1 -- 172.21.15.69:0/1255895269 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7ff9f4095e60 con 0x7ffa040ffc30 2023-07-06T20:16:53.975 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:16:53.975 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-07-06T20:16:53.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.975+0000 7ffa09bff700 1 -- 172.21.15.69:0/1255895269 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff9f00a1220 msgr2=0x7ff9f00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:53.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.975+0000 7ffa09bff700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff9f00a1220 0x7ff9f00a36d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009bf0 tx=0x7ff9ec009260 comp rx=0 tx=0).stop 2023-07-06T20:16:53.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.975+0000 7ffa09bff700 1 -- 172.21.15.69:0/1255895269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa040ffc30 msgr2=0x7ffa04197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:53.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.975+0000 7ffa09bff700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa040ffc30 0x7ffa04197b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400b620 tx=0x7ff9f4005740 comp rx=0 tx=0).stop 2023-07-06T20:16:53.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.975+0000 7ffa09bff700 1 -- 172.21.15.69:0/1255895269 shutdown_connections 2023-07-06T20:16:53.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.975+0000 7ffa09bff700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff9f00a1220 0x7ff9f00a36d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.975+0000 7ffa09bff700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa041017b0 0x7ffa0419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.975+0000 7ffa09bff700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa040ffc30 0x7ffa04197b50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.975+0000 7ffa09bff700 1 --2- 172.21.15.69:0/1255895269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa04100e20 0x7ffa04198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:53.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.975+0000 7ffa09bff700 1 -- 172.21.15.69:0/1255895269 >> 172.21.15.69:0/1255895269 conn(0x7ffa040fb210 msgr2=0x7ffa041066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:53.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.975+0000 7ffa09bff700 1 -- 172.21.15.69:0/1255895269 shutdown_connections 2023-07-06T20:16:53.979 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:53.975+0000 7ffa09bff700 1 -- 172.21.15.69:0/1255895269 wait complete. 2023-07-06T20:16:53.979 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 1 2023-07-06T20:16:53.982 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:53 smithi156 ceph-mon[129280]: Deploying daemon mds.l on smithi156 2023-07-06T20:16:53.982 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:53 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1779132547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:16:53.982 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:53 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/441704634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:16:53.982 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 06 20:16:53 smithi156 systemd[1]: Starting Ceph mds.l for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:16:54.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:53 smithi069 ceph-mon[126399]: Deploying daemon mds.l on smithi156 2023-07-06T20:16:54.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:53 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1779132547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:16:54.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:53 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/441704634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:16:54.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:53 smithi098 ceph-mon[128968]: Deploying daemon mds.l on smithi156 2023-07-06T20:16:54.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:53 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1779132547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:16:54.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:53 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/441704634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:16:54.529 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 06 20:16:54 smithi156 podman[154309]: 2023-07-06T20:16:54.529 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-mds[154349]: -- 172.21.15.156:0/2555237777 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558c79be11e0 con 0x558c79bb6c00 2023-07-06T20:16:54.529 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-l[154325]: starting mds.l at 2023-07-06T20:16:54.856 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 06 20:16:54 smithi156 bash[154309]: 9e1555364e4c309a5def1c4a96b7077a5059c6d708e6b0d522414807ac0bcd82 2023-07-06T20:16:54.857 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 06 20:16:54 smithi156 systemd[1]: Started Ceph mds.l for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:16:55.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:54 smithi069 ceph-mon[126399]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:55.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:54 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1255895269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:16:55.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:54 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.156:6832/1091328636,v1:172.21.15.156:6833/1091328636] up:boot 2023-07-06T20:16:55.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:54 smithi069 ceph-mon[126399]: fsmap 1 up:standby 2023-07-06T20:16:55.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-06T20:16:55.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:55.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:55.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:55.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:16:55.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:16:55.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:55.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:54 smithi098 ceph-mon[128968]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:55.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:54 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1255895269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:16:55.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:54 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.156:6832/1091328636,v1:172.21.15.156:6833/1091328636] up:boot 2023-07-06T20:16:55.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:54 smithi098 ceph-mon[128968]: fsmap 1 up:standby 2023-07-06T20:16:55.296 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-06T20:16:55.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:55.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:55.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:55.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:16:55.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:16:55.297 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:55.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-mon[129280]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1255895269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:16:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.156:6832/1091328636,v1:172.21.15.156:6833/1091328636] up:boot 2023-07-06T20:16:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-mon[129280]: fsmap 1 up:standby 2023-07-06T20:16:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-06T20:16:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:55.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:55.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:55.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:16:55.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:16:55.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:55.996 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:16:56.163 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:55 smithi098 ceph-mon[128968]: Deploying daemon mds.k on smithi098 2023-07-06T20:16:56.163 INFO:journalctl@ceph.mds.k.smithi098.stdout:Jul 06 20:16:56 smithi098 systemd[1]: Starting Ceph mds.k for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:16:56.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.200+0000 7ff214dff700 1 -- 172.21.15.69:0/3856770466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff210100e20 msgr2=0x7ff210101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:56.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.200+0000 7ff214dff700 1 --2- 172.21.15.69:0/3856770466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff210100e20 0x7ff210101270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff204009990 tx=0x7ff204009ca0 comp rx=0 tx=0).stop 2023-07-06T20:16:56.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.200+0000 7ff214dff700 1 -- 172.21.15.69:0/3856770466 shutdown_connections 2023-07-06T20:16:56.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.200+0000 7ff214dff700 1 --2- 172.21.15.69:0/3856770466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2101017b0 0x7ff210108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:56.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.200+0000 7ff214dff700 1 --2- 172.21.15.69:0/3856770466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2100ffc30 0x7ff210100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:56.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.200+0000 7ff214dff700 1 --2- 172.21.15.69:0/3856770466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff210100e20 0x7ff210101270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:56.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.200+0000 7ff214dff700 1 -- 172.21.15.69:0/3856770466 >> 172.21.15.69:0/3856770466 conn(0x7ff2100fb210 msgr2=0x7ff2100fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:56.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.200+0000 7ff214dff700 1 -- 172.21.15.69:0/3856770466 shutdown_connections 2023-07-06T20:16:56.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.200+0000 7ff214dff700 1 -- 172.21.15.69:0/3856770466 wait complete. 2023-07-06T20:16:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.201+0000 7ff214dff700 1 Processor -- start 2023-07-06T20:16:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.201+0000 7ff214dff700 1 -- start start 2023-07-06T20:16:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.201+0000 7ff214dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2100ffc30 0x7ff210197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.201+0000 7ff214dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff210100e20 0x7ff210198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.201+0000 7ff214dff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2101017b0 0x7ff21019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.201+0000 7ff214dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21019f6b0 con 0x7ff2100ffc30 2023-07-06T20:16:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.201+0000 7ff214dff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21019f7f0 con 0x7ff2101017b0 2023-07-06T20:16:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.201+0000 7ff214dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21019f980 con 0x7ff210100e20 2023-07-06T20:16:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.201+0000 7ff20e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2100ffc30 0x7ff210197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.201+0000 7ff20e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2100ffc30 0x7ff210197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60024/0 (socket says 172.21.15.69:60024) 2023-07-06T20:16:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.201+0000 7ff20e59c700 1 -- 172.21.15.69:0/3375408696 learned_addr learned my addr 172.21.15.69:0/3375408696 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:16:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.201+0000 7ff20ed9d700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2101017b0 0x7ff21019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.202+0000 7ff20dd9b700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff210100e20 0x7ff210198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.202+0000 7ff20e59c700 1 -- 172.21.15.69:0/3375408696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff210100e20 msgr2=0x7ff210198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.202+0000 7ff20e59c700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff210100e20 0x7ff210198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.202+0000 7ff20e59c700 1 -- 172.21.15.69:0/3375408696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2101017b0 msgr2=0x7ff21019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.202+0000 7ff20e59c700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2101017b0 0x7ff21019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.202+0000 7ff20e59c700 1 -- 172.21.15.69:0/3375408696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2040095f0 con 0x7ff2100ffc30 2023-07-06T20:16:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.202+0000 7ff20e59c700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2100ffc30 0x7ff210197b30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff20000e970 tx=0x7ff20000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.202+0000 7ff1ff7fe700 1 -- 172.21.15.69:0/3375408696 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff200009790 con 0x7ff2100ffc30 2023-07-06T20:16:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.202+0000 7ff214dff700 1 -- 172.21.15.69:0/3375408696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21019fc60 con 0x7ff2100ffc30 2023-07-06T20:16:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.202+0000 7ff1ff7fe700 1 -- 172.21.15.69:0/3375408696 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff20000c7e0 con 0x7ff2100ffc30 2023-07-06T20:16:56.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.202+0000 7ff1ff7fe700 1 -- 172.21.15.69:0/3375408696 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff200003c70 con 0x7ff2100ffc30 2023-07-06T20:16:56.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.202+0000 7ff214dff700 1 -- 172.21.15.69:0/3375408696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2101a01c0 con 0x7ff2100ffc30 2023-07-06T20:16:56.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.203+0000 7ff214dff700 1 -- 172.21.15.69:0/3375408696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21010b450 con 0x7ff2100ffc30 2023-07-06T20:16:56.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.205+0000 7ff1ff7fe700 1 -- 172.21.15.69:0/3375408696 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff20000cc50 con 0x7ff2100ffc30 2023-07-06T20:16:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.207+0000 7ff1ff7fe700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff1f80a11a0 0x7ff1f80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:56.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.208+0000 7ff1ff7fe700 1 -- 172.21.15.69:0/3375408696 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff200014070 con 0x7ff2100ffc30 2023-07-06T20:16:56.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.208+0000 7ff20dd9b700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff1f80a11a0 0x7ff1f80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:56.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.208+0000 7ff20dd9b700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff1f80a11a0 0x7ff1f80a3650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff20400b2e0 tx=0x7ff204005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:56.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.212+0000 7ff1ff7fe700 1 -- 172.21.15.69:0/3375408696 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff2000961f0 con 0x7ff2100ffc30 2023-07-06T20:16:56.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:55 smithi069 ceph-mon[126399]: Deploying daemon mds.k on smithi098 2023-07-06T20:16:56.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:55 smithi156 ceph-mon[129280]: Deploying daemon mds.k on smithi098 2023-07-06T20:16:56.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.450+0000 7ff214dff700 1 -- 172.21.15.69:0/3375408696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff21010b610 con 0x7ff2100ffc30 2023-07-06T20:16:56.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.450+0000 7ff1ff7fe700 1 -- 172.21.15.69:0/3375408696 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7ff200003dd0 con 0x7ff2100ffc30 2023-07-06T20:16:56.452 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:16:56.452 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-06T20:16:56.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.453+0000 7ff214dff700 1 -- 172.21.15.69:0/3375408696 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff1f80a11a0 msgr2=0x7ff1f80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:56.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.453+0000 7ff214dff700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff1f80a11a0 0x7ff1f80a3650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff20400b2e0 tx=0x7ff204005fb0 comp rx=0 tx=0).stop 2023-07-06T20:16:56.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.453+0000 7ff214dff700 1 -- 172.21.15.69:0/3375408696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2100ffc30 msgr2=0x7ff210197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:56.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.453+0000 7ff214dff700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2100ffc30 0x7ff210197b30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff20000e970 tx=0x7ff20000ec80 comp rx=0 tx=0).stop 2023-07-06T20:16:56.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.453+0000 7ff214dff700 1 -- 172.21.15.69:0/3375408696 shutdown_connections 2023-07-06T20:16:56.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.453+0000 7ff214dff700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff1f80a11a0 0x7ff1f80a3650 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:56.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.453+0000 7ff214dff700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2101017b0 0x7ff21019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:56.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.453+0000 7ff214dff700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2100ffc30 0x7ff210197b30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:56.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.453+0000 7ff214dff700 1 --2- 172.21.15.69:0/3375408696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff210100e20 0x7ff210198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:56.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.453+0000 7ff214dff700 1 -- 172.21.15.69:0/3375408696 >> 172.21.15.69:0/3375408696 conn(0x7ff2100fb210 msgr2=0x7ff2101066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:56.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.453+0000 7ff214dff700 1 -- 172.21.15.69:0/3375408696 shutdown_connections 2023-07-06T20:16:56.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:56.453+0000 7ff214dff700 1 -- 172.21.15.69:0/3375408696 wait complete. 2023-07-06T20:16:56.455 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 2 2023-07-06T20:16:56.747 INFO:journalctl@ceph.mds.k.smithi098.stdout:Jul 06 20:16:56 smithi098 podman[153360]: 2023-07-06T20:16:57.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:56 smithi098 ceph-mon[128968]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:57.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:56 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3375408696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:16:57.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:57.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:57.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:57.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:16:57.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:16:57.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:57.126 INFO:journalctl@ceph.mds.k.smithi098.stdout:Jul 06 20:16:56 smithi098 ceph-mds[153400]: -- 172.21.15.98:0/1404431807 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c9980971e0 con 0x55c99806c800 2023-07-06T20:16:57.127 INFO:journalctl@ceph.mds.k.smithi098.stdout:Jul 06 20:16:56 smithi098 bash[153360]: 9964ef061164a38c6ab4824af0d154365b5854f52142592282ba79a030b18745 2023-07-06T20:16:57.127 INFO:journalctl@ceph.mds.k.smithi098.stdout:Jul 06 20:16:56 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-k[153376]: starting mds.k at 2023-07-06T20:16:57.127 INFO:journalctl@ceph.mds.k.smithi098.stdout:Jul 06 20:16:56 smithi098 systemd[1]: Started Ceph mds.k for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:16:57.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:56 smithi069 ceph-mon[126399]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:57.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:56 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3375408696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:16:57.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:57.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:57.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:57.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:16:57.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:16:57.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:57.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:56 smithi156 ceph-mon[129280]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:57.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:56 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3375408696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:16:57.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:57.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:57.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:16:57.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:16:57.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:16:57.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:16:58.024 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:57 smithi069 ceph-mon[126399]: Deploying daemon mds.j on smithi069 2023-07-06T20:16:58.025 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:57 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.98:6832/3115382908,v1:172.21.15.98:6833/3115382908] up:boot 2023-07-06T20:16:58.025 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:57 smithi069 ceph-mon[126399]: fsmap 2 up:standby 2023-07-06T20:16:58.025 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-06T20:16:58.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:57 smithi156 ceph-mon[129280]: Deploying daemon mds.j on smithi069 2023-07-06T20:16:58.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:57 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.98:6832/3115382908,v1:172.21.15.98:6833/3115382908] up:boot 2023-07-06T20:16:58.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:57 smithi156 ceph-mon[129280]: fsmap 2 up:standby 2023-07-06T20:16:58.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-06T20:16:58.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:57 smithi098 ceph-mon[128968]: Deploying daemon mds.j on smithi069 2023-07-06T20:16:58.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:57 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.98:6832/3115382908,v1:172.21.15.98:6833/3115382908] up:boot 2023-07-06T20:16:58.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:57 smithi098 ceph-mon[128968]: fsmap 2 up:standby 2023-07-06T20:16:58.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-06T20:16:58.472 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:16:58.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.778+0000 7f8cf9dff700 1 -- 172.21.15.69:0/176340854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cf40730a0 msgr2=0x7f8cf4073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:58.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.778+0000 7f8cf9dff700 1 --2- 172.21.15.69:0/176340854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cf40730a0 0x7f8cf4073510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009990 tx=0x7f8ce8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:16:58.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.778+0000 7f8cf9dff700 1 -- 172.21.15.69:0/176340854 shutdown_connections 2023-07-06T20:16:58.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.778+0000 7f8cf9dff700 1 --2- 172.21.15.69:0/176340854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cf410c200 0x7f8cf410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:58.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.778+0000 7f8cf9dff700 1 --2- 172.21.15.69:0/176340854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf4074680 0x7f8cf4072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:58.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.778+0000 7f8cf9dff700 1 --2- 172.21.15.69:0/176340854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cf40730a0 0x7f8cf4073510 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:58.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.778+0000 7f8cf9dff700 1 -- 172.21.15.69:0/176340854 >> 172.21.15.69:0/176340854 conn(0x7f8cf40fb210 msgr2=0x7f8cf40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:58.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.778+0000 7f8cf9dff700 1 -- 172.21.15.69:0/176340854 shutdown_connections 2023-07-06T20:16:58.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.779+0000 7f8cf9dff700 1 -- 172.21.15.69:0/176340854 wait complete. 2023-07-06T20:16:58.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.780+0000 7f8cf9dff700 1 Processor -- start 2023-07-06T20:16:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.780+0000 7f8cf9dff700 1 -- start start 2023-07-06T20:16:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf9dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cf40730a0 0x7f8cf419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf9dff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf4074680 0x7f8cf419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf9dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cf410c200 0x7f8cf41a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf9dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf41a3b20 con 0x7f8cf410c200 2023-07-06T20:16:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf9dff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf41a3c60 con 0x7f8cf4074680 2023-07-06T20:16:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf9dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf41a3df0 con 0x7f8cf40730a0 2023-07-06T20:16:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cf410c200 0x7f8cf41a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cf410c200 0x7f8cf41a3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60048/0 (socket says 172.21.15.69:60048) 2023-07-06T20:16:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf3fff700 1 -- 172.21.15.69:0/2104422584 learned_addr learned my addr 172.21.15.69:0/2104422584 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:16:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf2ffd700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf4074680 0x7f8cf419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf37fe700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cf40730a0 0x7f8cf419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf2ffd700 1 -- 172.21.15.69:0/2104422584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cf40730a0 msgr2=0x7f8cf419bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf2ffd700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cf40730a0 0x7f8cf419bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf2ffd700 1 -- 172.21.15.69:0/2104422584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cf410c200 msgr2=0x7f8cf41a3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf2ffd700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cf410c200 0x7f8cf41a3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf2ffd700 1 -- 172.21.15.69:0/2104422584 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce80095f0 con 0x7f8cf4074680 2023-07-06T20:16:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.781+0000 7f8cf3fff700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cf410c200 0x7f8cf41a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:16:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.782+0000 7f8cf2ffd700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf4074680 0x7f8cf419c4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800b680 tx=0x7f8ce8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:58.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.782+0000 7f8cf0ff9700 1 -- 172.21.15.69:0/2104422584 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce801d070 con 0x7f8cf4074680 2023-07-06T20:16:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.782+0000 7f8cf0ff9700 1 -- 172.21.15.69:0/2104422584 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ce8021410 con 0x7f8cf4074680 2023-07-06T20:16:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.782+0000 7f8cf9dff700 1 -- 172.21.15.69:0/2104422584 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf41a4070 con 0x7f8cf4074680 2023-07-06T20:16:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.782+0000 7f8cf0ff9700 1 -- 172.21.15.69:0/2104422584 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8017650 con 0x7f8cf4074680 2023-07-06T20:16:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.782+0000 7f8cf9dff700 1 -- 172.21.15.69:0/2104422584 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf41a4490 con 0x7f8cf4074680 2023-07-06T20:16:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.783+0000 7f8cf9dff700 1 -- 172.21.15.69:0/2104422584 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf4101d50 con 0x7f8cf4074680 2023-07-06T20:16:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.785+0000 7f8cf0ff9700 1 -- 172.21.15.69:0/2104422584 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8ce80217f0 con 0x7f8cf4074680 2023-07-06T20:16:58.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.787+0000 7f8cf0ff9700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f8cdc0a11a0 0x7f8cdc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:16:58.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.787+0000 7f8cf0ff9700 1 -- 172.21.15.69:0/2104422584 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8ce80cdaf0 con 0x7f8cf4074680 2023-07-06T20:16:58.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.787+0000 7f8cf37fe700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f8cdc0a11a0 0x7f8cdc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:16:58.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.788+0000 7f8cf37fe700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f8cdc0a11a0 0x7f8cdc0a3650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4074a50 tx=0x7f8ce4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:16:58.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:58.789+0000 7f8cf0ff9700 1 -- 172.21.15.69:0/2104422584 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8ce80976d0 con 0x7f8cf4074680 2023-07-06T20:16:59.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.077+0000 7f8cf9dff700 1 -- 172.21.15.69:0/2104422584 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cf402c8e0 con 0x7f8cf4074680 2023-07-06T20:16:59.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.079+0000 7f8cf0ff9700 1 -- 172.21.15.69:0/2104422584 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7f8ce809acf0 con 0x7f8cf4074680 2023-07-06T20:16:59.080 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:16:59.080 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-06T20:16:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.081+0000 7f8cf9dff700 1 -- 172.21.15.69:0/2104422584 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f8cdc0a11a0 msgr2=0x7f8cdc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.081+0000 7f8cf9dff700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f8cdc0a11a0 0x7f8cdc0a3650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4074a50 tx=0x7f8ce4009260 comp rx=0 tx=0).stop 2023-07-06T20:16:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.081+0000 7f8cf9dff700 1 -- 172.21.15.69:0/2104422584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf4074680 msgr2=0x7f8cf419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:16:59.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.081+0000 7f8cf9dff700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf4074680 0x7f8cf419c4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800b680 tx=0x7f8ce8005740 comp rx=0 tx=0).stop 2023-07-06T20:16:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.081+0000 7f8cf9dff700 1 -- 172.21.15.69:0/2104422584 shutdown_connections 2023-07-06T20:16:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.081+0000 7f8cf9dff700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f8cdc0a11a0 0x7f8cdc0a3650 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.081+0000 7f8cf9dff700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cf410c200 0x7f8cf41a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.081+0000 7f8cf9dff700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cf40730a0 0x7f8cf419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.081+0000 7f8cf9dff700 1 --2- 172.21.15.69:0/2104422584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cf4074680 0x7f8cf419c4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:16:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.081+0000 7f8cf9dff700 1 -- 172.21.15.69:0/2104422584 >> 172.21.15.69:0/2104422584 conn(0x7f8cf40fb210 msgr2=0x7f8cf40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:16:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.081+0000 7f8cf9dff700 1 -- 172.21.15.69:0/2104422584 shutdown_connections 2023-07-06T20:16:59.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:16:59.081+0000 7f8cf9dff700 1 -- 172.21.15.69:0/2104422584 wait complete. 2023-07-06T20:16:59.084 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 3 2023-07-06T20:16:59.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:58 smithi069 ceph-mon[126399]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:59.259 INFO:journalctl@ceph.mds.j.smithi069.stdout:Jul 06 20:16:58 smithi069 systemd[1]: Starting Ceph mds.j for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:16:59.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:58 smithi156 ceph-mon[129280]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:59.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:58 smithi098 ceph-mon[128968]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:16:59.735 INFO:journalctl@ceph.mds.j.smithi069.stdout:Jul 06 20:16:59 smithi069 podman[160025]: 2023-07-06T20:16:59.735 INFO:journalctl@ceph.mds.j.smithi069.stdout:Jul 06 20:16:59 smithi069 ceph-mds[160065]: -- 172.21.15.69:0/3251233811 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5584d1ed71e0 con 0x5584d1eac800 2023-07-06T20:16:59.736 INFO:journalctl@ceph.mds.j.smithi069.stdout:Jul 06 20:16:59 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-j[160041]: starting mds.j at 2023-07-06T20:16:59.736 INFO:journalctl@ceph.mds.j.smithi069.stdout:Jul 06 20:16:59 smithi069 bash[160025]: e88abd072023dc792400532130fff244da4a8598f4d1351a84f5439c2fc9908e 2023-07-06T20:17:00.009 INFO:journalctl@ceph.mds.j.smithi069.stdout:Jul 06 20:16:59 smithi069 systemd[1]: Started Ceph mds.j for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:17:00.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:59 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2104422584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:00.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:59 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:00.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:59 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:00.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:59 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:00.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:59 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:00.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:59 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:00.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:16:59 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:00.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:59 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2104422584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:00.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:59 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:00.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:59 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:00.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:59 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:00.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:59 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:00.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:59 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:00.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:16:59 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:00.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:59 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2104422584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:00.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:59 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:00.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:59 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:00.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:59 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:00.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:59 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:00.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:59 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:00.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:16:59 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:00 smithi069 ceph-mon[126399]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:00 smithi069 ceph-mon[126399]: Deploying daemon mds.d on smithi069 2023-07-06T20:17:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:00 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.69:6834/3728004874,v1:172.21.15.69:6835/3728004874] up:boot 2023-07-06T20:17:01.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:00 smithi069 ceph-mon[126399]: fsmap 3 up:standby 2023-07-06T20:17:01.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-06T20:17:01.103 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:17:01.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.308+0000 7f4fd03ff700 1 -- 172.21.15.69:0/3666072072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fc80730a0 msgr2=0x7f4fc8073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:01.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.308+0000 7f4fd03ff700 1 --2- 172.21.15.69:0/3666072072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fc80730a0 0x7f4fc8073510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009990 tx=0x7f4fb8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:01.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.308+0000 7f4fd03ff700 1 -- 172.21.15.69:0/3666072072 shutdown_connections 2023-07-06T20:17:01.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.308+0000 7f4fd03ff700 1 --2- 172.21.15.69:0/3666072072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc810c200 0x7f4fc810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:01.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.308+0000 7f4fd03ff700 1 --2- 172.21.15.69:0/3666072072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc8074680 0x7f4fc8072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:01.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.308+0000 7f4fd03ff700 1 --2- 172.21.15.69:0/3666072072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fc80730a0 0x7f4fc8073510 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:01.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.308+0000 7f4fd03ff700 1 -- 172.21.15.69:0/3666072072 >> 172.21.15.69:0/3666072072 conn(0x7f4fc80fb210 msgr2=0x7f4fc80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:01.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.309+0000 7f4fd03ff700 1 -- 172.21.15.69:0/3666072072 shutdown_connections 2023-07-06T20:17:01.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.309+0000 7f4fd03ff700 1 -- 172.21.15.69:0/3666072072 wait complete. 2023-07-06T20:17:01.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.310+0000 7f4fd03ff700 1 Processor -- start 2023-07-06T20:17:01.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.310+0000 7f4fd03ff700 1 -- start start 2023-07-06T20:17:01.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.310+0000 7f4fd03ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fc80730a0 0x7f4fc819bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:01.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.310+0000 7f4fd03ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc8074680 0x7f4fc819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:01.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.310+0000 7f4fd03ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc810c200 0x7f4fc81a3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:01.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.310+0000 7f4fd03ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81a3ae0 con 0x7f4fc80730a0 2023-07-06T20:17:01.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.310+0000 7f4fd03ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81a3c20 con 0x7f4fc810c200 2023-07-06T20:17:01.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.310+0000 7f4fd03ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81a3db0 con 0x7f4fc8074680 2023-07-06T20:17:01.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.311+0000 7f4fce19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fc80730a0 0x7f4fc819bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:01.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.311+0000 7f4fce19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fc80730a0 0x7f4fc819bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60076/0 (socket says 172.21.15.69:60076) 2023-07-06T20:17:01.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.311+0000 7f4fce19b700 1 -- 172.21.15.69:0/1808427143 learned_addr learned my addr 172.21.15.69:0/1808427143 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:01.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.311+0000 7f4fcd99a700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc8074680 0x7f4fc819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:01.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.311+0000 7f4fce99c700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc810c200 0x7f4fc81a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:01.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.311+0000 7f4fce19b700 1 -- 172.21.15.69:0/1808427143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc8074680 msgr2=0x7f4fc819c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:01.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.311+0000 7f4fce19b700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc8074680 0x7f4fc819c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:01.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.311+0000 7f4fce19b700 1 -- 172.21.15.69:0/1808427143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc810c200 msgr2=0x7f4fc81a3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:01.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.311+0000 7f4fce19b700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc810c200 0x7f4fc81a3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:01.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.311+0000 7f4fce19b700 1 -- 172.21.15.69:0/1808427143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb80095f0 con 0x7f4fc80730a0 2023-07-06T20:17:01.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.311+0000 7f4fce19b700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fc80730a0 0x7f4fc819bf60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400e970 tx=0x7f4fc400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:01.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.311+0000 7f4fbf7fe700 1 -- 172.21.15.69:0/1808427143 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4009790 con 0x7f4fc80730a0 2023-07-06T20:17:01.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.312+0000 7f4fbf7fe700 1 -- 172.21.15.69:0/1808427143 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc400c7e0 con 0x7f4fc80730a0 2023-07-06T20:17:01.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.312+0000 7f4fd03ff700 1 -- 172.21.15.69:0/1808427143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc81a4090 con 0x7f4fc80730a0 2023-07-06T20:17:01.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.312+0000 7f4fbf7fe700 1 -- 172.21.15.69:0/1808427143 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40054e0 con 0x7f4fc80730a0 2023-07-06T20:17:01.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.312+0000 7f4fd03ff700 1 -- 172.21.15.69:0/1808427143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc81a45f0 con 0x7f4fc80730a0 2023-07-06T20:17:01.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.313+0000 7f4fd03ff700 1 -- 172.21.15.69:0/1808427143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc8101d50 con 0x7f4fc80730a0 2023-07-06T20:17:01.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.315+0000 7f4fbf7fe700 1 -- 172.21.15.69:0/1808427143 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4fc400c950 con 0x7f4fc80730a0 2023-07-06T20:17:01.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.317+0000 7f4fbf7fe700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4fb40a1220 0x7f4fb40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:01.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.317+0000 7f4fbf7fe700 1 -- 172.21.15.69:0/1808427143 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4fc4014070 con 0x7f4fc80730a0 2023-07-06T20:17:01.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.318+0000 7f4fcd99a700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4fb40a1220 0x7f4fb40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:01.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.318+0000 7f4fcd99a700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4fb40a1220 0x7f4fb40a36d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800b680 tx=0x7f4fb8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:01.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.319+0000 7f4fbf7fe700 1 -- 172.21.15.69:0/1808427143 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4fc4096460 con 0x7f4fc80730a0 2023-07-06T20:17:01.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:00 smithi156 ceph-mon[129280]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:01.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:00 smithi156 ceph-mon[129280]: Deploying daemon mds.d on smithi069 2023-07-06T20:17:01.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:00 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.69:6834/3728004874,v1:172.21.15.69:6835/3728004874] up:boot 2023-07-06T20:17:01.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:00 smithi156 ceph-mon[129280]: fsmap 3 up:standby 2023-07-06T20:17:01.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-06T20:17:01.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:00 smithi098 ceph-mon[128968]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:01.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:00 smithi098 ceph-mon[128968]: Deploying daemon mds.d on smithi069 2023-07-06T20:17:01.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:00 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.69:6834/3728004874,v1:172.21.15.69:6835/3728004874] up:boot 2023-07-06T20:17:01.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:00 smithi098 ceph-mon[128968]: fsmap 3 up:standby 2023-07-06T20:17:01.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-06T20:17:01.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.596+0000 7f4fd03ff700 1 -- 172.21.15.69:0/1808427143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fc802c8e0 con 0x7f4fc80730a0 2023-07-06T20:17:01.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.597+0000 7f4fbf7fe700 1 -- 172.21.15.69:0/1808427143 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7f4fc4099a80 con 0x7f4fc80730a0 2023-07-06T20:17:01.599 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:01.599 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-06T20:17:01.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.599+0000 7f4fd03ff700 1 -- 172.21.15.69:0/1808427143 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4fb40a1220 msgr2=0x7f4fb40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:01.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.599+0000 7f4fd03ff700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4fb40a1220 0x7f4fb40a36d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800b680 tx=0x7f4fb8005fd0 comp rx=0 tx=0).stop 2023-07-06T20:17:01.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.599+0000 7f4fd03ff700 1 -- 172.21.15.69:0/1808427143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fc80730a0 msgr2=0x7f4fc819bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.599+0000 7f4fd03ff700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fc80730a0 0x7f4fc819bf60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400e970 tx=0x7f4fc400ec80 comp rx=0 tx=0).stop 2023-07-06T20:17:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.599+0000 7f4fd03ff700 1 -- 172.21.15.69:0/1808427143 shutdown_connections 2023-07-06T20:17:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.599+0000 7f4fd03ff700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4fb40a1220 0x7f4fb40a36d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.599+0000 7f4fd03ff700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc810c200 0x7f4fc81a3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.599+0000 7f4fd03ff700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fc80730a0 0x7f4fc819bf60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.599+0000 7f4fd03ff700 1 --2- 172.21.15.69:0/1808427143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fc8074680 0x7f4fc819c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.599+0000 7f4fd03ff700 1 -- 172.21.15.69:0/1808427143 >> 172.21.15.69:0/1808427143 conn(0x7f4fc80fb210 msgr2=0x7f4fc80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:01.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.599+0000 7f4fd03ff700 1 -- 172.21.15.69:0/1808427143 shutdown_connections 2023-07-06T20:17:01.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:01.599+0000 7f4fd03ff700 1 -- 172.21.15.69:0/1808427143 wait complete. 2023-07-06T20:17:01.603 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 4 2023-07-06T20:17:02.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:01 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1808427143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:02.259 INFO:journalctl@ceph.mds.d.smithi069.stdout:Jul 06 20:17:01 smithi069 systemd[1]: Starting Ceph mds.d for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:17:02.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:01 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1808427143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:02.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:01 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1808427143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:02.630 INFO:journalctl@ceph.mds.d.smithi069.stdout:Jul 06 20:17:02 smithi069 podman[160414]: 2023-07-06T20:17:02.941 INFO:journalctl@ceph.mds.d.smithi069.stdout:Jul 06 20:17:02 smithi069 bash[160414]: 83dd9508bcaba346cccc26c44066ae023ebe4c4592fef6966a175e7e8624713f 2023-07-06T20:17:02.941 INFO:journalctl@ceph.mds.d.smithi069.stdout:Jul 06 20:17:02 smithi069 ceph-mds[160455]: -- 172.21.15.69:0/105197107 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563d287c51e0 con 0x563d2879ac00 2023-07-06T20:17:02.941 INFO:journalctl@ceph.mds.d.smithi069.stdout:Jul 06 20:17:02 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-d[160431]: starting mds.d at 2023-07-06T20:17:02.941 INFO:journalctl@ceph.mds.d.smithi069.stdout:Jul 06 20:17:02 smithi069 systemd[1]: Started Ceph mds.d for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:17:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:02 smithi069 ceph-mon[126399]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:03.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:03.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:02 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:03.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:02 smithi156 ceph-mon[129280]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:03.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:03.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:03.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:03.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:03.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:03.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:02 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:03.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:02 smithi098 ceph-mon[128968]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:03.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:03.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:03.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:03.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:03.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:03.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:02 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:03.620 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:17:03.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.828+0000 7f6eb6fff700 1 -- 172.21.15.69:0/3014057945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6eb00ffc30 msgr2=0x7f6eb0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:03.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.828+0000 7f6eb6fff700 1 --2- 172.21.15.69:0/3014057945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6eb00ffc30 0x7f6eb0100040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009990 tx=0x7f6eac009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:03.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.828+0000 7f6eb6fff700 1 -- 172.21.15.69:0/3014057945 shutdown_connections 2023-07-06T20:17:03.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.828+0000 7f6eb6fff700 1 --2- 172.21.15.69:0/3014057945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eb01017b0 0x7f6eb0108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:03.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.828+0000 7f6eb6fff700 1 --2- 172.21.15.69:0/3014057945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6eb00ffc30 0x7f6eb0100040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:03.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.828+0000 7f6eb6fff700 1 --2- 172.21.15.69:0/3014057945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6eb0100e20 0x7f6eb0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:03.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.828+0000 7f6eb6fff700 1 -- 172.21.15.69:0/3014057945 >> 172.21.15.69:0/3014057945 conn(0x7f6eb00fb210 msgr2=0x7f6eb00fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:03.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.829+0000 7f6eb6fff700 1 -- 172.21.15.69:0/3014057945 shutdown_connections 2023-07-06T20:17:03.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.829+0000 7f6eb6fff700 1 -- 172.21.15.69:0/3014057945 wait complete. 2023-07-06T20:17:03.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.830+0000 7f6eb6fff700 1 Processor -- start 2023-07-06T20:17:03.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.830+0000 7f6eb6fff700 1 -- start start 2023-07-06T20:17:03.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb6fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6eb00ffc30 0x7f6eb0197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:03.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb6fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6eb0100e20 0x7f6eb01980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:03.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb6fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eb01017b0 0x7f6eb019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:03.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb6fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb019f6e0 con 0x7f6eb00ffc30 2023-07-06T20:17:03.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb6fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb019f820 con 0x7f6eb01017b0 2023-07-06T20:17:03.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb6fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb019f9b0 con 0x7f6eb0100e20 2023-07-06T20:17:03.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb4d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6eb00ffc30 0x7f6eb0197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:03.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb4d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6eb00ffc30 0x7f6eb0197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59140/0 (socket says 172.21.15.69:59140) 2023-07-06T20:17:03.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb4d9b700 1 -- 172.21.15.69:0/2389274860 learned_addr learned my addr 172.21.15.69:0/2389274860 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:03.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6ea7fff700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6eb0100e20 0x7f6eb01980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:03.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb559c700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eb01017b0 0x7f6eb019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:03.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb4d9b700 1 -- 172.21.15.69:0/2389274860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6eb0100e20 msgr2=0x7f6eb01980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:03.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb4d9b700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6eb0100e20 0x7f6eb01980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:03.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb4d9b700 1 -- 172.21.15.69:0/2389274860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eb01017b0 msgr2=0x7f6eb019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:03.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb4d9b700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eb01017b0 0x7f6eb019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:03.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.831+0000 7f6eb4d9b700 1 -- 172.21.15.69:0/2389274860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac0095f0 con 0x7f6eb00ffc30 2023-07-06T20:17:03.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.832+0000 7f6eb4d9b700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6eb00ffc30 0x7f6eb0197b60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6eac004760 tx=0x7f6eac009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:03.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.832+0000 7f6ea5ffb700 1 -- 172.21.15.69:0/2389274860 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac01d070 con 0x7f6eb00ffc30 2023-07-06T20:17:03.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.832+0000 7f6eb6fff700 1 -- 172.21.15.69:0/2389274860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb019fc30 con 0x7f6eb00ffc30 2023-07-06T20:17:03.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.832+0000 7f6ea5ffb700 1 -- 172.21.15.69:0/2389274860 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eac004020 con 0x7f6eb00ffc30 2023-07-06T20:17:03.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.832+0000 7f6ea5ffb700 1 -- 172.21.15.69:0/2389274860 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac005030 con 0x7f6eb00ffc30 2023-07-06T20:17:03.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.832+0000 7f6eb6fff700 1 -- 172.21.15.69:0/2389274860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb01a0050 con 0x7f6eb00ffc30 2023-07-06T20:17:03.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.833+0000 7f6eb6fff700 1 -- 172.21.15.69:0/2389274860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb0192310 con 0x7f6eb00ffc30 2023-07-06T20:17:03.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.835+0000 7f6ea5ffb700 1 -- 172.21.15.69:0/2389274860 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6eac003b60 con 0x7f6eb00ffc30 2023-07-06T20:17:03.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.837+0000 7f6ea5ffb700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6e980a1220 0x7f6e980a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:03.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.837+0000 7f6ea5ffb700 1 -- 172.21.15.69:0/2389274860 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6eac0cd400 con 0x7f6eb00ffc30 2023-07-06T20:17:03.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.838+0000 7f6ea7fff700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6e980a1220 0x7f6e980a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:03.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.838+0000 7f6ea7fff700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6e980a1220 0x7f6e980a36d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0005950 tx=0x7f6ea000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:03.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:03.841+0000 7f6ea5ffb700 1 -- 172.21.15.69:0/2389274860 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6eac096fe0 con 0x7f6eb00ffc30 2023-07-06T20:17:04.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.084+0000 7f6eb6fff700 1 -- 172.21.15.69:0/2389274860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6eb002c8e0 con 0x7f6eb00ffc30 2023-07-06T20:17:04.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.085+0000 7f6ea5ffb700 1 -- 172.21.15.69:0/2389274860 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3528 (secure 0 0 0) 0x7f6eac026de0 con 0x7f6eb00ffc30 2023-07-06T20:17:04.087 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:04.087 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-06T20:17:04.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.087+0000 7f6eb6fff700 1 -- 172.21.15.69:0/2389274860 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6e980a1220 msgr2=0x7f6e980a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:04.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.087+0000 7f6eb6fff700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6e980a1220 0x7f6e980a36d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0005950 tx=0x7f6ea000b410 comp rx=0 tx=0).stop 2023-07-06T20:17:04.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.087+0000 7f6eb6fff700 1 -- 172.21.15.69:0/2389274860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6eb00ffc30 msgr2=0x7f6eb0197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:04.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.087+0000 7f6eb6fff700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6eb00ffc30 0x7f6eb0197b60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6eac004760 tx=0x7f6eac009580 comp rx=0 tx=0).stop 2023-07-06T20:17:04.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.088+0000 7f6eb6fff700 1 -- 172.21.15.69:0/2389274860 shutdown_connections 2023-07-06T20:17:04.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.088+0000 7f6eb6fff700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6e980a1220 0x7f6e980a36d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:04.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.088+0000 7f6eb6fff700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eb01017b0 0x7f6eb019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:04.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.088+0000 7f6eb6fff700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6eb00ffc30 0x7f6eb0197b60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:04.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.088+0000 7f6eb6fff700 1 --2- 172.21.15.69:0/2389274860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6eb0100e20 0x7f6eb01980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:04.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.088+0000 7f6eb6fff700 1 -- 172.21.15.69:0/2389274860 >> 172.21.15.69:0/2389274860 conn(0x7f6eb00fb210 msgr2=0x7f6eb01066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:04.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.088+0000 7f6eb6fff700 1 -- 172.21.15.69:0/2389274860 shutdown_connections 2023-07-06T20:17:04.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:04.088+0000 7f6eb6fff700 1 -- 172.21.15.69:0/2389274860 wait complete. 2023-07-06T20:17:04.091 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 5 2023-07-06T20:17:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:03 smithi098 ceph-mon[128968]: Deploying daemon mds.e on smithi098 2023-07-06T20:17:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:03 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.69:6836/896595344,v1:172.21.15.69:6837/896595344] up:boot 2023-07-06T20:17:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:03 smithi098 ceph-mon[128968]: fsmap 4 up:standby 2023-07-06T20:17:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:03 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-06T20:17:04.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:03 smithi069 ceph-mon[126399]: Deploying daemon mds.e on smithi098 2023-07-06T20:17:04.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:03 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.69:6836/896595344,v1:172.21.15.69:6837/896595344] up:boot 2023-07-06T20:17:04.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:03 smithi069 ceph-mon[126399]: fsmap 4 up:standby 2023-07-06T20:17:04.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:03 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-06T20:17:04.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:03 smithi156 ceph-mon[129280]: Deploying daemon mds.e on smithi098 2023-07-06T20:17:04.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:03 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.69:6836/896595344,v1:172.21.15.69:6837/896595344] up:boot 2023-07-06T20:17:04.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:03 smithi156 ceph-mon[129280]: fsmap 4 up:standby 2023-07-06T20:17:04.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:03 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-06T20:17:04.846 INFO:journalctl@ceph.mds.e.smithi098.stdout:Jul 06 20:17:04 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-e[153716]: starting mds.e at 2023-07-06T20:17:04.846 INFO:journalctl@ceph.mds.e.smithi098.stdout:Jul 06 20:17:04 smithi098 bash[153700]: 9ad1d4ad1424f96ea71fc54012393e4faca882b56cc012dc66fdb5a1e34171f5 2023-07-06T20:17:05.125 INFO:journalctl@ceph.mds.e.smithi098.stdout:Jul 06 20:17:04 smithi098 systemd[1]: Started Ceph mds.e for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:17:05.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:04 smithi098 ceph-mon[128968]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:05.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:04 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2389274860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:05.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:04 smithi069 ceph-mon[126399]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:05.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:04 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2389274860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:05.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:04 smithi156 ceph-mon[129280]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:05.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:04 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2389274860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:06.109 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:17:06.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:05 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:06.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:05 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:06.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:05 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:06.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:05 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:06.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:05 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:06.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:05 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:06.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:05 smithi069 ceph-mon[126399]: Deploying daemon mds.f on smithi156 2023-07-06T20:17:06.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:05 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] up:boot 2023-07-06T20:17:06.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:05 smithi069 ceph-mon[126399]: fsmap 5 up:standby 2023-07-06T20:17:06.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:05 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-06T20:17:06.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:05 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:06.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:05 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:06.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:05 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:06.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:05 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:06.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:05 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:06.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:05 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:06.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:05 smithi156 ceph-mon[129280]: Deploying daemon mds.f on smithi156 2023-07-06T20:17:06.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:05 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] up:boot 2023-07-06T20:17:06.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:05 smithi156 ceph-mon[129280]: fsmap 5 up:standby 2023-07-06T20:17:06.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:05 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-06T20:17:06.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.313+0000 7f14105ff700 1 -- 172.21.15.69:0/3958978698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1408100e20 msgr2=0x7f1408101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:06.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.313+0000 7f14105ff700 1 --2- 172.21.15.69:0/3958978698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1408100e20 0x7f1408101270 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009990 tx=0x7f13f8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:06.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.314+0000 7f14105ff700 1 -- 172.21.15.69:0/3958978698 shutdown_connections 2023-07-06T20:17:06.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.314+0000 7f14105ff700 1 --2- 172.21.15.69:0/3958978698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14081017b0 0x7f1408108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:06.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.314+0000 7f14105ff700 1 --2- 172.21.15.69:0/3958978698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14080ffc30 0x7f1408100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:06.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.314+0000 7f14105ff700 1 --2- 172.21.15.69:0/3958978698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1408100e20 0x7f1408101270 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:06.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.314+0000 7f14105ff700 1 -- 172.21.15.69:0/3958978698 >> 172.21.15.69:0/3958978698 conn(0x7f14080fb210 msgr2=0x7f14080fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:06.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.314+0000 7f14105ff700 1 -- 172.21.15.69:0/3958978698 shutdown_connections 2023-07-06T20:17:06.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.314+0000 7f14105ff700 1 -- 172.21.15.69:0/3958978698 wait complete. 2023-07-06T20:17:06.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.315+0000 7f14105ff700 1 Processor -- start 2023-07-06T20:17:06.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.315+0000 7f14105ff700 1 -- start start 2023-07-06T20:17:06.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.315+0000 7f14105ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14080ffc30 0x7f14081959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:06.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.315+0000 7f14105ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1408100e20 0x7f1408195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:06.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.315+0000 7f14105ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14081017b0 0x7f140819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:06.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.315+0000 7f14105ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140819d540 con 0x7f1408100e20 2023-07-06T20:17:06.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.315+0000 7f14105ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140819d680 con 0x7f14081017b0 2023-07-06T20:17:06.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.315+0000 7f14105ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140819d810 con 0x7f14080ffc30 2023-07-06T20:17:06.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.315+0000 7f140e39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14080ffc30 0x7f14081959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:06.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.315+0000 7f140db9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1408100e20 0x7f1408195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:06.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.315+0000 7f140e39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14080ffc30 0x7f14081959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:41166/0 (socket says 172.21.15.69:41166) 2023-07-06T20:17:06.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.315+0000 7f140e39b700 1 -- 172.21.15.69:0/2382654428 learned_addr learned my addr 172.21.15.69:0/2382654428 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:06.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f140eb9c700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14081017b0 0x7f140819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:06.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f140db9a700 1 -- 172.21.15.69:0/2382654428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14080ffc30 msgr2=0x7f14081959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:06.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f140db9a700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14080ffc30 0x7f14081959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:06.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f140db9a700 1 -- 172.21.15.69:0/2382654428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14081017b0 msgr2=0x7f140819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:06.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f140db9a700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14081017b0 0x7f140819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:06.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f140db9a700 1 -- 172.21.15.69:0/2382654428 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f80095f0 con 0x7f1408100e20 2023-07-06T20:17:06.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f140eb9c700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14081017b0 0x7f140819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:06.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f140e39b700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14080ffc30 0x7f14081959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:06.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f140db9a700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1408100e20 0x7f1408195f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13f800b620 tx=0x7f13f8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:06.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f13ff7fe700 1 -- 172.21.15.69:0/2382654428 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f801d070 con 0x7f1408100e20 2023-07-06T20:17:06.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f13ff7fe700 1 -- 172.21.15.69:0/2382654428 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f8021410 con 0x7f1408100e20 2023-07-06T20:17:06.322 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f14105ff700 1 -- 172.21.15.69:0/2382654428 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140819da90 con 0x7f1408100e20 2023-07-06T20:17:06.322 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.316+0000 7f13ff7fe700 1 -- 172.21.15.69:0/2382654428 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8017650 con 0x7f1408100e20 2023-07-06T20:17:06.322 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.317+0000 7f14105ff700 1 -- 172.21.15.69:0/2382654428 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140819df10 con 0x7f1408100e20 2023-07-06T20:17:06.322 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.318+0000 7f14105ff700 1 -- 172.21.15.69:0/2382654428 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1408107180 con 0x7f1408100e20 2023-07-06T20:17:06.323 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.319+0000 7f13ff7fe700 1 -- 172.21.15.69:0/2382654428 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f13f8021580 con 0x7f1408100e20 2023-07-06T20:17:06.323 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.321+0000 7f13ff7fe700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f13f40a12a0 0x7f13f40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:06.323 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.321+0000 7f13ff7fe700 1 -- 172.21.15.69:0/2382654428 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f13f80cd8e0 con 0x7f1408100e20 2023-07-06T20:17:06.323 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.321+0000 7f140e39b700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f13f40a12a0 0x7f13f40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:06.324 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.322+0000 7f140e39b700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f13f40a12a0 0x7f13f40a3750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1404005950 tx=0x7f1404009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:06.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.324+0000 7f13ff7fe700 1 -- 172.21.15.69:0/2382654428 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f13f80974c0 con 0x7f1408100e20 2023-07-06T20:17:06.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:05 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:06.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:05 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:06.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:05 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:06.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:05 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:06.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:05 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:06.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:05 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:06.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:05 smithi098 ceph-mon[128968]: Deploying daemon mds.f on smithi156 2023-07-06T20:17:06.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:05 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] up:boot 2023-07-06T20:17:06.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:05 smithi098 ceph-mon[128968]: fsmap 5 up:standby 2023-07-06T20:17:06.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:05 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-06T20:17:06.566 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.565+0000 7f14105ff700 1 -- 172.21.15.69:0/2382654428 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f140802c8e0 con 0x7f1408100e20 2023-07-06T20:17:06.567 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.565+0000 7f13ff7fe700 1 -- 172.21.15.69:0/2382654428 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4282 (secure 0 0 0) 0x7f13f809aae0 con 0x7f1408100e20 2023-07-06T20:17:06.567 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:06.568 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-06T20:17:06.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.568+0000 7f14105ff700 1 -- 172.21.15.69:0/2382654428 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f13f40a12a0 msgr2=0x7f13f40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:06.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.568+0000 7f14105ff700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f13f40a12a0 0x7f13f40a3750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1404005950 tx=0x7f1404009260 comp rx=0 tx=0).stop 2023-07-06T20:17:06.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.568+0000 7f14105ff700 1 -- 172.21.15.69:0/2382654428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1408100e20 msgr2=0x7f1408195f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:06.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.568+0000 7f14105ff700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1408100e20 0x7f1408195f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13f800b620 tx=0x7f13f8005740 comp rx=0 tx=0).stop 2023-07-06T20:17:06.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.568+0000 7f14105ff700 1 -- 172.21.15.69:0/2382654428 shutdown_connections 2023-07-06T20:17:06.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.568+0000 7f14105ff700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f13f40a12a0 0x7f13f40a3750 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:06.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.568+0000 7f14105ff700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14081017b0 0x7f140819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:06.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.568+0000 7f14105ff700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14080ffc30 0x7f14081959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:06.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.568+0000 7f14105ff700 1 --2- 172.21.15.69:0/2382654428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1408100e20 0x7f1408195f00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:06.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.568+0000 7f14105ff700 1 -- 172.21.15.69:0/2382654428 >> 172.21.15.69:0/2382654428 conn(0x7f14080fb210 msgr2=0x7f14081042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:06.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.568+0000 7f14105ff700 1 -- 172.21.15.69:0/2382654428 shutdown_connections 2023-07-06T20:17:06.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:06.568+0000 7f14105ff700 1 -- 172.21.15.69:0/2382654428 wait complete. 2023-07-06T20:17:06.572 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2023-07-06T20:17:06.857 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:17:06 smithi156 systemd[1]: Starting Ceph mds.f for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:17:07.109 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:17:06 smithi156 podman[154649]: 2023-07-06T20:17:07.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:06 smithi156 ceph-mon[129280]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:07.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:06 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2382654428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:07.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:06 smithi069 ceph-mon[126399]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:07.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:06 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2382654428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:07.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:06 smithi098 ceph-mon[128968]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:07.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:06 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2382654428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:07.606 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:17:07 smithi156 ceph-mds[154689]: -- 172.21.15.156:0/1869872770 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f5c5a491e0 con 0x55f5c5a1e800 2023-07-06T20:17:07.607 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:17:07 smithi156 bash[154649]: 147cd4571e9e25f3b16660b2f1e74b789afe5a15d5fe5a5b31411b832f75d783 2023-07-06T20:17:07.607 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:17:07 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-f[154665]: starting mds.f at 2023-07-06T20:17:07.607 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:17:07 smithi156 systemd[1]: Started Ceph mds.f for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:17:08.495 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:08.495 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:08.495 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:08.495 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:08.495 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:08.495 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:08.496 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:08 smithi156 ceph-mon[129280]: Deploying daemon mds.c on smithi156 2023-07-06T20:17:08.496 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:08 smithi156 ceph-mon[129280]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:08.496 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:08 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.156:6834/2525319359,v1:172.21.15.156:6835/2525319359] up:boot 2023-07-06T20:17:08.496 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:08 smithi156 ceph-mon[129280]: fsmap 6 up:standby 2023-07-06T20:17:08.496 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-06T20:17:08.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:08.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:08.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:08.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:08.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:08.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:08.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:08 smithi069 ceph-mon[126399]: Deploying daemon mds.c on smithi156 2023-07-06T20:17:08.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:08 smithi069 ceph-mon[126399]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:08.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:08 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.156:6834/2525319359,v1:172.21.15.156:6835/2525319359] up:boot 2023-07-06T20:17:08.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:08 smithi069 ceph-mon[126399]: fsmap 6 up:standby 2023-07-06T20:17:08.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-06T20:17:08.589 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:17:08.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:08.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:08.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:08.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:08.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:08.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:08.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:08 smithi098 ceph-mon[128968]: Deploying daemon mds.c on smithi156 2023-07-06T20:17:08.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:08 smithi098 ceph-mon[128968]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:08.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:08 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.156:6834/2525319359,v1:172.21.15.156:6835/2525319359] up:boot 2023-07-06T20:17:08.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:08 smithi098 ceph-mon[128968]: fsmap 6 up:standby 2023-07-06T20:17:08.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-06T20:17:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.792+0000 7ffbe29ff700 1 -- 172.21.15.69:0/1061098849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbdc0ffc30 msgr2=0x7ffbdc100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.792+0000 7ffbe29ff700 1 --2- 172.21.15.69:0/1061098849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbdc0ffc30 0x7ffbdc100040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009990 tx=0x7ffbcc009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.792+0000 7ffbe29ff700 1 -- 172.21.15.69:0/1061098849 shutdown_connections 2023-07-06T20:17:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.792+0000 7ffbe29ff700 1 --2- 172.21.15.69:0/1061098849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbdc1017b0 0x7ffbdc108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.792+0000 7ffbe29ff700 1 --2- 172.21.15.69:0/1061098849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbdc0ffc30 0x7ffbdc100040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.792+0000 7ffbe29ff700 1 --2- 172.21.15.69:0/1061098849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbdc100e20 0x7ffbdc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.792+0000 7ffbe29ff700 1 -- 172.21.15.69:0/1061098849 >> 172.21.15.69:0/1061098849 conn(0x7ffbdc0fb210 msgr2=0x7ffbdc0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.792+0000 7ffbe29ff700 1 -- 172.21.15.69:0/1061098849 shutdown_connections 2023-07-06T20:17:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.792+0000 7ffbe29ff700 1 -- 172.21.15.69:0/1061098849 wait complete. 2023-07-06T20:17:08.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.793+0000 7ffbe29ff700 1 Processor -- start 2023-07-06T20:17:08.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.793+0000 7ffbe29ff700 1 -- start start 2023-07-06T20:17:08.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.793+0000 7ffbe29ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbdc0ffc30 0x7ffbdc0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:08.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.793+0000 7ffbe29ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbdc100e20 0x7ffbdc072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:08.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.793+0000 7ffbe29ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbdc1017b0 0x7ffbdc072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:08.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.793+0000 7ffbe29ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc073710 con 0x7ffbdc1017b0 2023-07-06T20:17:08.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.793+0000 7ffbe29ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc073880 con 0x7ffbdc0ffc30 2023-07-06T20:17:08.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbe29ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc19f8e0 con 0x7ffbdc100e20 2023-07-06T20:17:08.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbe0f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbdc1017b0 0x7ffbdc072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:08.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbdb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbdc100e20 0x7ffbdc072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:08.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbdb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbdc100e20 0x7ffbdc072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:41170/0 (socket says 172.21.15.69:41170) 2023-07-06T20:17:08.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbe0f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbdc1017b0 0x7ffbdc072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59164/0 (socket says 172.21.15.69:59164) 2023-07-06T20:17:08.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbdb7fe700 1 -- 172.21.15.69:0/724621172 learned_addr learned my addr 172.21.15.69:0/724621172 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:08.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbdbfff700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbdc0ffc30 0x7ffbdc0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:08.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbe0f9c700 1 -- 172.21.15.69:0/724621172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbdc100e20 msgr2=0x7ffbdc072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:08.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbe0f9c700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbdc100e20 0x7ffbdc072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:08.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbe0f9c700 1 -- 172.21.15.69:0/724621172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbdc0ffc30 msgr2=0x7ffbdc0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:08.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbe0f9c700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbdc0ffc30 0x7ffbdc0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:08.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbe0f9c700 1 -- 172.21.15.69:0/724621172 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd0009520 con 0x7ffbdc1017b0 2023-07-06T20:17:08.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbe0f9c700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbdc1017b0 0x7ffbdc072f70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400efc0 tx=0x7ffbd400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:08.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.794+0000 7ffbd97fa700 1 -- 172.21.15.69:0/724621172 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4018070 con 0x7ffbdc1017b0 2023-07-06T20:17:08.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.795+0000 7ffbe29ff700 1 -- 172.21.15.69:0/724621172 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbcc0095f0 con 0x7ffbdc1017b0 2023-07-06T20:17:08.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.795+0000 7ffbd97fa700 1 -- 172.21.15.69:0/724621172 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbd400f040 con 0x7ffbdc1017b0 2023-07-06T20:17:08.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.795+0000 7ffbd97fa700 1 -- 172.21.15.69:0/724621172 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd400cd40 con 0x7ffbdc1017b0 2023-07-06T20:17:08.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.795+0000 7ffbe29ff700 1 -- 172.21.15.69:0/724621172 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbdc19ff30 con 0x7ffbdc1017b0 2023-07-06T20:17:08.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.796+0000 7ffbe29ff700 1 -- 172.21.15.69:0/724621172 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbdc10b3a0 con 0x7ffbdc1017b0 2023-07-06T20:17:08.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.798+0000 7ffbd97fa700 1 -- 172.21.15.69:0/724621172 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ffbd4003a00 con 0x7ffbdc1017b0 2023-07-06T20:17:08.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.799+0000 7ffbd97fa700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffbc40a11a0 0x7ffbc40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:08.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.799+0000 7ffbd97fa700 1 -- 172.21.15.69:0/724621172 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffbd40cc0d0 con 0x7ffbdc1017b0 2023-07-06T20:17:08.802 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.799+0000 7ffbdbfff700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffbc40a11a0 0x7ffbc40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:08.802 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.800+0000 7ffbdbfff700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffbc40a11a0 0x7ffbc40a3650 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc006010 tx=0x7ffbcc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:08.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:08.803+0000 7ffbd97fa700 1 -- 172.21.15.69:0/724621172 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ffbd4095cb0 con 0x7ffbdc1017b0 2023-07-06T20:17:08.857 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:17:08 smithi156 systemd[1]: Starting Ceph mds.c for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:17:09.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.050+0000 7ffbe29ff700 1 -- 172.21.15.69:0/724621172 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbdc10b560 con 0x7ffbdc1017b0 2023-07-06T20:17:09.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.050+0000 7ffbd97fa700 1 -- 172.21.15.69:0/724621172 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5042 (secure 0 0 0) 0x7ffbd40992d0 con 0x7ffbdc1017b0 2023-07-06T20:17:09.053 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:09.053 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-06T20:17:09.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.053+0000 7ffbe29ff700 1 -- 172.21.15.69:0/724621172 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffbc40a11a0 msgr2=0x7ffbc40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:09.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.053+0000 7ffbe29ff700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffbc40a11a0 0x7ffbc40a3650 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc006010 tx=0x7ffbcc005960 comp rx=0 tx=0).stop 2023-07-06T20:17:09.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.053+0000 7ffbe29ff700 1 -- 172.21.15.69:0/724621172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbdc1017b0 msgr2=0x7ffbdc072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:09.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.053+0000 7ffbe29ff700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbdc1017b0 0x7ffbdc072f70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400efc0 tx=0x7ffbd400c3a0 comp rx=0 tx=0).stop 2023-07-06T20:17:09.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.053+0000 7ffbe29ff700 1 -- 172.21.15.69:0/724621172 shutdown_connections 2023-07-06T20:17:09.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.053+0000 7ffbe29ff700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffbc40a11a0 0x7ffbc40a3650 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:09.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.053+0000 7ffbe29ff700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbdc1017b0 0x7ffbdc072f70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:09.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.053+0000 7ffbe29ff700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbdc0ffc30 0x7ffbdc0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:09.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.053+0000 7ffbe29ff700 1 --2- 172.21.15.69:0/724621172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbdc100e20 0x7ffbdc072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:09.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.053+0000 7ffbe29ff700 1 -- 172.21.15.69:0/724621172 >> 172.21.15.69:0/724621172 conn(0x7ffbdc0fb210 msgr2=0x7ffbdc0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:09.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.053+0000 7ffbe29ff700 1 -- 172.21.15.69:0/724621172 shutdown_connections 2023-07-06T20:17:09.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:09.053+0000 7ffbe29ff700 1 -- 172.21.15.69:0/724621172 wait complete. 2023-07-06T20:17:09.057 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 7 2023-07-06T20:17:09.324 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:09 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/724621172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:09.325 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:17:09 smithi156 podman[154990]: 2023-07-06T20:17:09.325 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:17:09 smithi156 bash[154990]: 4c54792087d80dcbb1e8037c9e22e573fb6227170555c861946e82d4b0bef3e1 2023-07-06T20:17:09.325 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:17:09 smithi156 ceph-mds[155030]: -- 172.21.15.156:0/624025219 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e9cf0351e0 con 0x55e9cf00ac00 2023-07-06T20:17:09.325 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:17:09 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-c[155006]: starting mds.c at 2023-07-06T20:17:09.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:09 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/724621172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:09.606 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:17:09 smithi156 systemd[1]: Started Ceph mds.c for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:17:09.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:09 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/724621172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:10.614 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:10 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:10.614 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:10 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:10.614 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:10 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:10.614 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:10 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:10.615 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:10 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:10.615 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:10 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:10.615 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:10 smithi098 ceph-mon[128968]: Deploying daemon mds.h on smithi098 2023-07-06T20:17:10.615 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:10 smithi098 ceph-mon[128968]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:10.615 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:10 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.156:6836/595692705,v1:172.21.15.156:6837/595692705] up:boot 2023-07-06T20:17:10.615 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:10 smithi098 ceph-mon[128968]: fsmap 7 up:standby 2023-07-06T20:17:10.615 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:10 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-06T20:17:10.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:10.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:10.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:10.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:10.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:10.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:10.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:10 smithi069 ceph-mon[126399]: Deploying daemon mds.h on smithi098 2023-07-06T20:17:10.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:10 smithi069 ceph-mon[126399]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:10.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:10 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.156:6836/595692705,v1:172.21.15.156:6837/595692705] up:boot 2023-07-06T20:17:10.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:10 smithi069 ceph-mon[126399]: fsmap 7 up:standby 2023-07-06T20:17:10.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-06T20:17:10.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:10 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:10.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:10 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:10.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:10 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:10.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:10 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:10.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:10 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:10.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:10 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:10.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:10 smithi156 ceph-mon[129280]: Deploying daemon mds.h on smithi098 2023-07-06T20:17:10.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:10 smithi156 ceph-mon[129280]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:10.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:10 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.156:6836/595692705,v1:172.21.15.156:6837/595692705] up:boot 2023-07-06T20:17:10.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:10 smithi156 ceph-mon[129280]: fsmap 7 up:standby 2023-07-06T20:17:10.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:10 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-06T20:17:11.074 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:17:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.279+0000 7f8133bff700 1 -- 172.21.15.69:0/3358762650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f812c074680 msgr2=0x7f812c072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:11.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.279+0000 7f8133bff700 1 --2- 172.21.15.69:0/3358762650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f812c074680 0x7f812c072ad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8128009990 tx=0x7f8128009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.279+0000 7f8133bff700 1 -- 172.21.15.69:0/3358762650 shutdown_connections 2023-07-06T20:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.279+0000 7f8133bff700 1 --2- 172.21.15.69:0/3358762650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f812c10c200 0x7f812c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.279+0000 7f8133bff700 1 --2- 172.21.15.69:0/3358762650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f812c074680 0x7f812c072ad0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.279+0000 7f8133bff700 1 --2- 172.21.15.69:0/3358762650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c0730a0 0x7f812c073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.279+0000 7f8133bff700 1 -- 172.21.15.69:0/3358762650 >> 172.21.15.69:0/3358762650 conn(0x7f812c0fb210 msgr2=0x7f812c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.279+0000 7f8133bff700 1 -- 172.21.15.69:0/3358762650 shutdown_connections 2023-07-06T20:17:11.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.279+0000 7f8133bff700 1 -- 172.21.15.69:0/3358762650 wait complete. 2023-07-06T20:17:11.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.280+0000 7f8133bff700 1 Processor -- start 2023-07-06T20:17:11.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.281+0000 7f8133bff700 1 -- start start 2023-07-06T20:17:11.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.281+0000 7f8133bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f812c0730a0 0x7f812c19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.281+0000 7f8133bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f812c074680 0x7f812c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.281+0000 7f8133bff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c10c200 0x7f812c1a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.281+0000 7f8133bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c1a3af0 con 0x7f812c074680 2023-07-06T20:17:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.281+0000 7f8133bff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c1a3c30 con 0x7f812c10c200 2023-07-06T20:17:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.281+0000 7f8133bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c1a3dc0 con 0x7f812c0730a0 2023-07-06T20:17:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.281+0000 7f813119a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f812c074680 0x7f812c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.281+0000 7f813199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f812c0730a0 0x7f812c19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:11.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.281+0000 7f813219c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c10c200 0x7f812c1a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.281+0000 7f813199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f812c0730a0 0x7f812c19bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:41198/0 (socket says 172.21.15.69:41198) 2023-07-06T20:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.281+0000 7f813199b700 1 -- 172.21.15.69:0/2217508126 learned_addr learned my addr 172.21.15.69:0/2217508126 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f813199b700 1 -- 172.21.15.69:0/2217508126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c10c200 msgr2=0x7f812c1a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f813199b700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c10c200 0x7f812c1a3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f813199b700 1 -- 172.21.15.69:0/2217508126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f812c074680 msgr2=0x7f812c19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f813199b700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f812c074680 0x7f812c19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:11.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f813199b700 1 -- 172.21.15.69:0/2217508126 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81280095f0 con 0x7f812c0730a0 2023-07-06T20:17:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f813119a700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f812c074680 0x7f812c19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f813219c700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c10c200 0x7f812c1a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f813199b700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f812c0730a0 0x7f812c19bf70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8128005720 tx=0x7f8128009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f8122ffd700 1 -- 172.21.15.69:0/2217508126 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812801d070 con 0x7f812c0730a0 2023-07-06T20:17:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f8122ffd700 1 -- 172.21.15.69:0/2217508126 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8128021410 con 0x7f812c0730a0 2023-07-06T20:17:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f8133bff700 1 -- 172.21.15.69:0/2217508126 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c1a4040 con 0x7f812c0730a0 2023-07-06T20:17:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f8122ffd700 1 -- 172.21.15.69:0/2217508126 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128017670 con 0x7f812c0730a0 2023-07-06T20:17:11.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.282+0000 7f8133bff700 1 -- 172.21.15.69:0/2217508126 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c1a4460 con 0x7f812c0730a0 2023-07-06T20:17:11.288 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.283+0000 7f8133bff700 1 -- 172.21.15.69:0/2217508126 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812c101d50 con 0x7f812c0730a0 2023-07-06T20:17:11.288 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.285+0000 7f8122ffd700 1 -- 172.21.15.69:0/2217508126 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8128021580 con 0x7f812c0730a0 2023-07-06T20:17:11.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.287+0000 7f8122ffd700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f81180a11a0 0x7f81180a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:11.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.287+0000 7f8122ffd700 1 -- 172.21.15.69:0/2217508126 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f81280cd8b0 con 0x7f812c0730a0 2023-07-06T20:17:11.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.287+0000 7f813119a700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f81180a11a0 0x7f81180a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:11.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.287+0000 7f813119a700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f81180a11a0 0x7f81180a3650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f812c0743a0 tx=0x7f811c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:11.291 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.289+0000 7f8122ffd700 1 -- 172.21.15.69:0/2217508126 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8128097490 con 0x7f812c0730a0 2023-07-06T20:17:11.362 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:17:10 smithi098 systemd[1]: Starting Ceph mds.h for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:17:11.538 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.536+0000 7f8133bff700 1 -- 172.21.15.69:0/2217508126 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f812c02cd30 con 0x7f812c0730a0 2023-07-06T20:17:11.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.537+0000 7f8122ffd700 1 -- 172.21.15.69:0/2217508126 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5799 (secure 0 0 0) 0x7f812809aab0 con 0x7f812c0730a0 2023-07-06T20:17:11.539 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:11.539 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-06T20:17:11.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.539+0000 7f8133bff700 1 -- 172.21.15.69:0/2217508126 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f81180a11a0 msgr2=0x7f81180a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:11.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.539+0000 7f8133bff700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f81180a11a0 0x7f81180a3650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f812c0743a0 tx=0x7f811c009260 comp rx=0 tx=0).stop 2023-07-06T20:17:11.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.539+0000 7f8133bff700 1 -- 172.21.15.69:0/2217508126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f812c0730a0 msgr2=0x7f812c19bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:11.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.539+0000 7f8133bff700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f812c0730a0 0x7f812c19bf70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8128005720 tx=0x7f8128009580 comp rx=0 tx=0).stop 2023-07-06T20:17:11.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.539+0000 7f8133bff700 1 -- 172.21.15.69:0/2217508126 shutdown_connections 2023-07-06T20:17:11.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.539+0000 7f8133bff700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f81180a11a0 0x7f81180a3650 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:11.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.539+0000 7f8133bff700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f812c10c200 0x7f812c1a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:11.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.539+0000 7f8133bff700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f812c0730a0 0x7f812c19bf70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:11.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.539+0000 7f8133bff700 1 --2- 172.21.15.69:0/2217508126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f812c074680 0x7f812c19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:11.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.539+0000 7f8133bff700 1 -- 172.21.15.69:0/2217508126 >> 172.21.15.69:0/2217508126 conn(0x7f812c0fb210 msgr2=0x7f812c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:11.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.540+0000 7f8133bff700 1 -- 172.21.15.69:0/2217508126 shutdown_connections 2023-07-06T20:17:11.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:11.540+0000 7f8133bff700 1 -- 172.21.15.69:0/2217508126 wait complete. 2023-07-06T20:17:11.544 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 8 2023-07-06T20:17:11.625 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:17:11 smithi098 podman[154041]: 2023-07-06T20:17:11.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:11 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2217508126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:11.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:11 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2217508126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:12.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:11 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2217508126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:12.125 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:17:11 smithi098 bash[154041]: e3ba5aef4eb5acfe129410742011b59fb7980220ab7a9f8a8f3c030a766bc4e9 2023-07-06T20:17:12.125 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:17:11 smithi098 ceph-mds[154081]: -- 172.21.15.98:0/1048481501 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aa3b4ad1e0 con 0x55aa3b482800 2023-07-06T20:17:12.126 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:17:11 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-h[154057]: starting mds.h at 2023-07-06T20:17:12.126 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:17:11 smithi098 systemd[1]: Started Ceph mds.h for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:17:12.977 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:12 smithi156 ceph-mon[129280]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:12.977 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:12.977 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:12.977 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:12.977 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:12.977 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:12.978 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:12.978 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:12 smithi156 ceph-mon[129280]: Deploying daemon mds.i on smithi156 2023-07-06T20:17:12.978 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:12 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] up:boot 2023-07-06T20:17:12.978 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:12 smithi156 ceph-mon[129280]: fsmap 8 up:standby 2023-07-06T20:17:12.978 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-06T20:17:13.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:12 smithi069 ceph-mon[126399]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:13.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:13.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:13.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:13.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:13.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:13.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:13.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:12 smithi069 ceph-mon[126399]: Deploying daemon mds.i on smithi156 2023-07-06T20:17:13.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:12 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] up:boot 2023-07-06T20:17:13.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:12 smithi069 ceph-mon[126399]: fsmap 8 up:standby 2023-07-06T20:17:13.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-06T20:17:13.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:12 smithi098 ceph-mon[128968]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:13.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:13.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:13.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:13.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:13.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:13.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:13.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:12 smithi098 ceph-mon[128968]: Deploying daemon mds.i on smithi156 2023-07-06T20:17:13.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:12 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] up:boot 2023-07-06T20:17:13.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:12 smithi098 ceph-mon[128968]: fsmap 8 up:standby 2023-07-06T20:17:13.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-06T20:17:13.357 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:17:13 smithi156 systemd[1]: Starting Ceph mds.i for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:17:13.561 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:17:13.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.767+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2090941640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cac100e20 msgr2=0x7f7cac101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:13.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.767+0000 7f7cb13ff700 1 --2- 172.21.15.69:0/2090941640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cac100e20 0x7f7cac101270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009990 tx=0x7f7ca0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:13.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.767+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2090941640 shutdown_connections 2023-07-06T20:17:13.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.767+0000 7f7cb13ff700 1 --2- 172.21.15.69:0/2090941640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cac1017b0 0x7f7cac108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:13.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.767+0000 7f7cb13ff700 1 --2- 172.21.15.69:0/2090941640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cac0ffc30 0x7f7cac100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:13.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.767+0000 7f7cb13ff700 1 --2- 172.21.15.69:0/2090941640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cac100e20 0x7f7cac101270 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:13.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.767+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2090941640 >> 172.21.15.69:0/2090941640 conn(0x7f7cac0fb210 msgr2=0x7f7cac0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:13.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.767+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2090941640 shutdown_connections 2023-07-06T20:17:13.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.767+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2090941640 wait complete. 2023-07-06T20:17:13.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.768+0000 7f7cb13ff700 1 Processor -- start 2023-07-06T20:17:13.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.768+0000 7f7cb13ff700 1 -- start start 2023-07-06T20:17:13.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.768+0000 7f7cb13ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cac0ffc30 0x7f7cac0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:13.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7cb13ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cac100e20 0x7f7cac072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:13.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7cb13ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cac1017b0 0x7f7cac072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:13.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7cb13ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac073710 con 0x7f7cac0ffc30 2023-07-06T20:17:13.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7cb13ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac073880 con 0x7f7cac100e20 2023-07-06T20:17:13.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7cb13ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac19f990 con 0x7f7cac1017b0 2023-07-06T20:17:13.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7caaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cac0ffc30 0x7f7cac0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:13.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7caaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cac0ffc30 0x7f7cac0743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37888/0 (socket says 172.21.15.69:37888) 2023-07-06T20:17:13.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7caaffd700 1 -- 172.21.15.69:0/2826937192 learned_addr learned my addr 172.21.15.69:0/2826937192 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:13.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7cab7fe700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cac1017b0 0x7f7cac072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:13.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7caa7fc700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cac100e20 0x7f7cac072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:13.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7caaffd700 1 -- 172.21.15.69:0/2826937192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cac1017b0 msgr2=0x7f7cac072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:13.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7caaffd700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cac1017b0 0x7f7cac072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:13.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7caaffd700 1 -- 172.21.15.69:0/2826937192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cac100e20 msgr2=0x7f7cac072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:13.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7caaffd700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cac100e20 0x7f7cac072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:13.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7caaffd700 1 -- 172.21.15.69:0/2826937192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca00095f0 con 0x7f7cac0ffc30 2023-07-06T20:17:13.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7caa7fc700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cac100e20 0x7f7cac072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:17:13.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7caaffd700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cac0ffc30 0x7f7cac0743f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e970 tx=0x7f7c9c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:13.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.769+0000 7f7c93fff700 1 -- 172.21.15.69:0/2826937192 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c009790 con 0x7f7cac0ffc30 2023-07-06T20:17:13.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.770+0000 7f7c93fff700 1 -- 172.21.15.69:0/2826937192 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c9c00eec0 con 0x7f7cac0ffc30 2023-07-06T20:17:13.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.770+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2826937192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cac19fc70 con 0x7f7cac0ffc30 2023-07-06T20:17:13.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.770+0000 7f7c93fff700 1 -- 172.21.15.69:0/2826937192 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c003cf0 con 0x7f7cac0ffc30 2023-07-06T20:17:13.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.770+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2826937192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cac1a01d0 con 0x7f7cac0ffc30 2023-07-06T20:17:13.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.771+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2826937192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cac10b450 con 0x7f7cac0ffc30 2023-07-06T20:17:13.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.773+0000 7f7c93fff700 1 -- 172.21.15.69:0/2826937192 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7c9c00ccd0 con 0x7f7cac0ffc30 2023-07-06T20:17:13.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.774+0000 7f7c93fff700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7c940a1220 0x7f7c940a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:13.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.774+0000 7f7c93fff700 1 -- 172.21.15.69:0/2826937192 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7c9c014070 con 0x7f7cac0ffc30 2023-07-06T20:17:13.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.774+0000 7f7caa7fc700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7c940a1220 0x7f7c940a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:13.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.775+0000 7f7caa7fc700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7c940a1220 0x7f7c940a36d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0000c00 tx=0x7f7ca0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:13.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:13.778+0000 7f7c93fff700 1 -- 172.21.15.69:0/2826937192 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7c9c096210 con 0x7f7cac0ffc30 2023-07-06T20:17:13.804 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:17:13 smithi156 podman[155328]: 2023-07-06T20:17:13.804 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:17:13 smithi156 ceph-mds[155369]: -- 172.21.15.156:0/3320731020 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b51f5b31e0 con 0x55b51f588800 2023-07-06T20:17:13.804 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:17:13 smithi156 bash[155328]: 760cf5fd834a563e3f2ccaedd75dae3bcfcd87a00b60792d8d0eb2080f5569d9 2023-07-06T20:17:13.805 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:17:13 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-i[155344]: starting mds.i at 2023-07-06T20:17:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.031+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2826937192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cac19fe00 con 0x7f7cac0ffc30 2023-07-06T20:17:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.032+0000 7f7c93fff700 1 -- 172.21.15.69:0/2826937192 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6556 (secure 0 0 0) 0x7f7cac19fe00 con 0x7f7cac0ffc30 2023-07-06T20:17:14.034 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:14.034 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-06T20:17:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.034+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2826937192 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7c940a1220 msgr2=0x7f7c940a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.034+0000 7f7cb13ff700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7c940a1220 0x7f7c940a36d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0000c00 tx=0x7f7ca0005fb0 comp rx=0 tx=0).stop 2023-07-06T20:17:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.034+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2826937192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cac0ffc30 msgr2=0x7f7cac0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.034+0000 7f7cb13ff700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cac0ffc30 0x7f7cac0743f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e970 tx=0x7f7c9c00ec80 comp rx=0 tx=0).stop 2023-07-06T20:17:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.034+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2826937192 shutdown_connections 2023-07-06T20:17:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.034+0000 7f7cb13ff700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7c940a1220 0x7f7c940a36d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.034+0000 7f7cb13ff700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cac1017b0 0x7f7cac072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.034+0000 7f7cb13ff700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cac0ffc30 0x7f7cac0743f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.034+0000 7f7cb13ff700 1 --2- 172.21.15.69:0/2826937192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cac100e20 0x7f7cac072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.034+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2826937192 >> 172.21.15.69:0/2826937192 conn(0x7f7cac0fb210 msgr2=0x7f7cac0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.034+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2826937192 shutdown_connections 2023-07-06T20:17:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:14.034+0000 7f7cb13ff700 1 -- 172.21.15.69:0/2826937192 wait complete. 2023-07-06T20:17:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 9 2023-07-06T20:17:14.106 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:17:13 smithi156 systemd[1]: Started Ceph mds.i for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:17:15.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:14 smithi156 ceph-mon[129280]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:15.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:14 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:15.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:14 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:15.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:14 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:15.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:14 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:15.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:14 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:15.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:14 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:15.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:14 smithi156 ceph-mon[129280]: Deploying daemon mds.g on smithi069 2023-07-06T20:17:15.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:14 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2826937192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:15.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:14 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.156:6838/4200610083,v1:172.21.15.156:6839/4200610083] up:boot 2023-07-06T20:17:15.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:14 smithi156 ceph-mon[129280]: fsmap 9 up:standby 2023-07-06T20:17:15.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:14 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-06T20:17:15.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:14 smithi069 ceph-mon[126399]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:15.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:15.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:15.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:15.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:15.122 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:15.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:15.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:14 smithi069 ceph-mon[126399]: Deploying daemon mds.g on smithi069 2023-07-06T20:17:15.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:14 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2826937192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:15.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:14 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.156:6838/4200610083,v1:172.21.15.156:6839/4200610083] up:boot 2023-07-06T20:17:15.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:14 smithi069 ceph-mon[126399]: fsmap 9 up:standby 2023-07-06T20:17:15.123 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:14 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-06T20:17:15.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:14 smithi098 ceph-mon[128968]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:15.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:14 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:15.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:14 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:15.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:14 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:15.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:14 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:15.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:14 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:15.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:14 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:15.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:14 smithi098 ceph-mon[128968]: Deploying daemon mds.g on smithi069 2023-07-06T20:17:15.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:14 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2826937192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:15.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:14 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.156:6838/4200610083,v1:172.21.15.156:6839/4200610083] up:boot 2023-07-06T20:17:15.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:14 smithi098 ceph-mon[128968]: fsmap 9 up:standby 2023-07-06T20:17:15.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:14 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-06T20:17:16.057 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:17:16.114 INFO:journalctl@ceph.mds.g.smithi069.stdout:Jul 06 20:17:16 smithi069 systemd[1]: Starting Ceph mds.g for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:17:16.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.273+0000 7f4deedff700 1 -- 172.21.15.69:0/2618806876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8074680 msgr2=0x7f4de8072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:16.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.273+0000 7f4deedff700 1 --2- 172.21.15.69:0/2618806876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8074680 0x7f4de8072ad0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009990 tx=0x7f4de4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:16.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.273+0000 7f4deedff700 1 -- 172.21.15.69:0/2618806876 shutdown_connections 2023-07-06T20:17:16.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.273+0000 7f4deedff700 1 --2- 172.21.15.69:0/2618806876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de810c200 0x7f4de810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:16.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.273+0000 7f4deedff700 1 --2- 172.21.15.69:0/2618806876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8074680 0x7f4de8072ad0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:16.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.273+0000 7f4deedff700 1 --2- 172.21.15.69:0/2618806876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de80730a0 0x7f4de8073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:16.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.273+0000 7f4deedff700 1 -- 172.21.15.69:0/2618806876 >> 172.21.15.69:0/2618806876 conn(0x7f4de80fb210 msgr2=0x7f4de80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:16.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.273+0000 7f4deedff700 1 -- 172.21.15.69:0/2618806876 shutdown_connections 2023-07-06T20:17:16.277 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.273+0000 7f4deedff700 1 -- 172.21.15.69:0/2618806876 wait complete. 2023-07-06T20:17:16.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.274+0000 7f4deedff700 1 Processor -- start 2023-07-06T20:17:16.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.274+0000 7f4deedff700 1 -- start start 2023-07-06T20:17:16.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4deedff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de80730a0 0x7f4de819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:16.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4deedff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8074680 0x7f4de819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:16.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4deedff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de810c200 0x7f4de81a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:16.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4deedff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de81a3b70 con 0x7f4de8074680 2023-07-06T20:17:16.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4deedff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de81a3cb0 con 0x7f4de810c200 2023-07-06T20:17:16.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4deedff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de81a3e40 con 0x7f4de80730a0 2023-07-06T20:17:16.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4ddffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8074680 0x7f4de819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:16.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4ddffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8074680 0x7f4de819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37892/0 (socket says 172.21.15.69:37892) 2023-07-06T20:17:16.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4ddffff700 1 -- 172.21.15.69:0/3855458547 learned_addr learned my addr 172.21.15.69:0/3855458547 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:16.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4ded39c700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de810c200 0x7f4de81a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:16.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4decb9b700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de80730a0 0x7f4de819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:16.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4ddffff700 1 -- 172.21.15.69:0/3855458547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de80730a0 msgr2=0x7f4de819bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:16.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4ddffff700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de80730a0 0x7f4de819bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:16.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4ddffff700 1 -- 172.21.15.69:0/3855458547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de810c200 msgr2=0x7f4de81a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:16.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4ddffff700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de810c200 0x7f4de81a35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:16.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.275+0000 7f4ddffff700 1 -- 172.21.15.69:0/3855458547 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de40095f0 con 0x7f4de8074680 2023-07-06T20:17:16.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.276+0000 7f4ddffff700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8074680 0x7f4de819c530 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800e970 tx=0x7f4dd800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:16.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.276+0000 7f4dddffb700 1 -- 172.21.15.69:0/3855458547 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8009790 con 0x7f4de8074680 2023-07-06T20:17:16.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.276+0000 7f4dddffb700 1 -- 172.21.15.69:0/3855458547 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dd800c7e0 con 0x7f4de8074680 2023-07-06T20:17:16.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.276+0000 7f4dddffb700 1 -- 172.21.15.69:0/3855458547 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8003c70 con 0x7f4de8074680 2023-07-06T20:17:16.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.276+0000 7f4deedff700 1 -- 172.21.15.69:0/3855458547 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de81a4120 con 0x7f4de8074680 2023-07-06T20:17:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.276+0000 7f4deedff700 1 -- 172.21.15.69:0/3855458547 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de81a45e0 con 0x7f4de8074680 2023-07-06T20:17:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.277+0000 7f4deedff700 1 -- 172.21.15.69:0/3855458547 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de8101d50 con 0x7f4de8074680 2023-07-06T20:17:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.279+0000 7f4dddffb700 1 -- 172.21.15.69:0/3855458547 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4dd800cc50 con 0x7f4de8074680 2023-07-06T20:17:16.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.280+0000 7f4dddffb700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4dd00a1220 0x7f4dd00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:16.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.280+0000 7f4dddffb700 1 -- 172.21.15.69:0/3855458547 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4dd8014070 con 0x7f4de8074680 2023-07-06T20:17:16.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.281+0000 7f4decb9b700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4dd00a1220 0x7f4dd00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:16.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.281+0000 7f4decb9b700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4dd00a1220 0x7f4dd00a36d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4de400b2e0 tx=0x7f4de4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.284+0000 7f4dddffb700 1 -- 172.21.15.69:0/3855458547 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4dd8096170 con 0x7f4de8074680 2023-07-06T20:17:16.567 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.565+0000 7f4deedff700 1 -- 172.21.15.69:0/3855458547 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4de802cd20 con 0x7f4de8074680 2023-07-06T20:17:16.567 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.565+0000 7f4dddffb700 1 -- 172.21.15.69:0/3855458547 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7318 (secure 0 0 0) 0x7f4dd8003dd0 con 0x7f4de8074680 2023-07-06T20:17:16.568 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:16.568 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-06T20:17:16.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.568+0000 7f4deedff700 1 -- 172.21.15.69:0/3855458547 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4dd00a1220 msgr2=0x7f4dd00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:16.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.568+0000 7f4deedff700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4dd00a1220 0x7f4dd00a36d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4de400b2e0 tx=0x7f4de4005fb0 comp rx=0 tx=0).stop 2023-07-06T20:17:16.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.568+0000 7f4deedff700 1 -- 172.21.15.69:0/3855458547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8074680 msgr2=0x7f4de819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:16.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.568+0000 7f4deedff700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8074680 0x7f4de819c530 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800e970 tx=0x7f4dd800ec80 comp rx=0 tx=0).stop 2023-07-06T20:17:16.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.568+0000 7f4deedff700 1 -- 172.21.15.69:0/3855458547 shutdown_connections 2023-07-06T20:17:16.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.568+0000 7f4deedff700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4dd00a1220 0x7f4dd00a36d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:16.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.568+0000 7f4deedff700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4de810c200 0x7f4de81a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:16.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.568+0000 7f4deedff700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de80730a0 0x7f4de819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:16.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.568+0000 7f4deedff700 1 --2- 172.21.15.69:0/3855458547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8074680 0x7f4de819c530 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:16.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.568+0000 7f4deedff700 1 -- 172.21.15.69:0/3855458547 >> 172.21.15.69:0/3855458547 conn(0x7f4de80fb210 msgr2=0x7f4de80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:16.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.568+0000 7f4deedff700 1 -- 172.21.15.69:0/3855458547 shutdown_connections 2023-07-06T20:17:16.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:16.568+0000 7f4deedff700 1 -- 172.21.15.69:0/3855458547 wait complete. 2023-07-06T20:17:16.572 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 10 2023-07-06T20:17:16.845 INFO:journalctl@ceph.mds.g.smithi069.stdout:Jul 06 20:17:16 smithi069 podman[161032]: 2023-07-06T20:17:17.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:16 smithi156 ceph-mon[129280]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:17.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:16 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3855458547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:17.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:16 smithi098 ceph-mon[128968]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:17.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:16 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3855458547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:17.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:16 smithi069 ceph-mon[126399]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:17.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:16 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3855458547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:17.259 INFO:journalctl@ceph.mds.g.smithi069.stdout:Jul 06 20:17:16 smithi069 bash[161032]: 137d376ad724484c7627383ead803af19e28c35d584e9d8f8574fe955a702d2b 2023-07-06T20:17:17.259 INFO:journalctl@ceph.mds.g.smithi069.stdout:Jul 06 20:17:16 smithi069 ceph-mds[161075]: -- 172.21.15.69:0/3023538775 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564c898e31e0 con 0x564c898b9000 2023-07-06T20:17:17.259 INFO:journalctl@ceph.mds.g.smithi069.stdout:Jul 06 20:17:16 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-g[161050]: starting mds.g at 2023-07-06T20:17:17.260 INFO:journalctl@ceph.mds.g.smithi069.stdout:Jul 06 20:17:16 smithi069 systemd[1]: Started Ceph mds.g for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:17:18.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:17 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:18.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:17 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:18.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:17 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:18.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:17 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:18.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:17 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:18.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:17 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:18.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:17 smithi069 ceph-mon[126399]: Deploying daemon mds.a on smithi069 2023-07-06T20:17:18.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:17 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.69:6838/1973475728,v1:172.21.15.69:6839/1973475728] up:boot 2023-07-06T20:17:18.224 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:17 smithi069 ceph-mon[126399]: fsmap 10 up:standby 2023-07-06T20:17:18.224 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:17 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-06T20:17:18.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:17 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:18.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:17 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:18.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:17 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:18.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:17 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:18.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:17 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:18.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:17 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:18.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:17 smithi156 ceph-mon[129280]: Deploying daemon mds.a on smithi069 2023-07-06T20:17:18.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:17 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.69:6838/1973475728,v1:172.21.15.69:6839/1973475728] up:boot 2023-07-06T20:17:18.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:17 smithi156 ceph-mon[129280]: fsmap 10 up:standby 2023-07-06T20:17:18.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:17 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-06T20:17:18.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:17 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:18.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:17 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:18.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:17 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:18.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:17 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:18.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:17 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:18.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:17 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:18.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:17 smithi098 ceph-mon[128968]: Deploying daemon mds.a on smithi069 2023-07-06T20:17:18.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:17 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.69:6838/1973475728,v1:172.21.15.69:6839/1973475728] up:boot 2023-07-06T20:17:18.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:17 smithi098 ceph-mon[128968]: fsmap 10 up:standby 2023-07-06T20:17:18.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:17 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-06T20:17:18.590 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:17:18.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.823+0000 7f998e1ff700 1 -- 172.21.15.69:0/1185404773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988107af0 msgr2=0x7f9988109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:18.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.823+0000 7f998e1ff700 1 --2- 172.21.15.69:0/1185404773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988107af0 0x7f9988109f50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f998000b0a0 tx=0x7f998000b3b0 comp rx=0 tx=0).stop 2023-07-06T20:17:18.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.823+0000 7f998e1ff700 1 -- 172.21.15.69:0/1185404773 shutdown_connections 2023-07-06T20:17:18.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.823+0000 7f998e1ff700 1 --2- 172.21.15.69:0/1185404773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988107af0 0x7f9988109f50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:18.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.823+0000 7f998e1ff700 1 --2- 172.21.15.69:0/1185404773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9988074600 0x7f9988072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:18.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.823+0000 7f998e1ff700 1 --2- 172.21.15.69:0/1185404773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9988073020 0x7f9988073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:18.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.823+0000 7f998e1ff700 1 -- 172.21.15.69:0/1185404773 >> 172.21.15.69:0/1185404773 conn(0x7f99880fb240 msgr2=0x7f99880fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:18.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.823+0000 7f998e1ff700 1 -- 172.21.15.69:0/1185404773 shutdown_connections 2023-07-06T20:17:18.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.823+0000 7f998e1ff700 1 -- 172.21.15.69:0/1185404773 wait complete. 2023-07-06T20:17:18.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.824+0000 7f998e1ff700 1 Processor -- start 2023-07-06T20:17:18.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f998e1ff700 1 -- start start 2023-07-06T20:17:18.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f998e1ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9988073020 0x7f9988197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:18.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f998e1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988074600 0x7f9988198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:18.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f998e1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9988107af0 0x7f998819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:18.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f998e1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998819f970 con 0x7f9988074600 2023-07-06T20:17:18.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f998e1ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998819fab0 con 0x7f9988073020 2023-07-06T20:17:18.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f998e1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998819fc40 con 0x7f9988107af0 2023-07-06T20:17:18.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f9986ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988074600 0x7f9988198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:18.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f9986ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988074600 0x7f9988198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37918/0 (socket says 172.21.15.69:37918) 2023-07-06T20:17:18.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f9986ffd700 1 -- 172.21.15.69:0/311294879 learned_addr learned my addr 172.21.15.69:0/311294879 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:18.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f99877fe700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9988073020 0x7f9988197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:18.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f9986ffd700 1 -- 172.21.15.69:0/311294879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9988107af0 msgr2=0x7f998819f3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T20:17:18.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f9986ffd700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9988107af0 0x7f998819f3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:18.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f9986ffd700 1 -- 172.21.15.69:0/311294879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9988073020 msgr2=0x7f9988197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:18.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.825+0000 7f9986ffd700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9988073020 0x7f9988197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:18.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.826+0000 7f9986ffd700 1 -- 172.21.15.69:0/311294879 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9980009d00 con 0x7f9988074600 2023-07-06T20:17:18.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.826+0000 7f9986ffd700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988074600 0x7f9988198330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f997c00e9a0 tx=0x7f997c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:18.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.826+0000 7f9984ff9700 1 -- 172.21.15.69:0/311294879 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c009790 con 0x7f9988074600 2023-07-06T20:17:18.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.826+0000 7f9984ff9700 1 -- 172.21.15.69:0/311294879 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f997c00cbd0 con 0x7f9988074600 2023-07-06T20:17:18.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.826+0000 7f998e1ff700 1 -- 172.21.15.69:0/311294879 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998819ff20 con 0x7f9988074600 2023-07-06T20:17:18.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.826+0000 7f9984ff9700 1 -- 172.21.15.69:0/311294879 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c018680 con 0x7f9988074600 2023-07-06T20:17:18.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.826+0000 7f998e1ff700 1 -- 172.21.15.69:0/311294879 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99881a03f0 con 0x7f9988074600 2023-07-06T20:17:18.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.827+0000 7f998e1ff700 1 -- 172.21.15.69:0/311294879 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9988106230 con 0x7f9988074600 2023-07-06T20:17:18.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.829+0000 7f9984ff9700 1 -- 172.21.15.69:0/311294879 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f997c00cd40 con 0x7f9988074600 2023-07-06T20:17:18.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.831+0000 7f9984ff9700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f99700a1220 0x7f99700a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:18.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.831+0000 7f9984ff9700 1 -- 172.21.15.69:0/311294879 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f997c014070 con 0x7f9988074600 2023-07-06T20:17:18.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.831+0000 7f99877fe700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f99700a1220 0x7f99700a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:18.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.832+0000 7f99877fe700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f99700a1220 0x7f99700a36d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9978009c60 tx=0x7f9978009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:18.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:18.835+0000 7f9984ff9700 1 -- 172.21.15.69:0/311294879 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f997c051770 con 0x7f9988074600 2023-07-06T20:17:19.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.105+0000 7f998e1ff700 1 -- 172.21.15.69:0/311294879 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f998802cce0 con 0x7f9988074600 2023-07-06T20:17:19.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.106+0000 7f9984ff9700 1 -- 172.21.15.69:0/311294879 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8076 (secure 0 0 0) 0x7f997c096280 con 0x7f9988074600 2023-07-06T20:17:19.113 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:19.113 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-06T20:17:19.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.111+0000 7f998e1ff700 1 -- 172.21.15.69:0/311294879 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f99700a1220 msgr2=0x7f99700a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:19.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.111+0000 7f998e1ff700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f99700a1220 0x7f99700a36d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9978009c60 tx=0x7f9978009260 comp rx=0 tx=0).stop 2023-07-06T20:17:19.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.111+0000 7f998e1ff700 1 -- 172.21.15.69:0/311294879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988074600 msgr2=0x7f9988198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:19.115 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.111+0000 7f998e1ff700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988074600 0x7f9988198330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f997c00e9a0 tx=0x7f997c00ecb0 comp rx=0 tx=0).stop 2023-07-06T20:17:19.115 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.111+0000 7f998e1ff700 1 -- 172.21.15.69:0/311294879 shutdown_connections 2023-07-06T20:17:19.115 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.111+0000 7f998e1ff700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f99700a1220 0x7f99700a36d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:19.115 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.111+0000 7f998e1ff700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9988107af0 0x7f998819f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:19.115 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.111+0000 7f998e1ff700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9988073020 0x7f9988197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:19.115 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.111+0000 7f998e1ff700 1 --2- 172.21.15.69:0/311294879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9988074600 0x7f9988198330 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:19.115 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.111+0000 7f998e1ff700 1 -- 172.21.15.69:0/311294879 >> 172.21.15.69:0/311294879 conn(0x7f99880fb240 msgr2=0x7f99880fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:19.115 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.111+0000 7f998e1ff700 1 -- 172.21.15.69:0/311294879 shutdown_connections 2023-07-06T20:17:19.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:19.111+0000 7f998e1ff700 1 -- 172.21.15.69:0/311294879 wait complete. 2023-07-06T20:17:19.116 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 11 2023-07-06T20:17:19.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:18 smithi069 ceph-mon[126399]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:19.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:18 smithi156 ceph-mon[129280]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:19.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:18 smithi098 ceph-mon[128968]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:19.759 INFO:journalctl@ceph.mds.a.smithi069.stdout:Jul 06 20:17:19 smithi069 systemd[1]: Starting Ceph mds.a for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:17:20.044 INFO:journalctl@ceph.mds.a.smithi069.stdout:Jul 06 20:17:19 smithi069 podman[161422]: 2023-07-06T20:17:20.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:19 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/311294879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:20.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:19 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/311294879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:20.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:19 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/311294879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:20.509 INFO:journalctl@ceph.mds.a.smithi069.stdout:Jul 06 20:17:20 smithi069 bash[161422]: 43fbceeea5e93cf7da62d00888361297e19e53126f98895e45a610f74d242f44 2023-07-06T20:17:20.509 INFO:journalctl@ceph.mds.a.smithi069.stdout:Jul 06 20:17:20 smithi069 ceph-mds[161462]: -- 172.21.15.69:0/3055164298 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f83c28f1e0 con 0x55f83c264800 2023-07-06T20:17:20.509 INFO:journalctl@ceph.mds.a.smithi069.stdout:Jul 06 20:17:20 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-a[161438]: starting mds.a at 2023-07-06T20:17:20.509 INFO:journalctl@ceph.mds.a.smithi069.stdout:Jul 06 20:17:20 smithi069 systemd[1]: Started Ceph mds.a for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:17:21.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:20 smithi098 ceph-mon[128968]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:21.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:21.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:21.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:21.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:21.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:21.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:21.142 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:17:21.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:20 smithi069 ceph-mon[126399]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:21.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:21.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:21.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:21.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:21.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:21.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:21.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.344+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3697047921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c41017b0 msgr2=0x7f80c4108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:21.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.344+0000 7f80ca7ff700 1 --2- 172.21.15.69:0/3697047921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c41017b0 0x7f80c4108070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00b0a0 tx=0x7f80bc00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:17:21.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.344+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3697047921 shutdown_connections 2023-07-06T20:17:21.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.344+0000 7f80ca7ff700 1 --2- 172.21.15.69:0/3697047921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c41017b0 0x7f80c4108070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:21.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.344+0000 7f80ca7ff700 1 --2- 172.21.15.69:0/3697047921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80c40ffc30 0x7f80c4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:21.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.344+0000 7f80ca7ff700 1 --2- 172.21.15.69:0/3697047921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80c4100e20 0x7f80c4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:21.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.344+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3697047921 >> 172.21.15.69:0/3697047921 conn(0x7f80c40fb210 msgr2=0x7f80c40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:21.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.344+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3697047921 shutdown_connections 2023-07-06T20:17:21.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.344+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3697047921 wait complete. 2023-07-06T20:17:21.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.345+0000 7f80ca7ff700 1 Processor -- start 2023-07-06T20:17:21.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.345+0000 7f80ca7ff700 1 -- start start 2023-07-06T20:17:21.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80ca7ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80c40ffc30 0x7f80c4197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:21.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80ca7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4100e20 0x7f80c41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:21.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80ca7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80c41017b0 0x7f80c419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:21.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80ca7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c419f7a0 con 0x7f80c4100e20 2023-07-06T20:17:21.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80ca7ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c419f910 con 0x7f80c40ffc30 2023-07-06T20:17:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80ca7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c419fad0 con 0x7f80c41017b0 2023-07-06T20:17:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80c37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4100e20 0x7f80c41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80c3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80c40ffc30 0x7f80c4197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80c37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4100e20 0x7f80c41981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37956/0 (socket says 172.21.15.69:37956) 2023-07-06T20:17:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80c37fe700 1 -- 172.21.15.69:0/3114750838 learned_addr learned my addr 172.21.15.69:0/3114750838 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80c8d9c700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80c41017b0 0x7f80c419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80c3fff700 1 -- 172.21.15.69:0/3114750838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80c41017b0 msgr2=0x7f80c419f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:21.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80c3fff700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80c41017b0 0x7f80c419f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80c3fff700 1 -- 172.21.15.69:0/3114750838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4100e20 msgr2=0x7f80c41981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80c3fff700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4100e20 0x7f80c41981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80c3fff700 1 -- 172.21.15.69:0/3114750838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc009d00 con 0x7f80c40ffc30 2023-07-06T20:17:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.346+0000 7f80c37fe700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4100e20 0x7f80c41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.347+0000 7f80c3fff700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80c40ffc30 0x7f80c4197c60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f80b400e9c0 tx=0x7f80b400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.347+0000 7f80c17fa700 1 -- 172.21.15.69:0/3114750838 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4004220 con 0x7f80c40ffc30 2023-07-06T20:17:21.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.347+0000 7f80c17fa700 1 -- 172.21.15.69:0/3114750838 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b4004380 con 0x7f80c40ffc30 2023-07-06T20:17:21.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.347+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3114750838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c419fdb0 con 0x7f80c40ffc30 2023-07-06T20:17:21.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.347+0000 7f80c17fa700 1 -- 172.21.15.69:0/3114750838 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4003be0 con 0x7f80c40ffc30 2023-07-06T20:17:21.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.347+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3114750838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c41a0200 con 0x7f80c40ffc30 2023-07-06T20:17:21.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.348+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3114750838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c410b460 con 0x7f80c40ffc30 2023-07-06T20:17:21.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.350+0000 7f80c17fa700 1 -- 172.21.15.69:0/3114750838 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f80b4003d40 con 0x7f80c40ffc30 2023-07-06T20:17:21.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.351+0000 7f80c17fa700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80ac0a11a0 0x7f80ac0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:21.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.352+0000 7f80c17fa700 1 -- 172.21.15.69:0/3114750838 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f80b4014070 con 0x7f80c40ffc30 2023-07-06T20:17:21.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.352+0000 7f80c37fe700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80ac0a11a0 0x7f80ac0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:21.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.352+0000 7f80c37fe700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80ac0a11a0 0x7f80ac0a3650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f80b8011550 tx=0x7f80b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:21.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.353+0000 7f80c17fa700 1 -- 172.21.15.69:0/3114750838 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f80b4096920 con 0x7f80c40ffc30 2023-07-06T20:17:21.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:20 smithi156 ceph-mon[129280]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:21.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:21.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:21.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:21.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-06T20:17:21.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-06T20:17:21.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:21.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.600+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3114750838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80c41a0a00 con 0x7f80c40ffc30 2023-07-06T20:17:21.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.602+0000 7f80c17fa700 1 -- 172.21.15.69:0/3114750838 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8834 (secure 0 0 0) 0x7f80b4099f40 con 0x7f80c40ffc30 2023-07-06T20:17:21.604 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:21.604 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-06T20:17:21.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.604+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3114750838 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80ac0a11a0 msgr2=0x7f80ac0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:21.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.604+0000 7f80ca7ff700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80ac0a11a0 0x7f80ac0a3650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f80b8011550 tx=0x7f80b8009260 comp rx=0 tx=0).stop 2023-07-06T20:17:21.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.604+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3114750838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80c40ffc30 msgr2=0x7f80c4197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:21.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.604+0000 7f80ca7ff700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80c40ffc30 0x7f80c4197c60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f80b400e9c0 tx=0x7f80b400ecd0 comp rx=0 tx=0).stop 2023-07-06T20:17:21.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.604+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3114750838 shutdown_connections 2023-07-06T20:17:21.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.604+0000 7f80ca7ff700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80ac0a11a0 0x7f80ac0a3650 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:21.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.604+0000 7f80ca7ff700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80c41017b0 0x7f80c419f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:21.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.604+0000 7f80ca7ff700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80c40ffc30 0x7f80c4197c60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:21.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.604+0000 7f80ca7ff700 1 --2- 172.21.15.69:0/3114750838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c4100e20 0x7f80c41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:21.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.604+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3114750838 >> 172.21.15.69:0/3114750838 conn(0x7f80c40fb210 msgr2=0x7f80c4106660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:21.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.604+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3114750838 shutdown_connections 2023-07-06T20:17:21.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:21.604+0000 7f80ca7ff700 1 -- 172.21.15.69:0/3114750838 wait complete. 2023-07-06T20:17:21.610 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 12 2023-07-06T20:17:21.875 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:17:21 smithi098 systemd[1]: Starting Ceph mds.b for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:17:22.246 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:21 smithi098 ceph-mon[128968]: Deploying daemon mds.b on smithi098 2023-07-06T20:17:22.247 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:21 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.69:6840/1464129965,v1:172.21.15.69:6841/1464129965] up:boot 2023-07-06T20:17:22.247 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:21 smithi098 ceph-mon[128968]: fsmap 11 up:standby 2023-07-06T20:17:22.247 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:21 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-06T20:17:22.247 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:21 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3114750838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:22.247 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:17:22 smithi098 podman[154382]: 2023-07-06T20:17:22.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:21 smithi069 ceph-mon[126399]: Deploying daemon mds.b on smithi098 2023-07-06T20:17:22.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:21 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.69:6840/1464129965,v1:172.21.15.69:6841/1464129965] up:boot 2023-07-06T20:17:22.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:21 smithi069 ceph-mon[126399]: fsmap 11 up:standby 2023-07-06T20:17:22.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:21 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-06T20:17:22.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:21 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3114750838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:22.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:21 smithi156 ceph-mon[129280]: Deploying daemon mds.b on smithi098 2023-07-06T20:17:22.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:21 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.69:6840/1464129965,v1:172.21.15.69:6841/1464129965] up:boot 2023-07-06T20:17:22.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:21 smithi156 ceph-mon[129280]: fsmap 11 up:standby 2023-07-06T20:17:22.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:21 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-06T20:17:22.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:21 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3114750838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:22.625 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:17:22 smithi098 ceph-mds[154423]: -- 172.21.15.98:0/400061751 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f196c631e0 con 0x55f196c39000 2023-07-06T20:17:22.626 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:17:22 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-b[154398]: starting mds.b at 2023-07-06T20:17:22.626 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:17:22 smithi098 bash[154382]: dca64888f178b8e992a29484b4adb244686c532c79351956ac2c639d2e4efa7c 2023-07-06T20:17:22.626 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:17:22 smithi098 systemd[1]: Started Ceph mds.b for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c. 2023-07-06T20:17:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:22 smithi069 ceph-mon[126399]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:22 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:17:23.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:22 smithi156 ceph-mon[129280]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:23.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:22 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:17:23.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:22 smithi098 ceph-mon[128968]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:23.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:23.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:22 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:17:23.625 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:17:23.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.830+0000 7f19991ff700 1 -- 172.21.15.69:0/3336804803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1994100690 msgr2=0x7f1994102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.830+0000 7f19991ff700 1 --2- 172.21.15.69:0/3336804803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1994100690 0x7f1994102a70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1984009990 tx=0x7f1984009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.830+0000 7f19991ff700 1 -- 172.21.15.69:0/3336804803 shutdown_connections 2023-07-06T20:17:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.830+0000 7f19991ff700 1 --2- 172.21.15.69:0/3336804803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19941058d0 0x7f1994107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.830+0000 7f19991ff700 1 --2- 172.21.15.69:0/3336804803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1994100690 0x7f1994102a70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.830+0000 7f19991ff700 1 --2- 172.21.15.69:0/3336804803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1994102fb0 0x7f1994105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:23.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.830+0000 7f19991ff700 1 -- 172.21.15.69:0/3336804803 >> 172.21.15.69:0/3336804803 conn(0x7f19940fa130 msgr2=0x7f19940fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:23.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.830+0000 7f19991ff700 1 -- 172.21.15.69:0/3336804803 shutdown_connections 2023-07-06T20:17:23.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.830+0000 7f19991ff700 1 -- 172.21.15.69:0/3336804803 wait complete. 2023-07-06T20:17:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.831+0000 7f19991ff700 1 Processor -- start 2023-07-06T20:17:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.832+0000 7f19991ff700 1 -- start start 2023-07-06T20:17:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.832+0000 7f19991ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1994100690 0x7f1994100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.832+0000 7f19991ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1994102fb0 0x7f19940fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.832+0000 7f19991ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19941058d0 0x7f19940feb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.832+0000 7f19991ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19940ff320 con 0x7f19941058d0 2023-07-06T20:17:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.832+0000 7f19991ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19940ff490 con 0x7f1994102fb0 2023-07-06T20:17:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.832+0000 7f19991ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199419f900 con 0x7f1994100690 2023-07-06T20:17:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.832+0000 7f1992d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1994100690 0x7f1994100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.832+0000 7f1992d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1994100690 0x7f1994100000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:55082/0 (socket says 172.21.15.69:55082) 2023-07-06T20:17:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.832+0000 7f1992d9d700 1 -- 172.21.15.69:0/2588760428 learned_addr learned my addr 172.21.15.69:0/2588760428 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.832+0000 7f199359e700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19941058d0 0x7f19940feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.832+0000 7f199259c700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1994102fb0 0x7f19940fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.833+0000 7f1992d9d700 1 -- 172.21.15.69:0/2588760428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1994102fb0 msgr2=0x7f19940fe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.833+0000 7f1992d9d700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1994102fb0 0x7f19940fe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.833+0000 7f1992d9d700 1 -- 172.21.15.69:0/2588760428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19941058d0 msgr2=0x7f19940feb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.833+0000 7f1992d9d700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19941058d0 0x7f19940feb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.833+0000 7f1992d9d700 1 -- 172.21.15.69:0/2588760428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19840095f0 con 0x7f1994100690 2023-07-06T20:17:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.833+0000 7f199359e700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19941058d0 0x7f19940feb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.833+0000 7f1992d9d700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1994100690 0x7f1994100000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f198400b680 tx=0x7f19840057b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.833+0000 7f197bfff700 1 -- 172.21.15.69:0/2588760428 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198401d070 con 0x7f1994100690 2023-07-06T20:17:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.833+0000 7f19991ff700 1 -- 172.21.15.69:0/2588760428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199419fb80 con 0x7f1994100690 2023-07-06T20:17:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.833+0000 7f197bfff700 1 -- 172.21.15.69:0/2588760428 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1984022470 con 0x7f1994100690 2023-07-06T20:17:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.833+0000 7f197bfff700 1 -- 172.21.15.69:0/2588760428 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984017650 con 0x7f1994100690 2023-07-06T20:17:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.833+0000 7f19991ff700 1 -- 172.21.15.69:0/2588760428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199419ffb0 con 0x7f1994100690 2023-07-06T20:17:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.834+0000 7f19991ff700 1 -- 172.21.15.69:0/2588760428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19940fbca0 con 0x7f1994100690 2023-07-06T20:17:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.836+0000 7f197bfff700 1 -- 172.21.15.69:0/2588760428 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f19840225e0 con 0x7f1994100690 2023-07-06T20:17:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.837+0000 7f197bfff700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f197c0a11a0 0x7f197c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.838+0000 7f197bfff700 1 -- 172.21.15.69:0/2588760428 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f19840cdbd0 con 0x7f1994100690 2023-07-06T20:17:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.838+0000 7f199259c700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f197c0a11a0 0x7f197c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.838+0000 7f199259c700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f197c0a11a0 0x7f197c0a3650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1988009c60 tx=0x7f1988009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:23.840+0000 7f197bfff700 1 -- 172.21.15.69:0/2588760428 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f19840977b0 con 0x7f1994100690 2023-07-06T20:17:24.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.092+0000 7f19991ff700 1 -- 172.21.15.69:0/2588760428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f199402c8e0 con 0x7f1994100690 2023-07-06T20:17:24.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.093+0000 7f197bfff700 1 -- 172.21.15.69:0/2588760428 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9592 (secure 0 0 0) 0x7f198409add0 con 0x7f1994100690 2023-07-06T20:17:24.095 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:24.095 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3235525224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3235525224},{"type":"v1","addr":"172.21.15.98:6839","nonce":3235525224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-07-06T20:17:24.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.095+0000 7f19991ff700 1 -- 172.21.15.69:0/2588760428 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f197c0a11a0 msgr2=0x7f197c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:24.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.095+0000 7f19991ff700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f197c0a11a0 0x7f197c0a3650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1988009c60 tx=0x7f1988009260 comp rx=0 tx=0).stop 2023-07-06T20:17:24.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.095+0000 7f19991ff700 1 -- 172.21.15.69:0/2588760428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1994100690 msgr2=0x7f1994100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:24.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.095+0000 7f19991ff700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1994100690 0x7f1994100000 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f198400b680 tx=0x7f19840057b0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.095+0000 7f19991ff700 1 -- 172.21.15.69:0/2588760428 shutdown_connections 2023-07-06T20:17:24.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.095+0000 7f19991ff700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f197c0a11a0 0x7f197c0a3650 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.095+0000 7f19991ff700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19941058d0 0x7f19940feb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.095+0000 7f19991ff700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1994100690 0x7f1994100000 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.095+0000 7f19991ff700 1 --2- 172.21.15.69:0/2588760428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1994102fb0 0x7f19940fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.095+0000 7f19991ff700 1 -- 172.21.15.69:0/2588760428 >> 172.21.15.69:0/2588760428 conn(0x7f19940fa130 msgr2=0x7f19941012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:24.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.096+0000 7f19991ff700 1 -- 172.21.15.69:0/2588760428 shutdown_connections 2023-07-06T20:17:24.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.096+0000 7f19991ff700 1 -- 172.21.15.69:0/2588760428 wait complete. 2023-07-06T20:17:24.099 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 13 2023-07-06T20:17:24.117 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:17:24.345 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.343+0000 7ffb669ff700 1 -- 172.21.15.69:0/3857587139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb600ffc30 msgr2=0x7ffb60100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:24.345 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.343+0000 7ffb669ff700 1 --2- 172.21.15.69:0/3857587139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb600ffc30 0x7ffb60100040 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009990 tx=0x7ffb50009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.343+0000 7ffb669ff700 1 -- 172.21.15.69:0/3857587139 shutdown_connections 2023-07-06T20:17:24.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.343+0000 7ffb669ff700 1 --2- 172.21.15.69:0/3857587139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb601017b0 0x7ffb60108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.343+0000 7ffb669ff700 1 --2- 172.21.15.69:0/3857587139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb600ffc30 0x7ffb60100040 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.343+0000 7ffb669ff700 1 --2- 172.21.15.69:0/3857587139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb60100e20 0x7ffb60101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.343+0000 7ffb669ff700 1 -- 172.21.15.69:0/3857587139 >> 172.21.15.69:0/3857587139 conn(0x7ffb600fb210 msgr2=0x7ffb600fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:24.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.343+0000 7ffb669ff700 1 -- 172.21.15.69:0/3857587139 shutdown_connections 2023-07-06T20:17:24.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.343+0000 7ffb669ff700 1 -- 172.21.15.69:0/3857587139 wait complete. 2023-07-06T20:17:24.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.344+0000 7ffb669ff700 1 Processor -- start 2023-07-06T20:17:24.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.344+0000 7ffb669ff700 1 -- start start 2023-07-06T20:17:24.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb669ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb600ffc30 0x7ffb600743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:24.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb669ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb60100e20 0x7ffb60072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:24.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb669ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb601017b0 0x7ffb60072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:24.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb669ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60073710 con 0x7ffb600ffc30 2023-07-06T20:17:24.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb669ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60073880 con 0x7ffb60100e20 2023-07-06T20:17:24.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb669ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6019f9c0 con 0x7ffb601017b0 2023-07-06T20:17:24.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb5ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb600ffc30 0x7ffb600743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:24.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb5f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb60100e20 0x7ffb60072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:24.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb5f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb60100e20 0x7ffb60072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:50732/0 (socket says 172.21.15.69:50732) 2023-07-06T20:17:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb5ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb600ffc30 0x7ffb600743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56106/0 (socket says 172.21.15.69:56106) 2023-07-06T20:17:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb5f7fe700 1 -- 172.21.15.69:0/1654151201 learned_addr learned my addr 172.21.15.69:0/1654151201 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb64f9c700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb601017b0 0x7ffb60072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb5f7fe700 1 -- 172.21.15.69:0/1654151201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb601017b0 msgr2=0x7ffb60072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb5f7fe700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb601017b0 0x7ffb60072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb5f7fe700 1 -- 172.21.15.69:0/1654151201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb600ffc30 msgr2=0x7ffb600743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb5f7fe700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb600ffc30 0x7ffb600743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.345+0000 7ffb5f7fe700 1 -- 172.21.15.69:0/1654151201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb500095f0 con 0x7ffb60100e20 2023-07-06T20:17:24.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.346+0000 7ffb5f7fe700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb60100e20 0x7ffb60072a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400e9a0 tx=0x7ffb5400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:24.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.346+0000 7ffb5d7fa700 1 -- 172.21.15.69:0/1654151201 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54009790 con 0x7ffb60100e20 2023-07-06T20:17:24.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.346+0000 7ffb669ff700 1 -- 172.21.15.69:0/1654151201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6019fca0 con 0x7ffb60100e20 2023-07-06T20:17:24.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.346+0000 7ffb5d7fa700 1 -- 172.21.15.69:0/1654151201 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb54010430 con 0x7ffb60100e20 2023-07-06T20:17:24.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.346+0000 7ffb5d7fa700 1 -- 172.21.15.69:0/1654151201 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54018680 con 0x7ffb60100e20 2023-07-06T20:17:24.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.346+0000 7ffb669ff700 1 -- 172.21.15.69:0/1654151201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb601a01b0 con 0x7ffb60100e20 2023-07-06T20:17:24.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.347+0000 7ffb669ff700 1 -- 172.21.15.69:0/1654151201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6010b450 con 0x7ffb60100e20 2023-07-06T20:17:24.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.349+0000 7ffb5d7fa700 1 -- 172.21.15.69:0/1654151201 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ffb540105a0 con 0x7ffb60100e20 2023-07-06T20:17:24.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.352+0000 7ffb5d7fa700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffb480a11a0 0x7ffb480a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:24.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.352+0000 7ffb5d7fa700 1 -- 172.21.15.69:0/1654151201 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffb54014070 con 0x7ffb60100e20 2023-07-06T20:17:24.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.352+0000 7ffb5ffff700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffb480a11a0 0x7ffb480a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:24.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.352+0000 7ffb5d7fa700 1 -- 172.21.15.69:0/1654151201 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ffb54096190 con 0x7ffb60100e20 2023-07-06T20:17:24.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.353+0000 7ffb5ffff700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffb480a11a0 0x7ffb480a3650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000b2e0 tx=0x7ffb50005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:24 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] up:boot 2023-07-06T20:17:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:24 smithi069 ceph-mon[126399]: fsmap 12 up:standby 2023-07-06T20:17:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:24 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-06T20:17:24.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:24 smithi069 ceph-mon[126399]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:24.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:24 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2588760428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:24.568 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.566+0000 7ffb669ff700 1 -- 172.21.15.69:0/1654151201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb60106ac0 con 0x7ffb60100e20 2023-07-06T20:17:24.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.568+0000 7ffb5d7fa700 1 -- 172.21.15.69:0/1654151201 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18368 (secure 0 0 0) 0x7ffb540997b0 con 0x7ffb60100e20 2023-07-06T20:17:24.570 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:24.570 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":79,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:16:42.349843+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:17:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.570+0000 7ffb669ff700 1 -- 172.21.15.69:0/1654151201 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffb480a11a0 msgr2=0x7ffb480a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.570+0000 7ffb669ff700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffb480a11a0 0x7ffb480a3650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000b2e0 tx=0x7ffb50005fb0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.570+0000 7ffb669ff700 1 -- 172.21.15.69:0/1654151201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb60100e20 msgr2=0x7ffb60072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.570+0000 7ffb669ff700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb60100e20 0x7ffb60072a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400e9a0 tx=0x7ffb5400ecb0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.570+0000 7ffb669ff700 1 -- 172.21.15.69:0/1654151201 shutdown_connections 2023-07-06T20:17:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.570+0000 7ffb669ff700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffb480a11a0 0x7ffb480a3650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.570+0000 7ffb669ff700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb601017b0 0x7ffb60072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.570+0000 7ffb669ff700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb600ffc30 0x7ffb600743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.570+0000 7ffb669ff700 1 --2- 172.21.15.69:0/1654151201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb60100e20 0x7ffb60072a30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.570+0000 7ffb669ff700 1 -- 172.21.15.69:0/1654151201 >> 172.21.15.69:0/1654151201 conn(0x7ffb600fb210 msgr2=0x7ffb600fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.570+0000 7ffb669ff700 1 -- 172.21.15.69:0/1654151201 shutdown_connections 2023-07-06T20:17:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.570+0000 7ffb669ff700 1 -- 172.21.15.69:0/1654151201 wait complete. 2023-07-06T20:17:24.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:14.022799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:17:24.591 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:17:24.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:24 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] up:boot 2023-07-06T20:17:24.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:24 smithi156 ceph-mon[129280]: fsmap 12 up:standby 2023-07-06T20:17:24.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:24 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-06T20:17:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:24 smithi156 ceph-mon[129280]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:24.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:24 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2588760428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:24.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:24 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] up:boot 2023-07-06T20:17:24.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:24 smithi098 ceph-mon[128968]: fsmap 12 up:standby 2023-07-06T20:17:24.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:24 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-06T20:17:24.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:24 smithi098 ceph-mon[128968]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:24.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:24 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2588760428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:24.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.809+0000 7f4e94fff700 1 -- 172.21.15.69:0/1466075666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e90107ae0 msgr2=0x7f4e90109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:24.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.809+0000 7f4e94fff700 1 --2- 172.21.15.69:0/1466075666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e90107ae0 0x7f4e90109f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800b0a0 tx=0x7f4e8800b3b0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.809+0000 7f4e94fff700 1 -- 172.21.15.69:0/1466075666 shutdown_connections 2023-07-06T20:17:24.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.809+0000 7f4e94fff700 1 --2- 172.21.15.69:0/1466075666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e90107ae0 0x7f4e90109f60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.809+0000 7f4e94fff700 1 --2- 172.21.15.69:0/1466075666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e90074680 0x7f4e90072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.809+0000 7f4e94fff700 1 --2- 172.21.15.69:0/1466075666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e900730a0 0x7f4e90073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.809+0000 7f4e94fff700 1 -- 172.21.15.69:0/1466075666 >> 172.21.15.69:0/1466075666 conn(0x7f4e900fb260 msgr2=0x7f4e900fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:24.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.809+0000 7f4e94fff700 1 -- 172.21.15.69:0/1466075666 shutdown_connections 2023-07-06T20:17:24.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.809+0000 7f4e94fff700 1 -- 172.21.15.69:0/1466075666 wait complete. 2023-07-06T20:17:24.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.810+0000 7f4e94fff700 1 Processor -- start 2023-07-06T20:17:24.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.810+0000 7f4e94fff700 1 -- start start 2023-07-06T20:17:24.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.810+0000 7f4e94fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e900730a0 0x7f4e90197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:24.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e94fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e90074680 0x7f4e901981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:24.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e94fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e90107ae0 0x7f4e9019f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:24.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e94fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9019f820 con 0x7f4e90107ae0 2023-07-06T20:17:24.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e94fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9019f990 con 0x7f4e900730a0 2023-07-06T20:17:24.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e94fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9019fb50 con 0x7f4e90074680 2023-07-06T20:17:24.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e8ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e90107ae0 0x7f4e9019f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:24.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e8ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e90107ae0 0x7f4e9019f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56128/0 (socket says 172.21.15.69:56128) 2023-07-06T20:17:24.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e8ed9d700 1 -- 172.21.15.69:0/1414266972 learned_addr learned my addr 172.21.15.69:0/1414266972 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:24.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e8e59c700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e900730a0 0x7f4e90197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:24.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e8dd9b700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e90074680 0x7f4e901981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:24.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e8ed9d700 1 -- 172.21.15.69:0/1414266972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e90074680 msgr2=0x7f4e901981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:24.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e8ed9d700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e90074680 0x7f4e901981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e8ed9d700 1 -- 172.21.15.69:0/1414266972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e900730a0 msgr2=0x7f4e90197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:24.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e8ed9d700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e900730a0 0x7f4e90197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:24.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e8ed9d700 1 -- 172.21.15.69:0/1414266972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e88009d00 con 0x7f4e90107ae0 2023-07-06T20:17:24.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e8e59c700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e900730a0 0x7f4e90197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:17:24.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.811+0000 7f4e8ed9d700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e90107ae0 0x7f4e9019f250 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009fd0 tx=0x7f4e8800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:24.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.812+0000 7f4e7f7fe700 1 -- 172.21.15.69:0/1414266972 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8800e040 con 0x7f4e90107ae0 2023-07-06T20:17:24.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.812+0000 7f4e7f7fe700 1 -- 172.21.15.69:0/1414266972 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e88003e10 con 0x7f4e90107ae0 2023-07-06T20:17:24.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.812+0000 7f4e94fff700 1 -- 172.21.15.69:0/1414266972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9019fdd0 con 0x7f4e90107ae0 2023-07-06T20:17:24.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.812+0000 7f4e7f7fe700 1 -- 172.21.15.69:0/1414266972 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8801b750 con 0x7f4e90107ae0 2023-07-06T20:17:24.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.812+0000 7f4e94fff700 1 -- 172.21.15.69:0/1414266972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e901a01a0 con 0x7f4e90107ae0 2023-07-06T20:17:24.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.813+0000 7f4e94fff700 1 -- 172.21.15.69:0/1414266972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e90106190 con 0x7f4e90107ae0 2023-07-06T20:17:24.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.815+0000 7f4e7f7fe700 1 -- 172.21.15.69:0/1414266972 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4e8801da30 con 0x7f4e90107ae0 2023-07-06T20:17:24.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.817+0000 7f4e7f7fe700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4e780a1250 0x7f4e780a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:24.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.818+0000 7f4e7f7fe700 1 -- 172.21.15.69:0/1414266972 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4e880cd980 con 0x7f4e90107ae0 2023-07-06T20:17:24.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.818+0000 7f4e8e59c700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4e780a1250 0x7f4e780a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:24.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.818+0000 7f4e8e59c700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4e780a1250 0x7f4e780a3700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e90198b80 tx=0x7f4e80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:24.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:24.821+0000 7f4e7f7fe700 1 -- 172.21.15.69:0/1414266972 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4e88097560 con 0x7f4e90107ae0 2023-07-06T20:17:25.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.063+0000 7f4e94fff700 1 -- 172.21.15.69:0/1414266972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e90061250 con 0x7f4e90107ae0 2023-07-06T20:17:25.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.063+0000 7f4e7f7fe700 1 -- 172.21.15.69:0/1414266972 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e8809ab80 con 0x7f4e90107ae0 2023-07-06T20:17:25.066 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2023-07-06T20:17:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.065+0000 7f4e94fff700 1 -- 172.21.15.69:0/1414266972 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4e780a1250 msgr2=0x7f4e780a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.065+0000 7f4e94fff700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4e780a1250 0x7f4e780a3700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e90198b80 tx=0x7f4e80009260 comp rx=0 tx=0).stop 2023-07-06T20:17:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.065+0000 7f4e94fff700 1 -- 172.21.15.69:0/1414266972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e90107ae0 msgr2=0x7f4e9019f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.065+0000 7f4e94fff700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e90107ae0 0x7f4e9019f250 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009fd0 tx=0x7f4e8800bce0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.066+0000 7f4e94fff700 1 -- 172.21.15.69:0/1414266972 shutdown_connections 2023-07-06T20:17:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.066+0000 7f4e94fff700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4e780a1250 0x7f4e780a3700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.066+0000 7f4e94fff700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e90107ae0 0x7f4e9019f250 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.066+0000 7f4e94fff700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e900730a0 0x7f4e90197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.066+0000 7f4e94fff700 1 --2- 172.21.15.69:0/1414266972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e90074680 0x7f4e901981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.066+0000 7f4e94fff700 1 -- 172.21.15.69:0/1414266972 >> 172.21.15.69:0/1414266972 conn(0x7f4e900fb260 msgr2=0x7f4e90109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:25.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.066+0000 7f4e94fff700 1 -- 172.21.15.69:0/1414266972 shutdown_connections 2023-07-06T20:17:25.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.066+0000 7f4e94fff700 1 -- 172.21.15.69:0/1414266972 wait complete. 2023-07-06T20:17:25.089 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T20:17:25.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.289+0000 7f30bb3ff700 1 -- 172.21.15.69:0/303293221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30b40730a0 msgr2=0x7f30b4073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:25.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.289+0000 7f30bb3ff700 1 --2- 172.21.15.69:0/303293221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30b40730a0 0x7f30b4073510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009990 tx=0x7f30a4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.289+0000 7f30bb3ff700 1 -- 172.21.15.69:0/303293221 shutdown_connections 2023-07-06T20:17:25.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.289+0000 7f30bb3ff700 1 --2- 172.21.15.69:0/303293221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b410c200 0x7f30b410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.289+0000 7f30bb3ff700 1 --2- 172.21.15.69:0/303293221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30b4074680 0x7f30b4072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.289+0000 7f30bb3ff700 1 --2- 172.21.15.69:0/303293221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30b40730a0 0x7f30b4073510 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.289+0000 7f30bb3ff700 1 -- 172.21.15.69:0/303293221 >> 172.21.15.69:0/303293221 conn(0x7f30b40fb210 msgr2=0x7f30b40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:25.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.289+0000 7f30bb3ff700 1 -- 172.21.15.69:0/303293221 shutdown_connections 2023-07-06T20:17:25.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.289+0000 7f30bb3ff700 1 -- 172.21.15.69:0/303293221 wait complete. 2023-07-06T20:17:25.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.290+0000 7f30bb3ff700 1 Processor -- start 2023-07-06T20:17:25.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.290+0000 7f30bb3ff700 1 -- start start 2023-07-06T20:17:25.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30bb3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30b40730a0 0x7f30b419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:25.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30bb3ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30b4074680 0x7f30b419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:25.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30bb3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b410c200 0x7f30b41a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:25.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30bb3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b41a3b80 con 0x7f30b40730a0 2023-07-06T20:17:25.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30bb3ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b41a3cc0 con 0x7f30b4074680 2023-07-06T20:17:25.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30bb3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b41a3e50 con 0x7f30b410c200 2023-07-06T20:17:25.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30b919b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30b40730a0 0x7f30b419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:25.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30b899a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30b4074680 0x7f30b419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:25.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30b899a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30b4074680 0x7f30b419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:50768/0 (socket says 172.21.15.69:50768) 2023-07-06T20:17:25.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30b899a700 1 -- 172.21.15.69:0/2518625770 learned_addr learned my addr 172.21.15.69:0/2518625770 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:25.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30b999c700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b410c200 0x7f30b41a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:25.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30b899a700 1 -- 172.21.15.69:0/2518625770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b410c200 msgr2=0x7f30b41a35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:25.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.291+0000 7f30b899a700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b410c200 0x7f30b41a35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.292+0000 7f30b899a700 1 -- 172.21.15.69:0/2518625770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30b40730a0 msgr2=0x7f30b419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:25.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.292+0000 7f30b899a700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30b40730a0 0x7f30b419c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.292+0000 7f30b899a700 1 -- 172.21.15.69:0/2518625770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a40095f0 con 0x7f30b4074680 2023-07-06T20:17:25.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.292+0000 7f30b899a700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30b4074680 0x7f30b419c540 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f30a400b680 tx=0x7f30a4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:25.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.292+0000 7f30aa7fc700 1 -- 172.21.15.69:0/2518625770 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a401d070 con 0x7f30b4074680 2023-07-06T20:17:25.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.292+0000 7f30bb3ff700 1 -- 172.21.15.69:0/2518625770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b41a40d0 con 0x7f30b4074680 2023-07-06T20:17:25.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.292+0000 7f30aa7fc700 1 -- 172.21.15.69:0/2518625770 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30a4021410 con 0x7f30b4074680 2023-07-06T20:17:25.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.292+0000 7f30aa7fc700 1 -- 172.21.15.69:0/2518625770 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4017650 con 0x7f30b4074680 2023-07-06T20:17:25.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.292+0000 7f30bb3ff700 1 -- 172.21.15.69:0/2518625770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b41a4550 con 0x7f30b4074680 2023-07-06T20:17:25.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.293+0000 7f30bb3ff700 1 -- 172.21.15.69:0/2518625770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b4101c40 con 0x7f30b4074680 2023-07-06T20:17:25.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.297+0000 7f30aa7fc700 1 -- 172.21.15.69:0/2518625770 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f30a4021580 con 0x7f30b4074680 2023-07-06T20:17:25.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.298+0000 7f30aa7fc700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f30a00a11a0 0x7f30a00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:25.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.298+0000 7f30aa7fc700 1 -- 172.21.15.69:0/2518625770 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f30a40cd9a0 con 0x7f30b4074680 2023-07-06T20:17:25.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.298+0000 7f30b919b700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f30a00a11a0 0x7f30a00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:25.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.299+0000 7f30b919b700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f30a00a11a0 0x7f30a00a3650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f30b4074a50 tx=0x7f30b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:25.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.302+0000 7f30aa7fc700 1 -- 172.21.15.69:0/2518625770 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f30a4097580 con 0x7f30b4074680 2023-07-06T20:17:25.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:25 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1654151201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:25.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:25 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1414266972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:17:25.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.545+0000 7f30bb3ff700 1 -- 172.21.15.69:0/2518625770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f30b4061250 con 0x7f30b4074680 2023-07-06T20:17:25.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.546+0000 7f30aa7fc700 1 -- 172.21.15.69:0/2518625770 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f30a409aba0 con 0x7f30b4074680 2023-07-06T20:17:25.548 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:25.548 INFO:teuthology.orchestra.run.smithi069.stdout:[ 2023-07-06T20:17:25.548 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:17:25.549 INFO:teuthology.orchestra.run.smithi069.stdout: "poolnum": 1, 2023-07-06T20:17:25.549 INFO:teuthology.orchestra.run.smithi069.stdout: "poolname": ".mgr" 2023-07-06T20:17:25.549 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:17:25.549 INFO:teuthology.orchestra.run.smithi069.stdout:] 2023-07-06T20:17:25.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.549+0000 7f30bb3ff700 1 -- 172.21.15.69:0/2518625770 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f30a00a11a0 msgr2=0x7f30a00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:25.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.549+0000 7f30bb3ff700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f30a00a11a0 0x7f30a00a3650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f30b4074a50 tx=0x7f30b0009260 comp rx=0 tx=0).stop 2023-07-06T20:17:25.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.549+0000 7f30bb3ff700 1 -- 172.21.15.69:0/2518625770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30b4074680 msgr2=0x7f30b419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:25.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.549+0000 7f30bb3ff700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30b4074680 0x7f30b419c540 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f30a400b680 tx=0x7f30a4005740 comp rx=0 tx=0).stop 2023-07-06T20:17:25.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.549+0000 7f30bb3ff700 1 -- 172.21.15.69:0/2518625770 shutdown_connections 2023-07-06T20:17:25.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.549+0000 7f30bb3ff700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f30a00a11a0 0x7f30a00a3650 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.549+0000 7f30bb3ff700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30b410c200 0x7f30b41a35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.549+0000 7f30bb3ff700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30b40730a0 0x7f30b419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.549+0000 7f30bb3ff700 1 --2- 172.21.15.69:0/2518625770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30b4074680 0x7f30b419c540 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.549+0000 7f30bb3ff700 1 -- 172.21.15.69:0/2518625770 >> 172.21.15.69:0/2518625770 conn(0x7f30b40fb210 msgr2=0x7f30b40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:25.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.549+0000 7f30bb3ff700 1 -- 172.21.15.69:0/2518625770 shutdown_connections 2023-07-06T20:17:25.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.549+0000 7f30bb3ff700 1 -- 172.21.15.69:0/2518625770 wait complete. 2023-07-06T20:17:25.571 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T20:17:25.572 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T20:17:25.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:25 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1654151201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:25.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:25 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1414266972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:17:25.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:25 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1654151201' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:25.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:25 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1414266972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:17:25.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.778+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/2888712935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e08100e30 msgr2=0x7f1e08101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:25.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.778+0000 7f1e0f3ff700 1 --2- 172.21.15.69:0/2888712935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e08100e30 0x7f1e08101280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009990 tx=0x7f1df8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.778+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/2888712935 shutdown_connections 2023-07-06T20:17:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.778+0000 7f1e0f3ff700 1 --2- 172.21.15.69:0/2888712935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e081017c0 0x7f1e08108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.778+0000 7f1e0f3ff700 1 --2- 172.21.15.69:0/2888712935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e080ffc40 0x7f1e08100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.778+0000 7f1e0f3ff700 1 --2- 172.21.15.69:0/2888712935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e08100e30 0x7f1e08101280 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.778+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/2888712935 >> 172.21.15.69:0/2888712935 conn(0x7f1e080fb260 msgr2=0x7f1e080fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.778+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/2888712935 shutdown_connections 2023-07-06T20:17:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.778+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/2888712935 wait complete. 2023-07-06T20:17:25.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.779+0000 7f1e0f3ff700 1 Processor -- start 2023-07-06T20:17:25.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.779+0000 7f1e0f3ff700 1 -- start start 2023-07-06T20:17:25.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0f3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e080ffc40 0x7f1e08197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:25.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0f3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e08100e30 0x7f1e081980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0f3ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e081017c0 0x7f1e0819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0f3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0819f700 con 0x7f1e080ffc40 2023-07-06T20:17:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0f3ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0819f840 con 0x7f1e081017c0 2023-07-06T20:17:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0f3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0819f9d0 con 0x7f1e08100e30 2023-07-06T20:17:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0d19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e080ffc40 0x7f1e08197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0d19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e080ffc40 0x7f1e08197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56176/0 (socket says 172.21.15.69:56176) 2023-07-06T20:17:25.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0d99c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e081017c0 0x7f1e0819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:25.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0d19b700 1 -- 172.21.15.69:0/3153417458 learned_addr learned my addr 172.21.15.69:0/3153417458 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:25.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0c99a700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e08100e30 0x7f1e081980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:25.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0d19b700 1 -- 172.21.15.69:0/3153417458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e08100e30 msgr2=0x7f1e081980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:25.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0d19b700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e08100e30 0x7f1e081980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0d19b700 1 -- 172.21.15.69:0/3153417458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e081017c0 msgr2=0x7f1e0819f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:25.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0d19b700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e081017c0 0x7f1e0819f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:25.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.780+0000 7f1e0d19b700 1 -- 172.21.15.69:0/3153417458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04009520 con 0x7f1e080ffc40 2023-07-06T20:17:25.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.781+0000 7f1e0d19b700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e080ffc40 0x7f1e08197b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400ea50 tx=0x7f1e0400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.781+0000 7f1dfe7fc700 1 -- 172.21.15.69:0/3153417458 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400cbd0 con 0x7f1e080ffc40 2023-07-06T20:17:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.781+0000 7f1dfe7fc700 1 -- 172.21.15.69:0/3153417458 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e0400cd30 con 0x7f1e080ffc40 2023-07-06T20:17:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.781+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/3153417458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df80095f0 con 0x7f1e080ffc40 2023-07-06T20:17:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.781+0000 7f1dfe7fc700 1 -- 172.21.15.69:0/3153417458 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04018680 con 0x7f1e080ffc40 2023-07-06T20:17:25.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.781+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/3153417458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e081a0080 con 0x7f1e080ffc40 2023-07-06T20:17:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.782+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/3153417458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0810b460 con 0x7f1e080ffc40 2023-07-06T20:17:25.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.785+0000 7f1dfe7fc700 1 -- 172.21.15.69:0/3153417458 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1e04010430 con 0x7f1e080ffc40 2023-07-06T20:17:25.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.786+0000 7f1dfe7fc700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1df40a11d0 0x7f1df40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:25.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.786+0000 7f1dfe7fc700 1 -- 172.21.15.69:0/3153417458 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1e04014070 con 0x7f1e080ffc40 2023-07-06T20:17:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.787+0000 7f1e0c99a700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1df40a11d0 0x7f1df40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:25.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.787+0000 7f1e0c99a700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1df40a11d0 0x7f1df40a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1df8006010 tx=0x7f1df8005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:25.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:25.793+0000 7f1dfe7fc700 1 -- 172.21.15.69:0/3153417458 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1e040964b0 con 0x7f1e080ffc40 2023-07-06T20:17:26.029 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.027+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/3153417458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1e0810b5f0 con 0x7f1e080ffc40 2023-07-06T20:17:26.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.579+0000 7f1dfe7fc700 1 -- 172.21.15.69:0/3153417458 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1e04099ad0 con 0x7f1e080ffc40 2023-07-06T20:17:26.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.584+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/3153417458 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1df40a11d0 msgr2=0x7f1df40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:26.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.584+0000 7f1e0f3ff700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1df40a11d0 0x7f1df40a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1df8006010 tx=0x7f1df8005a90 comp rx=0 tx=0).stop 2023-07-06T20:17:26.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.584+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/3153417458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e080ffc40 msgr2=0x7f1e08197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:26.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.584+0000 7f1e0f3ff700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e080ffc40 0x7f1e08197b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400ea50 tx=0x7f1e0400ed60 comp rx=0 tx=0).stop 2023-07-06T20:17:26.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.586+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/3153417458 shutdown_connections 2023-07-06T20:17:26.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.586+0000 7f1e0f3ff700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1df40a11d0 0x7f1df40a3680 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:26.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.586+0000 7f1e0f3ff700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e081017c0 0x7f1e0819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:26.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.586+0000 7f1e0f3ff700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e080ffc40 0x7f1e08197b80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:26.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.586+0000 7f1e0f3ff700 1 --2- 172.21.15.69:0/3153417458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e08100e30 0x7f1e081980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:26.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.586+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/3153417458 >> 172.21.15.69:0/3153417458 conn(0x7f1e080fb260 msgr2=0x7f1e080fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:26.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.586+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/3153417458 shutdown_connections 2023-07-06T20:17:26.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.586+0000 7f1e0f3ff700 1 -- 172.21.15.69:0/3153417458 wait complete. 2023-07-06T20:17:26.589 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' created 2023-07-06T20:17:26.608 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T20:17:26.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.813+0000 7f7e841ff700 1 -- 172.21.15.69:0/757752716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e7c100e60 msgr2=0x7f7e7c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:26.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.813+0000 7f7e841ff700 1 --2- 172.21.15.69:0/757752716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e7c100e60 0x7f7e7c1012d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009990 tx=0x7f7e6c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:26.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.813+0000 7f7e841ff700 1 -- 172.21.15.69:0/757752716 shutdown_connections 2023-07-06T20:17:26.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.813+0000 7f7e841ff700 1 --2- 172.21.15.69:0/757752716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c101810 0x7f7e7c1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:26.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.813+0000 7f7e841ff700 1 --2- 172.21.15.69:0/757752716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e7c0ffbd0 0x7f7e7c0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:26.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.813+0000 7f7e841ff700 1 --2- 172.21.15.69:0/757752716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e7c100e60 0x7f7e7c1012d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:26.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.813+0000 7f7e841ff700 1 -- 172.21.15.69:0/757752716 >> 172.21.15.69:0/757752716 conn(0x7f7e7c0fb1f0 msgr2=0x7f7e7c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:26.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.813+0000 7f7e841ff700 1 -- 172.21.15.69:0/757752716 shutdown_connections 2023-07-06T20:17:26.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.814+0000 7f7e841ff700 1 -- 172.21.15.69:0/757752716 wait complete. 2023-07-06T20:17:26.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.814+0000 7f7e841ff700 1 Processor -- start 2023-07-06T20:17:26.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.815+0000 7f7e841ff700 1 -- start start 2023-07-06T20:17:26.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.815+0000 7f7e841ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e7c0ffbd0 0x7f7e7c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:26.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.815+0000 7f7e841ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e7c100e60 0x7f7e7c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:26.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.815+0000 7f7e841ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c101810 0x7f7e7c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:26.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.815+0000 7f7e841ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c19f680 con 0x7f7e7c0ffbd0 2023-07-06T20:17:26.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.815+0000 7f7e841ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c19f7c0 con 0x7f7e7c100e60 2023-07-06T20:17:26.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.815+0000 7f7e841ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c19f950 con 0x7f7e7c101810 2023-07-06T20:17:26.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.815+0000 7f7e8279c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c101810 0x7f7e7c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:26.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.815+0000 7f7e8279c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c101810 0x7f7e7c19f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:55152/0 (socket says 172.21.15.69:55152) 2023-07-06T20:17:26.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.815+0000 7f7e8279c700 1 -- 172.21.15.69:0/3449139776 learned_addr learned my addr 172.21.15.69:0/3449139776 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:26.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.815+0000 7f7e81f9b700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e7c0ffbd0 0x7f7e7c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:26.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.815+0000 7f7e8179a700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e7c100e60 0x7f7e7c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:26.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.816+0000 7f7e8279c700 1 -- 172.21.15.69:0/3449139776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e7c100e60 msgr2=0x7f7e7c198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:26.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.816+0000 7f7e8279c700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e7c100e60 0x7f7e7c198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:26.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.816+0000 7f7e8279c700 1 -- 172.21.15.69:0/3449139776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e7c0ffbd0 msgr2=0x7f7e7c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:26.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.816+0000 7f7e8279c700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e7c0ffbd0 0x7f7e7c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:26.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.816+0000 7f7e8279c700 1 -- 172.21.15.69:0/3449139776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c0095f0 con 0x7f7e7c101810 2023-07-06T20:17:26.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.816+0000 7f7e8279c700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c101810 0x7f7e7c19f0b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400ef30 tx=0x7f7e7400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:26.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.816+0000 7f7e72ffd700 1 -- 172.21.15.69:0/3449139776 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74018070 con 0x7f7e7c101810 2023-07-06T20:17:26.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.816+0000 7f7e72ffd700 1 -- 172.21.15.69:0/3449139776 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e7400f040 con 0x7f7e7c101810 2023-07-06T20:17:26.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.816+0000 7f7e841ff700 1 -- 172.21.15.69:0/3449139776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7c19fbd0 con 0x7f7e7c101810 2023-07-06T20:17:26.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.816+0000 7f7e72ffd700 1 -- 172.21.15.69:0/3449139776 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400cdb0 con 0x7f7e7c101810 2023-07-06T20:17:26.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.816+0000 7f7e841ff700 1 -- 172.21.15.69:0/3449139776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7c1a00e0 con 0x7f7e7c101810 2023-07-06T20:17:26.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.818+0000 7f7e841ff700 1 -- 172.21.15.69:0/3449139776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c1922b0 con 0x7f7e7c101810 2023-07-06T20:17:26.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.820+0000 7f7e72ffd700 1 -- 172.21.15.69:0/3449139776 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7e74003a00 con 0x7f7e7c101810 2023-07-06T20:17:26.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.821+0000 7f7e72ffd700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7e680a1150 0x7f7e680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:26.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.821+0000 7f7e72ffd700 1 -- 172.21.15.69:0/3449139776 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f7e740cc350 con 0x7f7e7c101810 2023-07-06T20:17:26.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.821+0000 7f7e81f9b700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7e680a1150 0x7f7e680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:26.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.822+0000 7f7e81f9b700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7e680a1150 0x7f7e680a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009c60 tx=0x7f7e78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:26.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:26.823+0000 7f7e72ffd700 1 -- 172.21.15.69:0/3449139776 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7e74095e60 con 0x7f7e7c101810 2023-07-06T20:17:26.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:26 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:26.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:26 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:26.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:26 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:26.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:26 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:26.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:26 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2518625770' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-06T20:17:26.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:26 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:26.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:26 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:26.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:26 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:26.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:26 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:26.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:26 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2518625770' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-06T20:17:27.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:26 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:27.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:26 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:27.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:26 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:27.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:26 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:27.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:26 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2518625770' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-06T20:17:27.035 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.033+0000 7f7e841ff700 1 -- 172.21.15.69:0/3449139776 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7e7c02cd70 con 0x7f7e7c101810 2023-07-06T20:17:27.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.593+0000 7f7e72ffd700 1 -- 172.21.15.69:0/3449139776 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7e74099480 con 0x7f7e7c101810 2023-07-06T20:17:27.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.605+0000 7f7e841ff700 1 -- 172.21.15.69:0/3449139776 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7e680a1150 msgr2=0x7f7e680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:27.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.605+0000 7f7e841ff700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7e680a1150 0x7f7e680a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009c60 tx=0x7f7e78009260 comp rx=0 tx=0).stop 2023-07-06T20:17:27.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.605+0000 7f7e841ff700 1 -- 172.21.15.69:0/3449139776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c101810 msgr2=0x7f7e7c19f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:27.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.605+0000 7f7e841ff700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c101810 0x7f7e7c19f0b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400ef30 tx=0x7f7e7400c3a0 comp rx=0 tx=0).stop 2023-07-06T20:17:27.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.607+0000 7f7e841ff700 1 -- 172.21.15.69:0/3449139776 shutdown_connections 2023-07-06T20:17:27.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.607+0000 7f7e841ff700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7e680a1150 0x7f7e680a3600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:27.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.607+0000 7f7e841ff700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c101810 0x7f7e7c19f0b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:27.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.607+0000 7f7e841ff700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e7c0ffbd0 0x7f7e7c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:27.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.607+0000 7f7e841ff700 1 --2- 172.21.15.69:0/3449139776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e7c100e60 0x7f7e7c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:27.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.607+0000 7f7e841ff700 1 -- 172.21.15.69:0/3449139776 >> 172.21.15.69:0/3449139776 conn(0x7f7e7c0fb1f0 msgr2=0x7f7e7c106710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:27.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.609+0000 7f7e841ff700 1 -- 172.21.15.69:0/3449139776 shutdown_connections 2023-07-06T20:17:27.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.609+0000 7f7e841ff700 1 -- 172.21.15.69:0/3449139776 wait complete. 2023-07-06T20:17:27.613 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' created 2023-07-06T20:17:27.634 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T20:17:27.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.841+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2367662678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad00730a0 msgr2=0x7f9ad0073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:27.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.841+0000 7f9ad4fff700 1 --2- 172.21.15.69:0/2367662678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad00730a0 0x7f9ad0073510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009990 tx=0x7f9ac4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:27.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.841+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2367662678 shutdown_connections 2023-07-06T20:17:27.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.841+0000 7f9ad4fff700 1 --2- 172.21.15.69:0/2367662678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ad010c1f0 0x7f9ad010e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:27.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.841+0000 7f9ad4fff700 1 --2- 172.21.15.69:0/2367662678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ad0074680 0x7f9ad0072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:27.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.841+0000 7f9ad4fff700 1 --2- 172.21.15.69:0/2367662678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad00730a0 0x7f9ad0073510 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:27.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.841+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2367662678 >> 172.21.15.69:0/2367662678 conn(0x7f9ad00fb260 msgr2=0x7f9ad00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:27.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.841+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2367662678 shutdown_connections 2023-07-06T20:17:27.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.841+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2367662678 wait complete. 2023-07-06T20:17:27.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.842+0000 7f9ad4fff700 1 Processor -- start 2023-07-06T20:17:27.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.842+0000 7f9ad4fff700 1 -- start start 2023-07-06T20:17:27.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9ad4fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ad00730a0 0x7f9ad019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:27.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9ad4fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ad0074680 0x7f9ad019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:27.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9ad4fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad010c1f0 0x7f9ad01a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:27.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9ad4fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad01a3b60 con 0x7f9ad0074680 2023-07-06T20:17:27.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9ad4fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad01a3ca0 con 0x7f9ad010c1f0 2023-07-06T20:17:27.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9ad4fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad01a3e30 con 0x7f9ad00730a0 2023-07-06T20:17:27.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9ace59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ad00730a0 0x7f9ad019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:27.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9ace59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ad00730a0 0x7f9ad019bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:55164/0 (socket says 172.21.15.69:55164) 2023-07-06T20:17:27.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9aced9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad010c1f0 0x7f9ad01a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:27.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9ace59c700 1 -- 172.21.15.69:0/2369301515 learned_addr learned my addr 172.21.15.69:0/2369301515 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:27.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9acdd9b700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ad0074680 0x7f9ad019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:27.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9ace59c700 1 -- 172.21.15.69:0/2369301515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad010c1f0 msgr2=0x7f9ad01a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:27.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.843+0000 7f9ace59c700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad010c1f0 0x7f9ad01a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:27.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.844+0000 7f9ace59c700 1 -- 172.21.15.69:0/2369301515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ad0074680 msgr2=0x7f9ad019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:27.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.844+0000 7f9ace59c700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ad0074680 0x7f9ad019c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:27.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.844+0000 7f9ace59c700 1 -- 172.21.15.69:0/2369301515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac40095f0 con 0x7f9ad00730a0 2023-07-06T20:17:27.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.844+0000 7f9aced9d700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad010c1f0 0x7f9ad01a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:27.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.844+0000 7f9ace59c700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ad00730a0 0x7f9ad019bfe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000e970 tx=0x7f9ac000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:27.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.844+0000 7f9abf7fe700 1 -- 172.21.15.69:0/2369301515 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0009790 con 0x7f9ad00730a0 2023-07-06T20:17:27.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.844+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2369301515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad01a4110 con 0x7f9ad00730a0 2023-07-06T20:17:27.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.844+0000 7f9abf7fe700 1 -- 172.21.15.69:0/2369301515 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ac000c7e0 con 0x7f9ad00730a0 2023-07-06T20:17:27.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.844+0000 7f9abf7fe700 1 -- 172.21.15.69:0/2369301515 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac00054e0 con 0x7f9ad00730a0 2023-07-06T20:17:27.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.844+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2369301515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad01a4670 con 0x7f9ad00730a0 2023-07-06T20:17:27.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.845+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2369301515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad0101d40 con 0x7f9ad00730a0 2023-07-06T20:17:27.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.847+0000 7f9abf7fe700 1 -- 172.21.15.69:0/2369301515 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9ac000c950 con 0x7f9ad00730a0 2023-07-06T20:17:27.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.849+0000 7f9abf7fe700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9ab80a1150 0x7f9ab80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:27.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.849+0000 7f9abf7fe700 1 -- 172.21.15.69:0/2369301515 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f9ac0014070 con 0x7f9ad00730a0 2023-07-06T20:17:27.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.849+0000 7f9acdd9b700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9ab80a1150 0x7f9ab80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:27.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.849+0000 7f9acdd9b700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9ab80a1150 0x7f9ab80a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400b2e0 tx=0x7f9ac4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:27.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:27.852+0000 7f9abf7fe700 1 -- 172.21.15.69:0/2369301515 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9ac0096270 con 0x7f9ad00730a0 2023-07-06T20:17:27.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:27 smithi156 ceph-mon[129280]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:27.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:27 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3153417458' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-06T20:17:27.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:27.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:27.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:27.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:17:27.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:27.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:27 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3153417458' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-06T20:17:27.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:27 smithi156 ceph-mon[129280]: osdmap e80: 12 total, 12 up, 12 in 2023-07-06T20:17:27.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:27 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:17:27.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:27 smithi098 ceph-mon[128968]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:27.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:27 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3153417458' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-06T20:17:27.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:27.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:27.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:27.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:17:27.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:27.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:27 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3153417458' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-06T20:17:27.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:27 smithi098 ceph-mon[128968]: osdmap e80: 12 total, 12 up, 12 in 2023-07-06T20:17:27.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:27 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:17:28.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:27 smithi069 ceph-mon[126399]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:28.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:27 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3153417458' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-06T20:17:28.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:28.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:28.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:28.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:17:28.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:28.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:27 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3153417458' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-06T20:17:28.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:27 smithi069 ceph-mon[126399]: osdmap e80: 12 total, 12 up, 12 in 2023-07-06T20:17:28.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:27 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:17:28.143 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.142+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2369301515 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9ad002c8e0 con 0x7f9ad00730a0 2023-07-06T20:17:28.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.587+0000 7f9abf7fe700 1 -- 172.21.15.69:0/2369301515 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9ac0099890 con 0x7f9ad00730a0 2023-07-06T20:17:28.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.591+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2369301515 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9ab80a1150 msgr2=0x7f9ab80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:28.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.591+0000 7f9ad4fff700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9ab80a1150 0x7f9ab80a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400b2e0 tx=0x7f9ac4005fb0 comp rx=0 tx=0).stop 2023-07-06T20:17:28.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.591+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2369301515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ad00730a0 msgr2=0x7f9ad019bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:28.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.591+0000 7f9ad4fff700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ad00730a0 0x7f9ad019bfe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000e970 tx=0x7f9ac000ec80 comp rx=0 tx=0).stop 2023-07-06T20:17:28.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.591+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2369301515 shutdown_connections 2023-07-06T20:17:28.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.591+0000 7f9ad4fff700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9ab80a1150 0x7f9ab80a3600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:28.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.591+0000 7f9ad4fff700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad010c1f0 0x7f9ad01a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:28.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.591+0000 7f9ad4fff700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ad00730a0 0x7f9ad019bfe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.591+0000 7f9ad4fff700 1 --2- 172.21.15.69:0/2369301515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ad0074680 0x7f9ad019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.591+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2369301515 >> 172.21.15.69:0/2369301515 conn(0x7f9ad00fb260 msgr2=0x7f9ad00fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.592+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2369301515 shutdown_connections 2023-07-06T20:17:28.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.592+0000 7f9ad4fff700 1 -- 172.21.15.69:0/2369301515 wait complete. 2023-07-06T20:17:28.596 INFO:teuthology.orchestra.run.smithi069.stderr:new fs with metadata pool 2 and data pool 3 2023-07-06T20:17:28.614 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:17:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.815+0000 7f6166fff700 1 -- 172.21.15.69:0/1849204055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61600730a0 msgr2=0x7f6160073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.815+0000 7f6166fff700 1 --2- 172.21.15.69:0/1849204055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61600730a0 0x7f6160073510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6150009990 tx=0x7f6150009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.816+0000 7f6166fff700 1 -- 172.21.15.69:0/1849204055 shutdown_connections 2023-07-06T20:17:28.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.816+0000 7f6166fff700 1 --2- 172.21.15.69:0/1849204055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f616010c200 0x7f616010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:28.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.816+0000 7f6166fff700 1 --2- 172.21.15.69:0/1849204055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6160074680 0x7f6160072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:28.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.816+0000 7f6166fff700 1 --2- 172.21.15.69:0/1849204055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61600730a0 0x7f6160073510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:28.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.816+0000 7f6166fff700 1 -- 172.21.15.69:0/1849204055 >> 172.21.15.69:0/1849204055 conn(0x7f61600fb210 msgr2=0x7f61600fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:28.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.816+0000 7f6166fff700 1 -- 172.21.15.69:0/1849204055 shutdown_connections 2023-07-06T20:17:28.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.816+0000 7f6166fff700 1 -- 172.21.15.69:0/1849204055 wait complete. 2023-07-06T20:17:28.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f6166fff700 1 Processor -- start 2023-07-06T20:17:28.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f6166fff700 1 -- start start 2023-07-06T20:17:28.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f6166fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61600730a0 0x7f616019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:28.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f6166fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6160074680 0x7f616019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:28.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f6166fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f616010c200 0x7f61601a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:28.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f6166fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601a3b60 con 0x7f616010c200 2023-07-06T20:17:28.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f6166fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601a3ca0 con 0x7f6160074680 2023-07-06T20:17:28.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f6166fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601a3e30 con 0x7f61600730a0 2023-07-06T20:17:28.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f6164d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61600730a0 0x7f616019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:28.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f616559c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f616010c200 0x7f61601a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:28.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f6164d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61600730a0 0x7f616019bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:55196/0 (socket says 172.21.15.69:55196) 2023-07-06T20:17:28.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f6157fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6160074680 0x7f616019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:28.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.817+0000 7f6164d9b700 1 -- 172.21.15.69:0/1107717129 learned_addr learned my addr 172.21.15.69:0/1107717129 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:28.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.818+0000 7f6164d9b700 1 -- 172.21.15.69:0/1107717129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6160074680 msgr2=0x7f616019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.818+0000 7f6164d9b700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6160074680 0x7f616019c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.818+0000 7f6164d9b700 1 -- 172.21.15.69:0/1107717129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f616010c200 msgr2=0x7f61601a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.818+0000 7f6164d9b700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f616010c200 0x7f61601a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.818+0000 7f6164d9b700 1 -- 172.21.15.69:0/1107717129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61500095f0 con 0x7f61600730a0 2023-07-06T20:17:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.818+0000 7f616559c700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f616010c200 0x7f61601a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.818+0000 7f6164d9b700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61600730a0 0x7f616019bfe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f615c00e970 tx=0x7f615c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.818+0000 7f6155ffb700 1 -- 172.21.15.69:0/1107717129 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c009790 con 0x7f61600730a0 2023-07-06T20:17:28.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.818+0000 7f6166fff700 1 -- 172.21.15.69:0/1107717129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61601a4110 con 0x7f61600730a0 2023-07-06T20:17:28.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.818+0000 7f6155ffb700 1 -- 172.21.15.69:0/1107717129 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f615c00c7e0 con 0x7f61600730a0 2023-07-06T20:17:28.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.818+0000 7f6155ffb700 1 -- 172.21.15.69:0/1107717129 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c0054e0 con 0x7f61600730a0 2023-07-06T20:17:28.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.819+0000 7f6166fff700 1 -- 172.21.15.69:0/1107717129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61601a4620 con 0x7f61600730a0 2023-07-06T20:17:28.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.819+0000 7f6166fff700 1 -- 172.21.15.69:0/1107717129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6160101d50 con 0x7f61600730a0 2023-07-06T20:17:28.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.821+0000 7f6155ffb700 1 -- 172.21.15.69:0/1107717129 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f615c00c950 con 0x7f61600730a0 2023-07-06T20:17:28.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.823+0000 7f6155ffb700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f61480a11a0 0x7f61480a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:28.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.823+0000 7f6155ffb700 1 -- 172.21.15.69:0/1107717129 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f615c014070 con 0x7f61600730a0 2023-07-06T20:17:28.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.823+0000 7f6157fff700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f61480a11a0 0x7f61480a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:28.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.824+0000 7f6157fff700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f61480a11a0 0x7f61480a3650 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f615000b2e0 tx=0x7f6150005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:28.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:28.825+0000 7f6155ffb700 1 -- 172.21.15.69:0/1107717129 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f615c096900 con 0x7f61600730a0 2023-07-06T20:17:28.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:28 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3449139776' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-06T20:17:28.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:28 smithi156 ceph-mon[129280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-06T20:17:28.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:28 smithi156 ceph-mon[129280]: pgmap v265: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:28.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:28 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:28.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:28 smithi156 ceph-mon[129280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-06T20:17:28.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:28 smithi156 ceph-mon[129280]: osdmap e81: 12 total, 12 up, 12 in 2023-07-06T20:17:28.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:28 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2369301515' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-06T20:17:28.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:28 smithi156 ceph-mon[129280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-06T20:17:28.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:28 smithi156 ceph-mon[129280]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-06T20:17:28.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:28 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3449139776' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-06T20:17:28.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:28 smithi098 ceph-mon[128968]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-06T20:17:28.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:28 smithi098 ceph-mon[128968]: pgmap v265: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:28.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:28 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:28.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:28 smithi098 ceph-mon[128968]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-06T20:17:28.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:28 smithi098 ceph-mon[128968]: osdmap e81: 12 total, 12 up, 12 in 2023-07-06T20:17:28.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:28 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2369301515' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-06T20:17:28.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:28 smithi098 ceph-mon[128968]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-06T20:17:28.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:28 smithi098 ceph-mon[128968]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-06T20:17:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:28 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3449139776' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-06T20:17:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:28 smithi069 ceph-mon[126399]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-06T20:17:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:28 smithi069 ceph-mon[126399]: pgmap v265: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:17:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:28 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:28 smithi069 ceph-mon[126399]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-06T20:17:29.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:28 smithi069 ceph-mon[126399]: osdmap e81: 12 total, 12 up, 12 in 2023-07-06T20:17:29.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:28 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2369301515' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-06T20:17:29.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:28 smithi069 ceph-mon[126399]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-06T20:17:29.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:28 smithi069 ceph-mon[126399]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-06T20:17:29.040 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.038+0000 7f6166fff700 1 -- 172.21.15.69:0/1107717129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f616002c8e0 con 0x7f61600730a0 2023-07-06T20:17:29.041 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.040+0000 7f6155ffb700 1 -- 172.21.15.69:0/1107717129 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22078 (secure 0 0 0) 0x7f615c099f20 con 0x7f61600730a0 2023-07-06T20:17:29.042 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:29.042 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":82,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:17:28.144205+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:17:29.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.042+0000 7f6166fff700 1 -- 172.21.15.69:0/1107717129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f61480a11a0 msgr2=0x7f61480a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:29.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.042+0000 7f6166fff700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f61480a11a0 0x7f61480a3650 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f615000b2e0 tx=0x7f6150005fb0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.042+0000 7f6166fff700 1 -- 172.21.15.69:0/1107717129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61600730a0 msgr2=0x7f616019bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:29.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.042+0000 7f6166fff700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61600730a0 0x7f616019bfe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f615c00e970 tx=0x7f615c00ec80 comp rx=0 tx=0).stop 2023-07-06T20:17:29.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.042+0000 7f6166fff700 1 -- 172.21.15.69:0/1107717129 shutdown_connections 2023-07-06T20:17:29.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.042+0000 7f6166fff700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f61480a11a0 0x7f61480a3650 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.042+0000 7f6166fff700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f616010c200 0x7f61601a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.042+0000 7f6166fff700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61600730a0 0x7f616019bfe0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.042+0000 7f6166fff700 1 --2- 172.21.15.69:0/1107717129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6160074680 0x7f616019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.043+0000 7f6166fff700 1 -- 172.21.15.69:0/1107717129 >> 172.21.15.69:0/1107717129 conn(0x7f61600fb210 msgr2=0x7f61600fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:29.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.043+0000 7f6166fff700 1 -- 172.21.15.69:0/1107717129 shutdown_connections 2023-07-06T20:17:29.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.043+0000 7f6166fff700 1 -- 172.21.15.69:0/1107717129 wait complete. 2023-07-06T20:17:29.063 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:17:29.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.270+0000 7f1fdfdff700 1 -- 172.21.15.69:0/2988570613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd80ffc30 msgr2=0x7f1fd8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:29.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.270+0000 7f1fdfdff700 1 --2- 172.21.15.69:0/2988570613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd80ffc30 0x7f1fd8100040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009990 tx=0x7f1fd4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.270+0000 7f1fdfdff700 1 -- 172.21.15.69:0/2988570613 shutdown_connections 2023-07-06T20:17:29.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.270+0000 7f1fdfdff700 1 --2- 172.21.15.69:0/2988570613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd81017b0 0x7f1fd8108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.270+0000 7f1fdfdff700 1 --2- 172.21.15.69:0/2988570613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd80ffc30 0x7f1fd8100040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.270+0000 7f1fdfdff700 1 --2- 172.21.15.69:0/2988570613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fd8100e20 0x7f1fd8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.270+0000 7f1fdfdff700 1 -- 172.21.15.69:0/2988570613 >> 172.21.15.69:0/2988570613 conn(0x7f1fd80fb210 msgr2=0x7f1fd80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:29.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.270+0000 7f1fdfdff700 1 -- 172.21.15.69:0/2988570613 shutdown_connections 2023-07-06T20:17:29.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.270+0000 7f1fdfdff700 1 -- 172.21.15.69:0/2988570613 wait complete. 2023-07-06T20:17:29.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.271+0000 7f1fdfdff700 1 Processor -- start 2023-07-06T20:17:29.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.271+0000 7f1fdfdff700 1 -- start start 2023-07-06T20:17:29.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.271+0000 7f1fdfdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd80ffc30 0x7f1fd8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:29.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.271+0000 7f1fdfdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd8100e20 0x7f1fd8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:29.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.271+0000 7f1fdfdff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fd81017b0 0x7f1fd819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:29.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.271+0000 7f1fdfdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd819f670 con 0x7f1fd80ffc30 2023-07-06T20:17:29.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.271+0000 7f1fdfdff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd819f7b0 con 0x7f1fd81017b0 2023-07-06T20:17:29.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.271+0000 7f1fdfdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd819f940 con 0x7f1fd8100e20 2023-07-06T20:17:29.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.271+0000 7f1fddb9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd80ffc30 0x7f1fd8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:29.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.271+0000 7f1fddb9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd80ffc30 0x7f1fd8197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56228/0 (socket says 172.21.15.69:56228) 2023-07-06T20:17:29.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.271+0000 7f1fddb9b700 1 -- 172.21.15.69:0/660769007 learned_addr learned my addr 172.21.15.69:0/660769007 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:29.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.271+0000 7f1fdd39a700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd8100e20 0x7f1fd8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:29.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.272+0000 7f1fde39c700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fd81017b0 0x7f1fd819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:29.277 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.272+0000 7f1fddb9b700 1 -- 172.21.15.69:0/660769007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd8100e20 msgr2=0x7f1fd8198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:29.277 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.272+0000 7f1fddb9b700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd8100e20 0x7f1fd8198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.277 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.272+0000 7f1fddb9b700 1 -- 172.21.15.69:0/660769007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fd81017b0 msgr2=0x7f1fd819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:29.277 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.272+0000 7f1fddb9b700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fd81017b0 0x7f1fd819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.277 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.272+0000 7f1fddb9b700 1 -- 172.21.15.69:0/660769007 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd40095f0 con 0x7f1fd80ffc30 2023-07-06T20:17:29.277 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.272+0000 7f1fde39c700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fd81017b0 0x7f1fd819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:17:29.277 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.272+0000 7f1fddb9b700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd80ffc30 0x7f1fd8197af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4000c00 tx=0x7f1fd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:29.277 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.272+0000 7f1fceffd700 1 -- 172.21.15.69:0/660769007 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd401d070 con 0x7f1fd80ffc30 2023-07-06T20:17:29.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.272+0000 7f1fdfdff700 1 -- 172.21.15.69:0/660769007 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd819fbc0 con 0x7f1fd80ffc30 2023-07-06T20:17:29.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.272+0000 7f1fceffd700 1 -- 172.21.15.69:0/660769007 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fd4022470 con 0x7f1fd80ffc30 2023-07-06T20:17:29.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.272+0000 7f1fceffd700 1 -- 172.21.15.69:0/660769007 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4017650 con 0x7f1fd80ffc30 2023-07-06T20:17:29.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.273+0000 7f1fdfdff700 1 -- 172.21.15.69:0/660769007 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd81a0090 con 0x7f1fd80ffc30 2023-07-06T20:17:29.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.273+0000 7f1fdfdff700 1 -- 172.21.15.69:0/660769007 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd810b3a0 con 0x7f1fd80ffc30 2023-07-06T20:17:29.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.276+0000 7f1fceffd700 1 -- 172.21.15.69:0/660769007 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1fd40225e0 con 0x7f1fd80ffc30 2023-07-06T20:17:29.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.277+0000 7f1fceffd700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1fc40a11a0 0x7f1fc40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:29.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.277+0000 7f1fceffd700 1 -- 172.21.15.69:0/660769007 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1fd40ce120 con 0x7f1fd80ffc30 2023-07-06T20:17:29.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.277+0000 7f1fdd39a700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1fc40a11a0 0x7f1fc40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.278+0000 7f1fdd39a700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1fc40a11a0 0x7f1fc40a3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009c60 tx=0x7f1fc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:29.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.279+0000 7f1fceffd700 1 -- 172.21.15.69:0/660769007 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1fd4097a50 con 0x7f1fd80ffc30 2023-07-06T20:17:29.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.490+0000 7f1fdfdff700 1 -- 172.21.15.69:0/660769007 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fd81a0640 con 0x7f1fd80ffc30 2023-07-06T20:17:29.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.491+0000 7f1fceffd700 1 -- 172.21.15.69:0/660769007 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22078 (secure 0 0 0) 0x7f1fd409b070 con 0x7f1fd80ffc30 2023-07-06T20:17:29.493 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:29.494 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":82,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:17:28.144205+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:17:29.495 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.493+0000 7f1fdfdff700 1 -- 172.21.15.69:0/660769007 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1fc40a11a0 msgr2=0x7f1fc40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.493+0000 7f1fdfdff700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1fc40a11a0 0x7f1fc40a3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009c60 tx=0x7f1fc8009260 comp rx=0 tx=0).stop 2023-07-06T20:17:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.493+0000 7f1fdfdff700 1 -- 172.21.15.69:0/660769007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd80ffc30 msgr2=0x7f1fd8197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.493+0000 7f1fdfdff700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd80ffc30 0x7f1fd8197af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4000c00 tx=0x7f1fd4004730 comp rx=0 tx=0).stop 2023-07-06T20:17:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.493+0000 7f1fdfdff700 1 -- 172.21.15.69:0/660769007 shutdown_connections 2023-07-06T20:17:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.493+0000 7f1fdfdff700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1fc40a11a0 0x7f1fc40a3650 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.493+0000 7f1fdfdff700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fd81017b0 0x7f1fd819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.493+0000 7f1fdfdff700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd80ffc30 0x7f1fd8197af0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.493+0000 7f1fdfdff700 1 --2- 172.21.15.69:0/660769007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fd8100e20 0x7f1fd8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.493+0000 7f1fdfdff700 1 -- 172.21.15.69:0/660769007 >> 172.21.15.69:0/660769007 conn(0x7f1fd80fb210 msgr2=0x7f1fd80fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.494+0000 7f1fdfdff700 1 -- 172.21.15.69:0/660769007 shutdown_connections 2023-07-06T20:17:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.494+0000 7f1fdfdff700 1 -- 172.21.15.69:0/660769007 wait complete. 2023-07-06T20:17:29.514 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T20:17:29.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.727+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3794293805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d280ffc10 msgr2=0x7f4d28100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:29.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.727+0000 7f4d2e5ff700 1 --2- 172.21.15.69:0/3794293805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d280ffc10 0x7f4d28100020 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009990 tx=0x7f4d18009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.727+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3794293805 shutdown_connections 2023-07-06T20:17:29.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.727+0000 7f4d2e5ff700 1 --2- 172.21.15.69:0/3794293805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d28101790 0x7f4d28108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.727+0000 7f4d2e5ff700 1 --2- 172.21.15.69:0/3794293805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d280ffc10 0x7f4d28100020 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.727+0000 7f4d2e5ff700 1 --2- 172.21.15.69:0/3794293805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d28100e00 0x7f4d28101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.727+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3794293805 >> 172.21.15.69:0/3794293805 conn(0x7f4d280fb210 msgr2=0x7f4d280fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:29.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.727+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3794293805 shutdown_connections 2023-07-06T20:17:29.731 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.727+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3794293805 wait complete. 2023-07-06T20:17:29.732 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.728+0000 7f4d2e5ff700 1 Processor -- start 2023-07-06T20:17:29.732 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.728+0000 7f4d2e5ff700 1 -- start start 2023-07-06T20:17:29.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d2e5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d280ffc10 0x7f4d281959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:29.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d2e5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d28100e00 0x7f4d28195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:29.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d2e5ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d28101790 0x7f4d2819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:29.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d2e5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2819d530 con 0x7f4d280ffc10 2023-07-06T20:17:29.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d2e5ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2819d670 con 0x7f4d28101790 2023-07-06T20:17:29.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d2e5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2819d800 con 0x7f4d28100e00 2023-07-06T20:17:29.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d27fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d280ffc10 0x7f4d281959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d27fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d280ffc10 0x7f4d281959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56242/0 (socket says 172.21.15.69:56242) 2023-07-06T20:17:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d27fff700 1 -- 172.21.15.69:0/3689344771 learned_addr learned my addr 172.21.15.69:0/3689344771 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d277fe700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d28100e00 0x7f4d28195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d2cb9c700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d28101790 0x7f4d2819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d27fff700 1 -- 172.21.15.69:0/3689344771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d28100e00 msgr2=0x7f4d28195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d27fff700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d28100e00 0x7f4d28195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d27fff700 1 -- 172.21.15.69:0/3689344771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d28101790 msgr2=0x7f4d2819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d27fff700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d28101790 0x7f4d2819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d27fff700 1 -- 172.21.15.69:0/3689344771 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180095f0 con 0x7f4d280ffc10 2023-07-06T20:17:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.729+0000 7f4d27fff700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d280ffc10 0x7f4d281959b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005740 tx=0x7f4d18005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.730+0000 7f4d257fa700 1 -- 172.21.15.69:0/3689344771 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1801d070 con 0x7f4d280ffc10 2023-07-06T20:17:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.730+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3689344771 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2819da80 con 0x7f4d280ffc10 2023-07-06T20:17:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.730+0000 7f4d257fa700 1 -- 172.21.15.69:0/3689344771 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d18021410 con 0x7f4d280ffc10 2023-07-06T20:17:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.730+0000 7f4d257fa700 1 -- 172.21.15.69:0/3689344771 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18017650 con 0x7f4d280ffc10 2023-07-06T20:17:29.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.730+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3689344771 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2819dea0 con 0x7f4d280ffc10 2023-07-06T20:17:29.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.731+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3689344771 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d28106b40 con 0x7f4d280ffc10 2023-07-06T20:17:29.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.733+0000 7f4d257fa700 1 -- 172.21.15.69:0/3689344771 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4d18021580 con 0x7f4d280ffc10 2023-07-06T20:17:29.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.734+0000 7f4d257fa700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4d100a1150 0x7f4d100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:29.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.735+0000 7f4d257fa700 1 -- 172.21.15.69:0/3689344771 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4d180cde20 con 0x7f4d280ffc10 2023-07-06T20:17:29.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.735+0000 7f4d277fe700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4d100a1150 0x7f4d100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:29.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.735+0000 7f4d277fe700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4d100a1150 0x7f4d100a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009c60 tx=0x7f4d1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:29.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.736+0000 7f4d257fa700 1 -- 172.21.15.69:0/3689344771 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4d180976b0 con 0x7f4d280ffc10 2023-07-06T20:17:29.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:29 smithi156 ceph-mon[129280]: osdmap e82: 12 total, 12 up, 12 in 2023-07-06T20:17:29.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:29 smithi156 ceph-mon[129280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-06T20:17:29.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:29 smithi156 ceph-mon[129280]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-06T20:17:29.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:29 smithi156 ceph-mon[129280]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-06T20:17:29.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:29 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1107717129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:29.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:29 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:29.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:29 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:29.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:29 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/660769007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:29.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:29 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:29.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:29 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:29.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:29 smithi098 ceph-mon[128968]: osdmap e82: 12 total, 12 up, 12 in 2023-07-06T20:17:29.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:29 smithi098 ceph-mon[128968]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-06T20:17:29.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:29 smithi098 ceph-mon[128968]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-06T20:17:29.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:29 smithi098 ceph-mon[128968]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-06T20:17:29.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:29 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1107717129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:29.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:29 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:29.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:29 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:29.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:29 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/660769007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:29.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:29 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:29.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:29 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:29.995 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:29.993+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3689344771 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4d2802c8e0 con 0x7f4d280ffc10 2023-07-06T20:17:30.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:29 smithi069 ceph-mon[126399]: osdmap e82: 12 total, 12 up, 12 in 2023-07-06T20:17:30.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:29 smithi069 ceph-mon[126399]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-06T20:17:30.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:29 smithi069 ceph-mon[126399]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-06T20:17:30.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:29 smithi069 ceph-mon[126399]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-06T20:17:30.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:29 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1107717129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:30.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:29 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:30.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:29 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:30.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:29 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/660769007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:30.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:29 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:30.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:29 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:30.598 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.596+0000 7f4d257fa700 1 -- 172.21.15.69:0/3689344771 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4d1809acd0 con 0x7f4d280ffc10 2023-07-06T20:17:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.599+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3689344771 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4d100a1150 msgr2=0x7f4d100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.599+0000 7f4d2e5ff700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4d100a1150 0x7f4d100a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009c60 tx=0x7f4d1c009260 comp rx=0 tx=0).stop 2023-07-06T20:17:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.599+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3689344771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d280ffc10 msgr2=0x7f4d281959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.599+0000 7f4d2e5ff700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d280ffc10 0x7f4d281959b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005740 tx=0x7f4d18005770 comp rx=0 tx=0).stop 2023-07-06T20:17:30.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.599+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3689344771 shutdown_connections 2023-07-06T20:17:30.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.599+0000 7f4d2e5ff700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4d100a1150 0x7f4d100a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:30.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.599+0000 7f4d2e5ff700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d28101790 0x7f4d2819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:30.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.599+0000 7f4d2e5ff700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d280ffc10 0x7f4d281959b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:30.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.599+0000 7f4d2e5ff700 1 --2- 172.21.15.69:0/3689344771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d28100e00 0x7f4d28195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:30.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.599+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3689344771 >> 172.21.15.69:0/3689344771 conn(0x7f4d280fb210 msgr2=0x7f4d280fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:30.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.599+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3689344771 shutdown_connections 2023-07-06T20:17:30.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.599+0000 7f4d2e5ff700 1 -- 172.21.15.69:0/3689344771 wait complete. 2023-07-06T20:17:30.619 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-07-06T20:17:30.619 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-07-06T20:17:30.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.816+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/1715214252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bd4101790 msgr2=0x7f6bd4108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:30.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.816+0000 7f6bdc5ff700 1 --2- 172.21.15.69:0/1715214252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bd4101790 0x7f6bd4108050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00b0a0 tx=0x7f6bcc00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:17:30.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.816+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/1715214252 shutdown_connections 2023-07-06T20:17:30.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.816+0000 7f6bdc5ff700 1 --2- 172.21.15.69:0/1715214252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bd4101790 0x7f6bd4108050 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:30.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.816+0000 7f6bdc5ff700 1 --2- 172.21.15.69:0/1715214252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bd40ffc10 0x7f6bd4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:30.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.816+0000 7f6bdc5ff700 1 --2- 172.21.15.69:0/1715214252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bd4100e00 0x7f6bd4101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:30.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.816+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/1715214252 >> 172.21.15.69:0/1715214252 conn(0x7f6bd40fb210 msgr2=0x7f6bd40fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:30.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.816+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/1715214252 shutdown_connections 2023-07-06T20:17:30.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.816+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/1715214252 wait complete. 2023-07-06T20:17:30.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.817+0000 7f6bdc5ff700 1 Processor -- start 2023-07-06T20:17:30.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.817+0000 7f6bdc5ff700 1 -- start start 2023-07-06T20:17:30.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.817+0000 7f6bdc5ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bd40ffc10 0x7f6bd40743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:30.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.817+0000 7f6bdc5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bd4100e00 0x7f6bd4072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:30.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.817+0000 7f6bdc5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bd4101790 0x7f6bd4072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:30.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.817+0000 7f6bdc5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd4199930 con 0x7f6bd4101790 2023-07-06T20:17:30.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.817+0000 7f6bdc5ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd40734b0 con 0x7f6bd40ffc10 2023-07-06T20:17:30.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.817+0000 7f6bdc5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd4073670 con 0x7f6bd4100e00 2023-07-06T20:17:30.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.818+0000 7f6bdab9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bd4101790 0x7f6bd4072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:30.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.818+0000 7f6bd9b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bd4100e00 0x7f6bd4072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:30.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.818+0000 7f6bdab9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bd4101790 0x7f6bd4072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56264/0 (socket says 172.21.15.69:56264) 2023-07-06T20:17:30.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.818+0000 7f6bdab9c700 1 -- 172.21.15.69:0/2157710556 learned_addr learned my addr 172.21.15.69:0/2157710556 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:30.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.818+0000 7f6bda39b700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bd40ffc10 0x7f6bd40743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:30.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.818+0000 7f6bdab9c700 1 -- 172.21.15.69:0/2157710556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bd4100e00 msgr2=0x7f6bd4072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:30.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.818+0000 7f6bdab9c700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bd4100e00 0x7f6bd4072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:30.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.818+0000 7f6bdab9c700 1 -- 172.21.15.69:0/2157710556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bd40ffc10 msgr2=0x7f6bd40743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:30.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.818+0000 7f6bdab9c700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bd40ffc10 0x7f6bd40743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:30.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.818+0000 7f6bdab9c700 1 -- 172.21.15.69:0/2157710556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc009d00 con 0x7f6bd4101790 2023-07-06T20:17:30.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.818+0000 7f6bdab9c700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bd4101790 0x7f6bd4072f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00d010 tx=0x7f6bcc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:30.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.818+0000 7f6bcb7fe700 1 -- 172.21.15.69:0/2157710556 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc00e040 con 0x7f6bd4101790 2023-07-06T20:17:30.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.819+0000 7f6bcb7fe700 1 -- 172.21.15.69:0/2157710556 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bcc003e10 con 0x7f6bd4101790 2023-07-06T20:17:30.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.819+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/2157710556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd41a42e0 con 0x7f6bd4101790 2023-07-06T20:17:30.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.819+0000 7f6bcb7fe700 1 -- 172.21.15.69:0/2157710556 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0052a0 con 0x7f6bd4101790 2023-07-06T20:17:30.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.819+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/2157710556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd41a4650 con 0x7f6bd4101790 2023-07-06T20:17:30.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.820+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/2157710556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd410b440 con 0x7f6bd4101790 2023-07-06T20:17:30.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.821+0000 7f6bcb7fe700 1 -- 172.21.15.69:0/2157710556 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6bcc01d480 con 0x7f6bd4101790 2023-07-06T20:17:30.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.823+0000 7f6bcb7fe700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6bc00a1150 0x7f6bc00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:30.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.823+0000 7f6bcb7fe700 1 -- 172.21.15.69:0/2157710556 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6bcc0cdb10 con 0x7f6bd4101790 2023-07-06T20:17:30.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.823+0000 7f6bda39b700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6bc00a1150 0x7f6bc00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:30.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.823+0000 7f6bda39b700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6bc00a1150 0x7f6bc00a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4100c60 tx=0x7f6bd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:30.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:30.826+0000 7f6bcb7fe700 1 -- 172.21.15.69:0/2157710556 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6bcc097440 con 0x7f6bd4101790 2023-07-06T20:17:30.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:30 smithi156 ceph-mon[129280]: pgmap v268: 97 pgs: 21 creating+peering, 24 active+clean, 52 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2023-07-06T20:17:30.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:30 smithi156 ceph-mon[129280]: osdmap e83: 12 total, 12 up, 12 in 2023-07-06T20:17:30.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:30 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] up:active 2023-07-06T20:17:30.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:30 smithi156 ceph-mon[129280]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-06T20:17:30.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:30 smithi156 ceph-mon[129280]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-06T20:17:30.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:30 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3689344771' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-06T20:17:30.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:30 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:30.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:30 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:30.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:30 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:30.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:30 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:17:30.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:30 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:30.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:30 smithi098 ceph-mon[128968]: pgmap v268: 97 pgs: 21 creating+peering, 24 active+clean, 52 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2023-07-06T20:17:30.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:30 smithi098 ceph-mon[128968]: osdmap e83: 12 total, 12 up, 12 in 2023-07-06T20:17:30.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:30 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] up:active 2023-07-06T20:17:30.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:30 smithi098 ceph-mon[128968]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-06T20:17:30.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:30 smithi098 ceph-mon[128968]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-06T20:17:30.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:30 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3689344771' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-06T20:17:30.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:30 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:30.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:30 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:30.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:30 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:30.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:30 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:17:30.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:30 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:30 smithi069 ceph-mon[126399]: pgmap v268: 97 pgs: 21 creating+peering, 24 active+clean, 52 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2023-07-06T20:17:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:30 smithi069 ceph-mon[126399]: osdmap e83: 12 total, 12 up, 12 in 2023-07-06T20:17:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:30 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] up:active 2023-07-06T20:17:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:30 smithi069 ceph-mon[126399]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-06T20:17:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:30 smithi069 ceph-mon[126399]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-06T20:17:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:30 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3689344771' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-06T20:17:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:17:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:17:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:17:31.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.069+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/2157710556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f6bd4106a70 con 0x7f6bd4101790 2023-07-06T20:17:31.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.605+0000 7f6bcb7fe700 1 -- 172.21.15.69:0/2157710556 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f6bcc09aa60 con 0x7f6bd4101790 2023-07-06T20:17:31.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.608+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/2157710556 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6bc00a1150 msgr2=0x7f6bc00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:31.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.608+0000 7f6bdc5ff700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6bc00a1150 0x7f6bc00a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4100c60 tx=0x7f6bd0009260 comp rx=0 tx=0).stop 2023-07-06T20:17:31.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.608+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/2157710556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bd4101790 msgr2=0x7f6bd4072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:31.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.608+0000 7f6bdc5ff700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bd4101790 0x7f6bd4072f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00d010 tx=0x7f6bcc00bc80 comp rx=0 tx=0).stop 2023-07-06T20:17:31.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.608+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/2157710556 shutdown_connections 2023-07-06T20:17:31.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.608+0000 7f6bdc5ff700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6bc00a1150 0x7f6bc00a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:31.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.608+0000 7f6bdc5ff700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6bd4101790 0x7f6bd4072f70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:31.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.608+0000 7f6bdc5ff700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bd40ffc10 0x7f6bd40743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:31.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.608+0000 7f6bdc5ff700 1 --2- 172.21.15.69:0/2157710556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bd4100e00 0x7f6bd4072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:31.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.608+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/2157710556 >> 172.21.15.69:0/2157710556 conn(0x7f6bd40fb210 msgr2=0x7f6bd40fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:31.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.608+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/2157710556 shutdown_connections 2023-07-06T20:17:31.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.608+0000 7f6bdc5ff700 1 -- 172.21.15.69:0/2157710556 wait complete. 2023-07-06T20:17:31.632 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-06T20:17:31.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.829+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/4137147397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc41017a0 msgr2=0x7f3fc4108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:31.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.829+0000 7f3fcb7ff700 1 --2- 172.21.15.69:0/4137147397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc41017a0 0x7f3fc4108060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00b0a0 tx=0x7f3fbc00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:17:31.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.829+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/4137147397 shutdown_connections 2023-07-06T20:17:31.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.829+0000 7f3fcb7ff700 1 --2- 172.21.15.69:0/4137147397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc41017a0 0x7f3fc4108060 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:31.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.829+0000 7f3fcb7ff700 1 --2- 172.21.15.69:0/4137147397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fc40ffc20 0x7f3fc4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:31.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.829+0000 7f3fcb7ff700 1 --2- 172.21.15.69:0/4137147397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc4100e10 0x7f3fc4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:31.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.829+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/4137147397 >> 172.21.15.69:0/4137147397 conn(0x7f3fc40fb260 msgr2=0x7f3fc40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:31.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.829+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/4137147397 shutdown_connections 2023-07-06T20:17:31.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.830+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/4137147397 wait complete. 2023-07-06T20:17:31.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.830+0000 7f3fcb7ff700 1 Processor -- start 2023-07-06T20:17:31.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.831+0000 7f3fcb7ff700 1 -- start start 2023-07-06T20:17:31.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.831+0000 7f3fcb7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fc40ffc20 0x7f3fc4195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:31.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.831+0000 7f3fcb7ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc4100e10 0x7f3fc4195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:31.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.831+0000 7f3fcb7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc41017a0 0x7f3fc419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:31.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.831+0000 7f3fcb7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc419d610 con 0x7f3fc41017a0 2023-07-06T20:17:31.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.831+0000 7f3fcb7ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc419d750 con 0x7f3fc4100e10 2023-07-06T20:17:31.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.831+0000 7f3fcb7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc419d8e0 con 0x7f3fc40ffc20 2023-07-06T20:17:31.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.831+0000 7f3fc9d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc41017a0 0x7f3fc419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:31.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.831+0000 7f3fc959b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fc40ffc20 0x7f3fc4195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:31.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.831+0000 7f3fc9d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc41017a0 0x7f3fc419d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56290/0 (socket says 172.21.15.69:56290) 2023-07-06T20:17:31.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.831+0000 7f3fc8d9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc4100e10 0x7f3fc4195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:31.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.831+0000 7f3fc9d9c700 1 -- 172.21.15.69:0/3449282430 learned_addr learned my addr 172.21.15.69:0/3449282430 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:31.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.832+0000 7f3fc9d9c700 1 -- 172.21.15.69:0/3449282430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fc40ffc20 msgr2=0x7f3fc4195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:31.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.832+0000 7f3fc9d9c700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fc40ffc20 0x7f3fc4195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:31.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.832+0000 7f3fc9d9c700 1 -- 172.21.15.69:0/3449282430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc4100e10 msgr2=0x7f3fc4195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:31.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.832+0000 7f3fc9d9c700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc4100e10 0x7f3fc4195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:31.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.832+0000 7f3fc9d9c700 1 -- 172.21.15.69:0/3449282430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb4009520 con 0x7f3fc41017a0 2023-07-06T20:17:31.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.832+0000 7f3fc959b700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fc40ffc20 0x7f3fc4195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:31.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.832+0000 7f3fc9d9c700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc41017a0 0x7f3fc419d040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00d010 tx=0x7f3fbc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:31.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.832+0000 7f3fba7fc700 1 -- 172.21.15.69:0/3449282430 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc00e040 con 0x7f3fc41017a0 2023-07-06T20:17:31.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.832+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/3449282430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc009d00 con 0x7f3fc41017a0 2023-07-06T20:17:31.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.832+0000 7f3fba7fc700 1 -- 172.21.15.69:0/3449282430 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fbc003e10 con 0x7f3fc41017a0 2023-07-06T20:17:31.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.832+0000 7f3fba7fc700 1 -- 172.21.15.69:0/3449282430 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc0052a0 con 0x7f3fc41017a0 2023-07-06T20:17:31.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.832+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/3449282430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc419ddf0 con 0x7f3fc41017a0 2023-07-06T20:17:31.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.833+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/3449282430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc4106ab0 con 0x7f3fc41017a0 2023-07-06T20:17:31.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.835+0000 7f3fba7fc700 1 -- 172.21.15.69:0/3449282430 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3fbc01d480 con 0x7f3fc41017a0 2023-07-06T20:17:31.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.836+0000 7f3fba7fc700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fb00a11d0 0x7f3fb00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:31.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.836+0000 7f3fba7fc700 1 -- 172.21.15.69:0/3449282430 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3fbc0cd970 con 0x7f3fc41017a0 2023-07-06T20:17:31.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.837+0000 7f3fc959b700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fb00a11d0 0x7f3fb00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:31.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.837+0000 7f3fc959b700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fb00a11d0 0x7f3fb00a3680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4100c70 tx=0x7f3fc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:31.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:31.840+0000 7f3fba7fc700 1 -- 172.21.15.69:0/3449282430 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3fbc097200 con 0x7f3fc41017a0 2023-07-06T20:17:31.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:31 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3689344771' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-06T20:17:31.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:31 smithi156 ceph-mon[129280]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-06T20:17:31.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:31 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2157710556' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-06T20:17:31.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:31 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3689344771' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-06T20:17:31.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:31 smithi098 ceph-mon[128968]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-06T20:17:31.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:31 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2157710556' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-06T20:17:32.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:31 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3689344771' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-06T20:17:32.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:31 smithi069 ceph-mon[126399]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-06T20:17:32.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:31 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2157710556' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-06T20:17:32.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.086+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/3449282430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f3fc40612f0 con 0x7f3fc41017a0 2023-07-06T20:17:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.548+0000 7f3fba7fc700 1 -- 172.21.15.69:0/3449282430 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f3fbc09a820 con 0x7f3fc41017a0 2023-07-06T20:17:32.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.551+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/3449282430 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fb00a11d0 msgr2=0x7f3fb00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:32.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.551+0000 7f3fcb7ff700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fb00a11d0 0x7f3fb00a3680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4100c70 tx=0x7f3fc0009260 comp rx=0 tx=0).stop 2023-07-06T20:17:32.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.551+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/3449282430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc41017a0 msgr2=0x7f3fc419d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:32.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.551+0000 7f3fcb7ff700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc41017a0 0x7f3fc419d040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00d010 tx=0x7f3fbc00bc80 comp rx=0 tx=0).stop 2023-07-06T20:17:32.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.551+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/3449282430 shutdown_connections 2023-07-06T20:17:32.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.551+0000 7f3fcb7ff700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fb00a11d0 0x7f3fb00a3680 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:32.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.551+0000 7f3fcb7ff700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fc41017a0 0x7f3fc419d040 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:32.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.551+0000 7f3fcb7ff700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fc40ffc20 0x7f3fc4195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:32.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.551+0000 7f3fcb7ff700 1 --2- 172.21.15.69:0/3449282430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fc4100e10 0x7f3fc4195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:32.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.551+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/3449282430 >> 172.21.15.69:0/3449282430 conn(0x7f3fc40fb260 msgr2=0x7f3fc40fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:32.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.551+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/3449282430 shutdown_connections 2023-07-06T20:17:32.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.551+0000 7f3fcb7ff700 1 -- 172.21.15.69:0/3449282430 wait complete. 2023-07-06T20:17:32.576 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-06T20:17:32.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.771+0000 7fedb91ff700 1 -- 172.21.15.69:0/2299727634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4074680 msgr2=0x7fedb4072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:32.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.771+0000 7fedb91ff700 1 --2- 172.21.15.69:0/2299727634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4074680 0x7fedb4072ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feda4009990 tx=0x7feda4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:32.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.771+0000 7fedb91ff700 1 -- 172.21.15.69:0/2299727634 shutdown_connections 2023-07-06T20:17:32.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.771+0000 7fedb91ff700 1 --2- 172.21.15.69:0/2299727634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb4107ae0 0x7fedb4109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:32.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.771+0000 7fedb91ff700 1 --2- 172.21.15.69:0/2299727634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb4074680 0x7fedb4072ad0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:32.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.771+0000 7fedb91ff700 1 --2- 172.21.15.69:0/2299727634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedb40730a0 0x7fedb4073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:32.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.771+0000 7fedb91ff700 1 -- 172.21.15.69:0/2299727634 >> 172.21.15.69:0/2299727634 conn(0x7fedb40fb260 msgr2=0x7fedb40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:32.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.771+0000 7fedb91ff700 1 -- 172.21.15.69:0/2299727634 shutdown_connections 2023-07-06T20:17:32.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.771+0000 7fedb91ff700 1 -- 172.21.15.69:0/2299727634 wait complete. 2023-07-06T20:17:32.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.772+0000 7fedb91ff700 1 Processor -- start 2023-07-06T20:17:32.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.772+0000 7fedb91ff700 1 -- start start 2023-07-06T20:17:32.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb91ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb40730a0 0x7fedb4197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:32.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb91ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb4074680 0x7fedb4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:32.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb91ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedb4107ae0 0x7fedb419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:32.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb91ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb419f890 con 0x7fedb40730a0 2023-07-06T20:17:32.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb91ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb419fa00 con 0x7fedb4107ae0 2023-07-06T20:17:32.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb91ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb419fbc0 con 0x7fedb4074680 2023-07-06T20:17:32.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb2d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb40730a0 0x7fedb4197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:32.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb2d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb40730a0 0x7fedb4197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56306/0 (socket says 172.21.15.69:56306) 2023-07-06T20:17:32.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb2d9d700 1 -- 172.21.15.69:0/678088402 learned_addr learned my addr 172.21.15.69:0/678088402 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:32.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedabfff700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb4074680 0x7fedb4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:32.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb359e700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedb4107ae0 0x7fedb419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:32.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb2d9d700 1 -- 172.21.15.69:0/678088402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb4074680 msgr2=0x7fedb4198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:32.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb2d9d700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb4074680 0x7fedb4198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:32.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb2d9d700 1 -- 172.21.15.69:0/678088402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedb4107ae0 msgr2=0x7fedb419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:32.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb2d9d700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedb4107ae0 0x7fedb419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:32.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.773+0000 7fedb2d9d700 1 -- 172.21.15.69:0/678088402 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda40095f0 con 0x7fedb40730a0 2023-07-06T20:17:32.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.774+0000 7fedb2d9d700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb40730a0 0x7fedb4197d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feda4004710 tx=0x7feda4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:32.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.774+0000 7fedb0d99700 1 -- 172.21.15.69:0/678088402 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda401d070 con 0x7fedb40730a0 2023-07-06T20:17:32.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.774+0000 7fedb0d99700 1 -- 172.21.15.69:0/678088402 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda4021410 con 0x7fedb40730a0 2023-07-06T20:17:32.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.774+0000 7fedb91ff700 1 -- 172.21.15.69:0/678088402 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb419fe40 con 0x7fedb40730a0 2023-07-06T20:17:32.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.774+0000 7fedb0d99700 1 -- 172.21.15.69:0/678088402 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4017650 con 0x7fedb40730a0 2023-07-06T20:17:32.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.774+0000 7fedb91ff700 1 -- 172.21.15.69:0/678088402 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb41a0200 con 0x7fedb40730a0 2023-07-06T20:17:32.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.775+0000 7fedb91ff700 1 -- 172.21.15.69:0/678088402 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb410d650 con 0x7fedb40730a0 2023-07-06T20:17:32.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.777+0000 7fedb0d99700 1 -- 172.21.15.69:0/678088402 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feda4021580 con 0x7fedb40730a0 2023-07-06T20:17:32.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.778+0000 7fedb0d99700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feda00a1250 0x7feda00a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:32.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.778+0000 7fedb0d99700 1 -- 172.21.15.69:0/678088402 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7feda40cdc90 con 0x7fedb40730a0 2023-07-06T20:17:32.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.779+0000 7fedabfff700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feda00a1250 0x7feda00a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:32.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.779+0000 7fedabfff700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feda00a1250 0x7feda00a3700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fedb40743a0 tx=0x7fed9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:32.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:32.781+0000 7fedb0d99700 1 -- 172.21.15.69:0/678088402 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7feda4097520 con 0x7fedb40730a0 2023-07-06T20:17:32.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: pgmap v270: 97 pgs: 21 creating+peering, 60 active+clean, 16 unknown; 578 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2023-07-06T20:17:32.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-06T20:17:32.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2157710556' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-06T20:17:32.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-06T20:17:32.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-06T20:17:32.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-06T20:17:32.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-06T20:17:32.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3449282430' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-06T20:17:32.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] up:active 2023-07-06T20:17:32.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3449282430' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-06T20:17:32.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-06T20:17:32.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-06T20:17:32.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: Cluster is now healthy 2023-07-06T20:17:32.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2023-07-06T20:17:32.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2023-07-06T20:17:32.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:32 smithi156 ceph-mon[129280]: daemon mds.e is now active in filesystem cephfs as rank 2 2023-07-06T20:17:32.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: pgmap v270: 97 pgs: 21 creating+peering, 60 active+clean, 16 unknown; 578 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2023-07-06T20:17:32.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-06T20:17:32.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2157710556' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-06T20:17:32.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-06T20:17:32.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-06T20:17:32.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-06T20:17:32.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-06T20:17:32.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3449282430' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-06T20:17:32.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] up:active 2023-07-06T20:17:32.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3449282430' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-06T20:17:32.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-06T20:17:32.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-06T20:17:32.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: Cluster is now healthy 2023-07-06T20:17:32.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2023-07-06T20:17:32.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2023-07-06T20:17:32.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:32 smithi098 ceph-mon[128968]: daemon mds.e is now active in filesystem cephfs as rank 2 2023-07-06T20:17:33.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: pgmap v270: 97 pgs: 21 creating+peering, 60 active+clean, 16 unknown; 578 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2023-07-06T20:17:33.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-06T20:17:33.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2157710556' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-06T20:17:33.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-06T20:17:33.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-06T20:17:33.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-06T20:17:33.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-06T20:17:33.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3449282430' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-06T20:17:33.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] up:active 2023-07-06T20:17:33.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3449282430' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-06T20:17:33.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-06T20:17:33.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-06T20:17:33.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: Cluster is now healthy 2023-07-06T20:17:33.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2023-07-06T20:17:33.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2023-07-06T20:17:33.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:32 smithi069 ceph-mon[126399]: daemon mds.e is now active in filesystem cephfs as rank 2 2023-07-06T20:17:33.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.024+0000 7fedb91ff700 1 -- 172.21.15.69:0/678088402 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fedb410d7e0 con 0x7fedb40730a0 2023-07-06T20:17:33.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.556+0000 7fedb0d99700 1 -- 172.21.15.69:0/678088402 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7feda409ab40 con 0x7fedb40730a0 2023-07-06T20:17:33.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.559+0000 7fedb91ff700 1 -- 172.21.15.69:0/678088402 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feda00a1250 msgr2=0x7feda00a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:33.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.559+0000 7fedb91ff700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feda00a1250 0x7feda00a3700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fedb40743a0 tx=0x7fed9c009260 comp rx=0 tx=0).stop 2023-07-06T20:17:33.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.559+0000 7fedb91ff700 1 -- 172.21.15.69:0/678088402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb40730a0 msgr2=0x7fedb4197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:33.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.559+0000 7fedb91ff700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb40730a0 0x7fedb4197d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feda4004710 tx=0x7feda4009580 comp rx=0 tx=0).stop 2023-07-06T20:17:33.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.559+0000 7fedb91ff700 1 -- 172.21.15.69:0/678088402 shutdown_connections 2023-07-06T20:17:33.563 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.559+0000 7fedb91ff700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feda00a1250 0x7feda00a3700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:33.563 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.559+0000 7fedb91ff700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedb4107ae0 0x7fedb419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:33.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.559+0000 7fedb91ff700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedb40730a0 0x7fedb4197d50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:33.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.559+0000 7fedb91ff700 1 --2- 172.21.15.69:0/678088402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedb4074680 0x7fedb4198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:33.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.559+0000 7fedb91ff700 1 -- 172.21.15.69:0/678088402 >> 172.21.15.69:0/678088402 conn(0x7fedb40fb260 msgr2=0x7fedb4109da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:33.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.560+0000 7fedb91ff700 1 -- 172.21.15.69:0/678088402 shutdown_connections 2023-07-06T20:17:33.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.560+0000 7fedb91ff700 1 -- 172.21.15.69:0/678088402 wait complete. 2023-07-06T20:17:33.583 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-07-06T20:17:33.584 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-06T20:17:33.611 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:17:33 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-b[154398]: 2023-07-06T20:17:33.154+0000 7f0f9fcc0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T20:17:33.611 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:17:33 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-b[154398]: 2023-07-06T20:17:33.154+0000 7f0f9fcc0700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-07-06T20:17:33.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.785+0000 7f43a65ff700 1 -- 172.21.15.69:0/353249490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a0107ad0 msgr2=0x7f43a0109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:33.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.785+0000 7f43a65ff700 1 --2- 172.21.15.69:0/353249490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a0107ad0 0x7f43a0109f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f439800b0a0 tx=0x7f439800b3b0 comp rx=0 tx=0).stop 2023-07-06T20:17:33.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.786+0000 7f43a65ff700 1 -- 172.21.15.69:0/353249490 shutdown_connections 2023-07-06T20:17:33.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.786+0000 7f43a65ff700 1 --2- 172.21.15.69:0/353249490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a0107ad0 0x7f43a0109f30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:33.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.786+0000 7f43a65ff700 1 --2- 172.21.15.69:0/353249490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a0074600 0x7f43a0072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:33.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.786+0000 7f43a65ff700 1 --2- 172.21.15.69:0/353249490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43a0073020 0x7f43a0073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:33.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.786+0000 7f43a65ff700 1 -- 172.21.15.69:0/353249490 >> 172.21.15.69:0/353249490 conn(0x7f43a00fb260 msgr2=0x7f43a00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:33.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.786+0000 7f43a65ff700 1 -- 172.21.15.69:0/353249490 shutdown_connections 2023-07-06T20:17:33.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.786+0000 7f43a65ff700 1 -- 172.21.15.69:0/353249490 wait complete. 2023-07-06T20:17:33.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f43a65ff700 1 Processor -- start 2023-07-06T20:17:33.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f43a65ff700 1 -- start start 2023-07-06T20:17:33.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f43a65ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a0073020 0x7f43a0197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:33.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f43a65ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a0074600 0x7f43a0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:33.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f43a65ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43a0107ad0 0x7f43a019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:33.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f43a65ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a019f9d0 con 0x7f43a0073020 2023-07-06T20:17:33.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f43a65ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a019fb10 con 0x7f43a0107ad0 2023-07-06T20:17:33.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f43a65ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a019fca0 con 0x7f43a0074600 2023-07-06T20:17:33.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f439ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a0073020 0x7f43a0197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:33.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f439ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a0073020 0x7f43a0197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51816/0 (socket says 172.21.15.69:51816) 2023-07-06T20:17:33.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f439ffff700 1 -- 172.21.15.69:0/2470954708 learned_addr learned my addr 172.21.15.69:0/2470954708 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:33.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f43a4b9c700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43a0107ad0 0x7f43a019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:33.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.787+0000 7f439f7fe700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a0074600 0x7f43a0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:33.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.788+0000 7f439ffff700 1 -- 172.21.15.69:0/2470954708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a0074600 msgr2=0x7f43a0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:33.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.788+0000 7f439ffff700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a0074600 0x7f43a0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:33.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.788+0000 7f439ffff700 1 -- 172.21.15.69:0/2470954708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43a0107ad0 msgr2=0x7f43a019f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:33.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.788+0000 7f439ffff700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43a0107ad0 0x7f43a019f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:33.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.788+0000 7f439ffff700 1 -- 172.21.15.69:0/2470954708 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4398009d00 con 0x7f43a0073020 2023-07-06T20:17:33.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.788+0000 7f43a4b9c700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43a0107ad0 0x7f43a019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:33.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.788+0000 7f439ffff700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a0073020 0x7f43a0197e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f439000e9a0 tx=0x7f439000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:33.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.788+0000 7f439d7fa700 1 -- 172.21.15.69:0/2470954708 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4390009790 con 0x7f43a0073020 2023-07-06T20:17:33.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.788+0000 7f439d7fa700 1 -- 172.21.15.69:0/2470954708 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4390010430 con 0x7f43a0073020 2023-07-06T20:17:33.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.788+0000 7f43a65ff700 1 -- 172.21.15.69:0/2470954708 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a019ff80 con 0x7f43a0073020 2023-07-06T20:17:33.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.788+0000 7f439d7fa700 1 -- 172.21.15.69:0/2470954708 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4390018680 con 0x7f43a0073020 2023-07-06T20:17:33.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.788+0000 7f43a65ff700 1 -- 172.21.15.69:0/2470954708 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a01a03a0 con 0x7f43a0073020 2023-07-06T20:17:33.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.789+0000 7f43a65ff700 1 -- 172.21.15.69:0/2470954708 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a010d620 con 0x7f43a0073020 2023-07-06T20:17:33.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.791+0000 7f439d7fa700 1 -- 172.21.15.69:0/2470954708 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f43900105a0 con 0x7f43a0073020 2023-07-06T20:17:33.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.793+0000 7f439d7fa700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f43880a1150 0x7f43880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:33.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.793+0000 7f439d7fa700 1 -- 172.21.15.69:0/2470954708 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4390014070 con 0x7f43a0073020 2023-07-06T20:17:33.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.793+0000 7f439f7fe700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f43880a1150 0x7f43880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:33.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.793+0000 7f439f7fe700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f43880a1150 0x7f43880a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4394009c60 tx=0x7f4394009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:33.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:33.796+0000 7f439d7fa700 1 -- 172.21.15.69:0/2470954708 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4390051770 con 0x7f43a0073020 2023-07-06T20:17:33.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:33 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/678088402' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-06T20:17:33.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:33 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] up:active 2023-07-06T20:17:33.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:33 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/678088402' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-06T20:17:33.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:33 smithi098 ceph-mon[128968]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2023-07-06T20:17:33.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:33 smithi098 ceph-mon[128968]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:17:34.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:33 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/678088402' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-06T20:17:34.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:33 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] up:active 2023-07-06T20:17:34.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:33 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/678088402' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-06T20:17:34.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:33 smithi069 ceph-mon[126399]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2023-07-06T20:17:34.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:33 smithi069 ceph-mon[126399]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:17:34.031 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.030+0000 7f43a65ff700 1 -- 172.21.15.69:0/2470954708 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f43a00fced0 con 0x7f43880a1150 2023-07-06T20:17:34.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.078+0000 7f439d7fa700 1 -- 172.21.15.69:0/2470954708 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f43a00fced0 con 0x7f43880a1150 2023-07-06T20:17:34.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.080+0000 7f43a65ff700 1 -- 172.21.15.69:0/2470954708 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f43880a1150 msgr2=0x7f43880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:34.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.080+0000 7f43a65ff700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f43880a1150 0x7f43880a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4394009c60 tx=0x7f4394009260 comp rx=0 tx=0).stop 2023-07-06T20:17:34.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.080+0000 7f43a65ff700 1 -- 172.21.15.69:0/2470954708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a0073020 msgr2=0x7f43a0197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:34.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.080+0000 7f43a65ff700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a0073020 0x7f43a0197e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f439000e9a0 tx=0x7f439000ecb0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.080+0000 7f43a65ff700 1 -- 172.21.15.69:0/2470954708 shutdown_connections 2023-07-06T20:17:34.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.080+0000 7f43a65ff700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f43880a1150 0x7f43880a3600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.080+0000 7f43a65ff700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43a0107ad0 0x7f43a019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.080+0000 7f43a65ff700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43a0073020 0x7f43a0197e50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.080+0000 7f43a65ff700 1 --2- 172.21.15.69:0/2470954708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43a0074600 0x7f43a0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.080+0000 7f43a65ff700 1 -- 172.21.15.69:0/2470954708 >> 172.21.15.69:0/2470954708 conn(0x7f43a00fb260 msgr2=0x7f43a0109c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:34.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.081+0000 7f43a65ff700 1 -- 172.21.15.69:0/2470954708 shutdown_connections 2023-07-06T20:17:34.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.081+0000 7f43a65ff700 1 -- 172.21.15.69:0/2470954708 wait complete. 2023-07-06T20:17:34.101 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-06T20:17:34.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:33 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/678088402' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-06T20:17:34.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:33 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] up:active 2023-07-06T20:17:34.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:33 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/678088402' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-06T20:17:34.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:33 smithi156 ceph-mon[129280]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2023-07-06T20:17:34.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:33 smithi156 ceph-mon[129280]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:17:34.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.298+0000 7fc88bdff700 1 -- 172.21.15.69:0/1554344764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc884074680 msgr2=0x7fc884072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:34.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.298+0000 7fc88bdff700 1 --2- 172.21.15.69:0/1554344764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc884074680 0x7fc884072ad0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc880009990 tx=0x7fc880009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.298+0000 7fc88bdff700 1 -- 172.21.15.69:0/1554344764 shutdown_connections 2023-07-06T20:17:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.298+0000 7fc88bdff700 1 --2- 172.21.15.69:0/1554344764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc884107ae0 0x7fc884109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.298+0000 7fc88bdff700 1 --2- 172.21.15.69:0/1554344764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc884074680 0x7fc884072ad0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.298+0000 7fc88bdff700 1 --2- 172.21.15.69:0/1554344764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8840730a0 0x7fc884073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.298+0000 7fc88bdff700 1 -- 172.21.15.69:0/1554344764 >> 172.21.15.69:0/1554344764 conn(0x7fc8840fb260 msgr2=0x7fc8840fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.298+0000 7fc88bdff700 1 -- 172.21.15.69:0/1554344764 shutdown_connections 2023-07-06T20:17:34.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.298+0000 7fc88bdff700 1 -- 172.21.15.69:0/1554344764 wait complete. 2023-07-06T20:17:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.299+0000 7fc88bdff700 1 Processor -- start 2023-07-06T20:17:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.300+0000 7fc88bdff700 1 -- start start 2023-07-06T20:17:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.300+0000 7fc88bdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8840730a0 0x7fc884197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:34.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.300+0000 7fc88bdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc884074680 0x7fc884198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.300+0000 7fc88bdff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc884107ae0 0x7fc88419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.300+0000 7fc88bdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88419f900 con 0x7fc884074680 2023-07-06T20:17:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.300+0000 7fc88bdff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88419fa70 con 0x7fc884107ae0 2023-07-06T20:17:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.300+0000 7fc88bdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88419fc30 con 0x7fc8840730a0 2023-07-06T20:17:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.300+0000 7fc88939a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc884074680 0x7fc884198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.300+0000 7fc88939a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc884074680 0x7fc884198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51854/0 (socket says 172.21.15.69:51854) 2023-07-06T20:17:34.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.300+0000 7fc88939a700 1 -- 172.21.15.69:0/4016445219 learned_addr learned my addr 172.21.15.69:0/4016445219 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.300+0000 7fc889b9b700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8840730a0 0x7fc884197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.300+0000 7fc88a39c700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc884107ae0 0x7fc88419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.301+0000 7fc88939a700 1 -- 172.21.15.69:0/4016445219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8840730a0 msgr2=0x7fc884197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.301+0000 7fc88939a700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8840730a0 0x7fc884197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.301+0000 7fc88939a700 1 -- 172.21.15.69:0/4016445219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc884107ae0 msgr2=0x7fc88419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.301+0000 7fc88939a700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc884107ae0 0x7fc88419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.301+0000 7fc88939a700 1 -- 172.21.15.69:0/4016445219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8800095f0 con 0x7fc884074680 2023-07-06T20:17:34.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.301+0000 7fc88939a700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc884074680 0x7fc884198300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc87400d980 tx=0x7fc87400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:34.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.301+0000 7fc87affd700 1 -- 172.21.15.69:0/4016445219 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87400f890 con 0x7fc884074680 2023-07-06T20:17:34.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.301+0000 7fc87affd700 1 -- 172.21.15.69:0/4016445219 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc874010460 con 0x7fc884074680 2023-07-06T20:17:34.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.301+0000 7fc88bdff700 1 -- 172.21.15.69:0/4016445219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88419ff10 con 0x7fc884074680 2023-07-06T20:17:34.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.301+0000 7fc87affd700 1 -- 172.21.15.69:0/4016445219 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87400e610 con 0x7fc884074680 2023-07-06T20:17:34.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.301+0000 7fc88bdff700 1 -- 172.21.15.69:0/4016445219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8841a0330 con 0x7fc884074680 2023-07-06T20:17:34.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.302+0000 7fc88bdff700 1 -- 172.21.15.69:0/4016445219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88410d650 con 0x7fc884074680 2023-07-06T20:17:34.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.304+0000 7fc87affd700 1 -- 172.21.15.69:0/4016445219 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc8740108d0 con 0x7fc884074680 2023-07-06T20:17:34.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.305+0000 7fc87affd700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc8700a1150 0x7fc8700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:34.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.306+0000 7fc87affd700 1 -- 172.21.15.69:0/4016445219 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc8740cc770 con 0x7fc884074680 2023-07-06T20:17:34.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.306+0000 7fc889b9b700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc8700a1150 0x7fc8700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:34.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.306+0000 7fc889b9b700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc8700a1150 0x7fc8700a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc884199290 tx=0x7fc880005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:34.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.309+0000 7fc87affd700 1 -- 172.21.15.69:0/4016445219 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc874096010 con 0x7fc884074680 2023-07-06T20:17:34.509 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:17:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:17:34.036+0000 7f3cdc147700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-06T20:17:34.509 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:17:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:17:34.036+0000 7f3cdc147700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-06T20:17:34.509 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:17:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:17:34.036+0000 7f3cdc147700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-06T20:17:34.509 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:17:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:17:34.036+0000 7f3cdc147700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-06T20:17:34.510 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:17:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:17:34.036+0000 7f3cdc147700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-06T20:17:34.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.544+0000 7fc88bdff700 1 -- 172.21.15.69:0/4016445219 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8840fced0 con 0x7fc8700a1150 2023-07-06T20:17:34.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.548+0000 7fc87affd700 1 -- 172.21.15.69:0/4016445219 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fc8840fced0 con 0x7fc8700a1150 2023-07-06T20:17:34.550 INFO:teuthology.orchestra.run.smithi069.stdout:/volumes/_nogroup/sv_0/4e380ca5-2026-4fd5-8fd5-5714a5d722b6 2023-07-06T20:17:34.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.550+0000 7fc88bdff700 1 -- 172.21.15.69:0/4016445219 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc8700a1150 msgr2=0x7fc8700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:34.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.550+0000 7fc88bdff700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc8700a1150 0x7fc8700a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc884199290 tx=0x7fc880005fd0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.551+0000 7fc88bdff700 1 -- 172.21.15.69:0/4016445219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc884074680 msgr2=0x7fc884198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:34.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.551+0000 7fc88bdff700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc884074680 0x7fc884198300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc87400d980 tx=0x7fc87400dc90 comp rx=0 tx=0).stop 2023-07-06T20:17:34.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.551+0000 7fc88bdff700 1 -- 172.21.15.69:0/4016445219 shutdown_connections 2023-07-06T20:17:34.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.551+0000 7fc88bdff700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc8700a1150 0x7fc8700a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.551+0000 7fc88bdff700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc884107ae0 0x7fc88419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.551+0000 7fc88bdff700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8840730a0 0x7fc884197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.551+0000 7fc88bdff700 1 --2- 172.21.15.69:0/4016445219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc884074680 0x7fc884198300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.551+0000 7fc88bdff700 1 -- 172.21.15.69:0/4016445219 >> 172.21.15.69:0/4016445219 conn(0x7fc8840fb260 msgr2=0x7fc884109db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:34.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.551+0000 7fc88bdff700 1 -- 172.21.15.69:0/4016445219 shutdown_connections 2023-07-06T20:17:34.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.551+0000 7fc88bdff700 1 -- 172.21.15.69:0/4016445219 wait complete. 2023-07-06T20:17:34.572 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-06T20:17:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.770+0000 7f518c7ff700 1 -- 172.21.15.69:0/1508314672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5184102fb0 msgr2=0x7f5184105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.770+0000 7f518c7ff700 1 --2- 172.21.15.69:0/1508314672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5184102fb0 0x7f5184105390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5174009990 tx=0x7f5174009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.770+0000 7f518c7ff700 1 -- 172.21.15.69:0/1508314672 shutdown_connections 2023-07-06T20:17:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.770+0000 7f518c7ff700 1 --2- 172.21.15.69:0/1508314672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51841058d0 0x7f5184107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.770+0000 7f518c7ff700 1 --2- 172.21.15.69:0/1508314672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5184100690 0x7f5184102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.770+0000 7f518c7ff700 1 --2- 172.21.15.69:0/1508314672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5184102fb0 0x7f5184105390 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.770+0000 7f518c7ff700 1 -- 172.21.15.69:0/1508314672 >> 172.21.15.69:0/1508314672 conn(0x7f51840fa130 msgr2=0x7f51840fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.770+0000 7f518c7ff700 1 -- 172.21.15.69:0/1508314672 shutdown_connections 2023-07-06T20:17:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.770+0000 7f518c7ff700 1 -- 172.21.15.69:0/1508314672 wait complete. 2023-07-06T20:17:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.771+0000 7f518c7ff700 1 Processor -- start 2023-07-06T20:17:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.771+0000 7f518c7ff700 1 -- start start 2023-07-06T20:17:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.771+0000 7f518c7ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5184100690 0x7f5184100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.771+0000 7f518c7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5184102fb0 0x7f51840fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.771+0000 7f518c7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51841058d0 0x7f51840feb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.771+0000 7f518c7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51840ff320 con 0x7f5184102fb0 2023-07-06T20:17:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f518c7ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51840ff490 con 0x7f5184100690 2023-07-06T20:17:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f518c7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518419f9c0 con 0x7f51841058d0 2023-07-06T20:17:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f5189d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5184102fb0 0x7f51840fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f5189d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5184102fb0 0x7f51840fe640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51862/0 (socket says 172.21.15.69:51862) 2023-07-06T20:17:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f5189d9a700 1 -- 172.21.15.69:0/4144706263 learned_addr learned my addr 172.21.15.69:0/4144706263 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f518a59b700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5184100690 0x7f5184100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f518ad9c700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51841058d0 0x7f51840feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f5189d9a700 1 -- 172.21.15.69:0/4144706263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51841058d0 msgr2=0x7f51840feb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f5189d9a700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51841058d0 0x7f51840feb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f5189d9a700 1 -- 172.21.15.69:0/4144706263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5184100690 msgr2=0x7f5184100000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f5189d9a700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5184100690 0x7f5184100000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f5189d9a700 1 -- 172.21.15.69:0/4144706263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51740095f0 con 0x7f5184102fb0 2023-07-06T20:17:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f518ad9c700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51841058d0 0x7f51840feb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:17:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.772+0000 7f5189d9a700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5184102fb0 0x7f51840fe640 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f517400b680 tx=0x7f5174005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.773+0000 7f517b7fe700 1 -- 172.21.15.69:0/4144706263 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517401d070 con 0x7f5184102fb0 2023-07-06T20:17:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.773+0000 7f517b7fe700 1 -- 172.21.15.69:0/4144706263 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5174022470 con 0x7f5184102fb0 2023-07-06T20:17:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.773+0000 7f517b7fe700 1 -- 172.21.15.69:0/4144706263 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174017650 con 0x7f5184102fb0 2023-07-06T20:17:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.773+0000 7f518c7ff700 1 -- 172.21.15.69:0/4144706263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518419fc40 con 0x7f5184102fb0 2023-07-06T20:17:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.773+0000 7f518c7ff700 1 -- 172.21.15.69:0/4144706263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51841a0010 con 0x7f5184102fb0 2023-07-06T20:17:34.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.774+0000 7f518c7ff700 1 -- 172.21.15.69:0/4144706263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51840fbca0 con 0x7f5184102fb0 2023-07-06T20:17:34.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.776+0000 7f517b7fe700 1 -- 172.21.15.69:0/4144706263 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f51740225e0 con 0x7f5184102fb0 2023-07-06T20:17:34.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.777+0000 7f517b7fe700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f51700a1250 0x7f51700a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:34.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.777+0000 7f517b7fe700 1 -- 172.21.15.69:0/4144706263 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f51740ce3b0 con 0x7f5184102fb0 2023-07-06T20:17:34.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.777+0000 7f518a59b700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f51700a1250 0x7f51700a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:34.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.778+0000 7f518a59b700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f51700a1250 0x7f51700a3700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5180009c60 tx=0x7f5180009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:34.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:34.779+0000 7f517b7fe700 1 -- 172.21.15.69:0/4144706263 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5174097c40 con 0x7f5184102fb0 2023-07-06T20:17:34.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:34 smithi098 ceph-mon[128968]: pgmap v271: 97 pgs: 97 active+clean; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 5.3 KiB/s wr, 12 op/s 2023-07-06T20:17:34.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:34 smithi098 ceph-mon[128968]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:17:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:34 smithi069 ceph-mon[126399]: pgmap v271: 97 pgs: 97 active+clean; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 5.3 KiB/s wr, 12 op/s 2023-07-06T20:17:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:34 smithi069 ceph-mon[126399]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:17:35.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.010+0000 7f518c7ff700 1 -- 172.21.15.69:0/4144706263 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f5184060b10 con 0x7f51700a1250 2023-07-06T20:17:35.037 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.035+0000 7f517b7fe700 1 -- 172.21.15.69:0/4144706263 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5184060b10 con 0x7f51700a1250 2023-07-06T20:17:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.037+0000 7f518c7ff700 1 -- 172.21.15.69:0/4144706263 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f51700a1250 msgr2=0x7f51700a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.037+0000 7f518c7ff700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f51700a1250 0x7f51700a3700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5180009c60 tx=0x7f5180009260 comp rx=0 tx=0).stop 2023-07-06T20:17:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.037+0000 7f518c7ff700 1 -- 172.21.15.69:0/4144706263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5184102fb0 msgr2=0x7f51840fe640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.037+0000 7f518c7ff700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5184102fb0 0x7f51840fe640 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f517400b680 tx=0x7f5174005740 comp rx=0 tx=0).stop 2023-07-06T20:17:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.038+0000 7f518c7ff700 1 -- 172.21.15.69:0/4144706263 shutdown_connections 2023-07-06T20:17:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.038+0000 7f518c7ff700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f51700a1250 0x7f51700a3700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.038+0000 7f518c7ff700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51841058d0 0x7f51840feb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.038+0000 7f518c7ff700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5184100690 0x7f5184100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.038+0000 7f518c7ff700 1 --2- 172.21.15.69:0/4144706263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5184102fb0 0x7f51840fe640 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.038+0000 7f518c7ff700 1 -- 172.21.15.69:0/4144706263 >> 172.21.15.69:0/4144706263 conn(0x7f51840fa130 msgr2=0x7f51841012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.038+0000 7f518c7ff700 1 -- 172.21.15.69:0/4144706263 shutdown_connections 2023-07-06T20:17:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.038+0000 7f518c7ff700 1 -- 172.21.15.69:0/4144706263 wait complete. 2023-07-06T20:17:35.058 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-06T20:17:35.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:34 smithi156 ceph-mon[129280]: pgmap v271: 97 pgs: 97 active+clean; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 5.3 KiB/s wr, 12 op/s 2023-07-06T20:17:35.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:34 smithi156 ceph-mon[129280]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:17:35.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.256+0000 7ff1af5ff700 1 -- 172.21.15.69:0/1454087028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1a8100690 msgr2=0x7ff1a8102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:35.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.256+0000 7ff1af5ff700 1 --2- 172.21.15.69:0/1454087028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1a8100690 0x7ff1a8102a70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009990 tx=0x7ff1a4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.256+0000 7ff1af5ff700 1 -- 172.21.15.69:0/1454087028 shutdown_connections 2023-07-06T20:17:35.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.256+0000 7ff1af5ff700 1 --2- 172.21.15.69:0/1454087028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a81058d0 0x7ff1a8107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.256+0000 7ff1af5ff700 1 --2- 172.21.15.69:0/1454087028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1a8100690 0x7ff1a8102a70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.256+0000 7ff1af5ff700 1 --2- 172.21.15.69:0/1454087028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a8102fb0 0x7ff1a8105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.256+0000 7ff1af5ff700 1 -- 172.21.15.69:0/1454087028 >> 172.21.15.69:0/1454087028 conn(0x7ff1a80fa130 msgr2=0x7ff1a80fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:35.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.256+0000 7ff1af5ff700 1 -- 172.21.15.69:0/1454087028 shutdown_connections 2023-07-06T20:17:35.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.256+0000 7ff1af5ff700 1 -- 172.21.15.69:0/1454087028 wait complete. 2023-07-06T20:17:35.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.257+0000 7ff1af5ff700 1 Processor -- start 2023-07-06T20:17:35.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.258+0000 7ff1af5ff700 1 -- start start 2023-07-06T20:17:35.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.258+0000 7ff1af5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a8100690 0x7ff1a8195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:35.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.258+0000 7ff1af5ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a8102fb0 0x7ff1a8195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:35.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.258+0000 7ff1af5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1a81058d0 0x7ff1a819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:35.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.258+0000 7ff1af5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a819d500 con 0x7ff1a81058d0 2023-07-06T20:17:35.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.258+0000 7ff1af5ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a819d640 con 0x7ff1a8102fb0 2023-07-06T20:17:35.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.258+0000 7ff1af5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a819d7d0 con 0x7ff1a8100690 2023-07-06T20:17:35.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.258+0000 7ff1acb9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a8102fb0 0x7ff1a8195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:35.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.258+0000 7ff1acb9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a8102fb0 0x7ff1a8195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:56790/0 (socket says 172.21.15.69:56790) 2023-07-06T20:17:35.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.258+0000 7ff1ad39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a8100690 0x7ff1a8195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:35.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.258+0000 7ff1acb9a700 1 -- 172.21.15.69:0/2152468062 learned_addr learned my addr 172.21.15.69:0/2152468062 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:35.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.258+0000 7ff1adb9c700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1a81058d0 0x7ff1a819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:35.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.259+0000 7ff1ad39b700 1 -- 172.21.15.69:0/2152468062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a8102fb0 msgr2=0x7ff1a8195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:35.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.259+0000 7ff1ad39b700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a8102fb0 0x7ff1a8195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.259+0000 7ff1ad39b700 1 -- 172.21.15.69:0/2152468062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1a81058d0 msgr2=0x7ff1a819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:35.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.259+0000 7ff1ad39b700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1a81058d0 0x7ff1a819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.259+0000 7ff1ad39b700 1 -- 172.21.15.69:0/2152468062 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a40095f0 con 0x7ff1a8100690 2023-07-06T20:17:35.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.259+0000 7ff1adb9c700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1a81058d0 0x7ff1a819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:35.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.259+0000 7ff1ad39b700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a8100690 0x7ff1a8195980 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400b680 tx=0x7ff1a4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:35.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.259+0000 7ff19e7fc700 1 -- 172.21.15.69:0/2152468062 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a401d070 con 0x7ff1a8100690 2023-07-06T20:17:35.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.259+0000 7ff1af5ff700 1 -- 172.21.15.69:0/2152468062 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a819da50 con 0x7ff1a8100690 2023-07-06T20:17:35.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.259+0000 7ff19e7fc700 1 -- 172.21.15.69:0/2152468062 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1a4021410 con 0x7ff1a8100690 2023-07-06T20:17:35.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.259+0000 7ff19e7fc700 1 -- 172.21.15.69:0/2152468062 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4017650 con 0x7ff1a8100690 2023-07-06T20:17:35.265 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.260+0000 7ff1af5ff700 1 -- 172.21.15.69:0/2152468062 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a819ded0 con 0x7ff1a8100690 2023-07-06T20:17:35.265 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.260+0000 7ff1af5ff700 1 -- 172.21.15.69:0/2152468062 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a810b410 con 0x7ff1a8100690 2023-07-06T20:17:35.265 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.263+0000 7ff19e7fc700 1 -- 172.21.15.69:0/2152468062 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff1a4021810 con 0x7ff1a8100690 2023-07-06T20:17:35.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.264+0000 7ff19e7fc700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff1940a1150 0x7ff1940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:35.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.264+0000 7ff19e7fc700 1 -- 172.21.15.69:0/2152468062 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff1a40ce240 con 0x7ff1a8100690 2023-07-06T20:17:35.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.264+0000 7ff1acb9a700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff1940a1150 0x7ff1940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:35.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.265+0000 7ff1acb9a700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff1940a1150 0x7ff1940a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff198009d20 tx=0x7ff198009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:35.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.266+0000 7ff19e7fc700 1 -- 172.21.15.69:0/2152468062 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff1a4097b70 con 0x7ff1a8100690 2023-07-06T20:17:35.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.497+0000 7ff1af5ff700 1 -- 172.21.15.69:0/2152468062 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1a8102250 con 0x7ff1940a1150 2023-07-06T20:17:35.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.501+0000 7ff19e7fc700 1 -- 172.21.15.69:0/2152468062 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7ff1a8102250 con 0x7ff1940a1150 2023-07-06T20:17:35.503 INFO:teuthology.orchestra.run.smithi069.stdout:/volumes/_nogroup/sv_1/439e7144-7b1d-4988-b061-fb50b19330d0 2023-07-06T20:17:35.507 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.503+0000 7ff1af5ff700 1 -- 172.21.15.69:0/2152468062 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff1940a1150 msgr2=0x7ff1940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.503+0000 7ff1af5ff700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff1940a1150 0x7ff1940a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff198009d20 tx=0x7ff198009260 comp rx=0 tx=0).stop 2023-07-06T20:17:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.503+0000 7ff1af5ff700 1 -- 172.21.15.69:0/2152468062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a8100690 msgr2=0x7ff1a8195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.503+0000 7ff1af5ff700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a8100690 0x7ff1a8195980 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400b680 tx=0x7ff1a4009580 comp rx=0 tx=0).stop 2023-07-06T20:17:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.503+0000 7ff1af5ff700 1 -- 172.21.15.69:0/2152468062 shutdown_connections 2023-07-06T20:17:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.503+0000 7ff1af5ff700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff1940a1150 0x7ff1940a3600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.503+0000 7ff1af5ff700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1a81058d0 0x7ff1a819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.503+0000 7ff1af5ff700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1a8100690 0x7ff1a8195980 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.503+0000 7ff1af5ff700 1 --2- 172.21.15.69:0/2152468062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a8102fb0 0x7ff1a8195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.503+0000 7ff1af5ff700 1 -- 172.21.15.69:0/2152468062 >> 172.21.15.69:0/2152468062 conn(0x7ff1a80fa130 msgr2=0x7ff1a8101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:35.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.504+0000 7ff1af5ff700 1 -- 172.21.15.69:0/2152468062 shutdown_connections 2023-07-06T20:17:35.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.504+0000 7ff1af5ff700 1 -- 172.21.15.69:0/2152468062 wait complete. 2023-07-06T20:17:35.524 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:17:35.732 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.729+0000 7fa3815ff700 1 -- 172.21.15.69:0/65582918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa37c0730a0 msgr2=0x7fa37c073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:35.732 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.729+0000 7fa3815ff700 1 --2- 172.21.15.69:0/65582918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa37c0730a0 0x7fa37c073510 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa370009990 tx=0x7fa370009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.732 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.729+0000 7fa3815ff700 1 -- 172.21.15.69:0/65582918 shutdown_connections 2023-07-06T20:17:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.729+0000 7fa3815ff700 1 --2- 172.21.15.69:0/65582918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37c107af0 0x7fa37c109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.729+0000 7fa3815ff700 1 --2- 172.21.15.69:0/65582918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa37c074680 0x7fa37c072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.729+0000 7fa3815ff700 1 --2- 172.21.15.69:0/65582918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa37c0730a0 0x7fa37c073510 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.729+0000 7fa3815ff700 1 -- 172.21.15.69:0/65582918 >> 172.21.15.69:0/65582918 conn(0x7fa37c0fb210 msgr2=0x7fa37c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.729+0000 7fa3815ff700 1 -- 172.21.15.69:0/65582918 shutdown_connections 2023-07-06T20:17:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.729+0000 7fa3815ff700 1 -- 172.21.15.69:0/65582918 wait complete. 2023-07-06T20:17:35.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.730+0000 7fa3815ff700 1 Processor -- start 2023-07-06T20:17:35.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.730+0000 7fa3815ff700 1 -- start start 2023-07-06T20:17:35.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa3815ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa37c0730a0 0x7fa37c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:35.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa3815ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37c074680 0x7fa37c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:35.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa3815ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa37c107af0 0x7fa37c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa3815ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c19f700 con 0x7fa37c0730a0 2023-07-06T20:17:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa3815ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c19f840 con 0x7fa37c074680 2023-07-06T20:17:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa3815ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c19f9d0 con 0x7fa37c107af0 2023-07-06T20:17:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa37affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa37c0730a0 0x7fa37c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa37affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa37c0730a0 0x7fa37c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51904/0 (socket says 172.21.15.69:51904) 2023-07-06T20:17:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa37affd700 1 -- 172.21.15.69:0/2964499470 learned_addr learned my addr 172.21.15.69:0/2964499470 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:35.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa37b7fe700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa37c107af0 0x7fa37c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa37a7fc700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37c074680 0x7fa37c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa37affd700 1 -- 172.21.15.69:0/2964499470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa37c107af0 msgr2=0x7fa37c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa37affd700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa37c107af0 0x7fa37c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa37affd700 1 -- 172.21.15.69:0/2964499470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37c074680 msgr2=0x7fa37c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa37affd700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37c074680 0x7fa37c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.731+0000 7fa37affd700 1 -- 172.21.15.69:0/2964499470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3700095f0 con 0x7fa37c0730a0 2023-07-06T20:17:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.732+0000 7fa37affd700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa37c0730a0 0x7fa37c197b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00e970 tx=0x7fa36c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:35.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.732+0000 7fa363fff700 1 -- 172.21.15.69:0/2964499470 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c009790 con 0x7fa37c0730a0 2023-07-06T20:17:35.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.732+0000 7fa363fff700 1 -- 172.21.15.69:0/2964499470 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa36c00c7e0 con 0x7fa37c0730a0 2023-07-06T20:17:35.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.732+0000 7fa3815ff700 1 -- 172.21.15.69:0/2964499470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37c19fcb0 con 0x7fa37c0730a0 2023-07-06T20:17:35.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.732+0000 7fa363fff700 1 -- 172.21.15.69:0/2964499470 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c0054e0 con 0x7fa37c0730a0 2023-07-06T20:17:35.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.732+0000 7fa3815ff700 1 -- 172.21.15.69:0/2964499470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37c1a01c0 con 0x7fa37c0730a0 2023-07-06T20:17:35.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.733+0000 7fa3815ff700 1 -- 172.21.15.69:0/2964499470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37c1061a0 con 0x7fa37c0730a0 2023-07-06T20:17:35.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.735+0000 7fa363fff700 1 -- 172.21.15.69:0/2964499470 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa36c00c950 con 0x7fa37c0730a0 2023-07-06T20:17:35.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.737+0000 7fa363fff700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa3640a12a0 0x7fa3640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:35.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.738+0000 7fa363fff700 1 -- 172.21.15.69:0/2964499470 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa36c014070 con 0x7fa37c0730a0 2023-07-06T20:17:35.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.738+0000 7fa37a7fc700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa3640a12a0 0x7fa3640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:35.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.738+0000 7fa37a7fc700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa3640a12a0 0x7fa3640a3750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa37000b2e0 tx=0x7fa370005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:35.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.741+0000 7fa363fff700 1 -- 172.21.15.69:0/2964499470 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa36c096270 con 0x7fa37c0730a0 2023-07-06T20:17:35.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:35 smithi098 ceph-mon[128968]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:17:35.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:35 smithi098 ceph-mon[128968]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:17:35.982 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.980+0000 7fa3815ff700 1 -- 172.21.15.69:0/2964499470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa37c02c8e0 con 0x7fa37c0730a0 2023-07-06T20:17:35.983 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.981+0000 7fa363fff700 1 -- 172.21.15.69:0/2964499470 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10795 (secure 0 0 0) 0x7fa36c099890 con 0x7fa37c0730a0 2023-07-06T20:17:35.983 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:35.983 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-06T20:17:28.144582+0000","modified":"2023-07-06T20:17:33.557511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24539,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/3235525224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3235525224},{"type":"v1","addr":"172.21.15.98:6839","nonce":3235525224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:17:35.985 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.983+0000 7fa3815ff700 1 -- 172.21.15.69:0/2964499470 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa3640a12a0 msgr2=0x7fa3640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:35.985 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.983+0000 7fa3815ff700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa3640a12a0 0x7fa3640a3750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa37000b2e0 tx=0x7fa370005fb0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.985 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.983+0000 7fa3815ff700 1 -- 172.21.15.69:0/2964499470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa37c0730a0 msgr2=0x7fa37c197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:35.985 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.983+0000 7fa3815ff700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa37c0730a0 0x7fa37c197b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00e970 tx=0x7fa36c00ec80 comp rx=0 tx=0).stop 2023-07-06T20:17:35.986 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.983+0000 7fa3815ff700 1 -- 172.21.15.69:0/2964499470 shutdown_connections 2023-07-06T20:17:35.986 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.983+0000 7fa3815ff700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa3640a12a0 0x7fa3640a3750 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.986 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.983+0000 7fa3815ff700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa37c107af0 0x7fa37c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.986 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.983+0000 7fa3815ff700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa37c0730a0 0x7fa37c197b80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.986 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.983+0000 7fa3815ff700 1 --2- 172.21.15.69:0/2964499470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37c074680 0x7fa37c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:35.986 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.983+0000 7fa3815ff700 1 -- 172.21.15.69:0/2964499470 >> 172.21.15.69:0/2964499470 conn(0x7fa37c0fb210 msgr2=0x7fa37c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:35.986 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.983+0000 7fa3815ff700 1 -- 172.21.15.69:0/2964499470 shutdown_connections 2023-07-06T20:17:35.987 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:35.983+0000 7fa3815ff700 1 -- 172.21.15.69:0/2964499470 wait complete. 2023-07-06T20:17:35.987 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2023-07-06T20:17:36.001 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:35 smithi069 ceph-mon[126399]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:17:36.002 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:35 smithi069 ceph-mon[126399]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:17:36.004 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:17:36.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:35 smithi156 ceph-mon[129280]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:17:36.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:35 smithi156 ceph-mon[129280]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:17:36.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.196+0000 7f7aebdff700 1 -- 172.21.15.69:0/1432406681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ae41017b0 msgr2=0x7f7ae4108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:36.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.196+0000 7f7aebdff700 1 --2- 172.21.15.69:0/1432406681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ae41017b0 0x7f7ae4108070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009f90 tx=0x7f7adc00b330 comp rx=0 tx=0).stop 2023-07-06T20:17:36.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.196+0000 7f7aebdff700 1 -- 172.21.15.69:0/1432406681 shutdown_connections 2023-07-06T20:17:36.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.196+0000 7f7aebdff700 1 --2- 172.21.15.69:0/1432406681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ae41017b0 0x7f7ae4108070 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.196+0000 7f7aebdff700 1 --2- 172.21.15.69:0/1432406681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae40ffc30 0x7f7ae4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.196+0000 7f7aebdff700 1 --2- 172.21.15.69:0/1432406681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae4100e20 0x7f7ae4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.196+0000 7f7aebdff700 1 -- 172.21.15.69:0/1432406681 >> 172.21.15.69:0/1432406681 conn(0x7f7ae40fb210 msgr2=0x7f7ae40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:36.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.197+0000 7f7aebdff700 1 -- 172.21.15.69:0/1432406681 shutdown_connections 2023-07-06T20:17:36.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.197+0000 7f7aebdff700 1 -- 172.21.15.69:0/1432406681 wait complete. 2023-07-06T20:17:36.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7aebdff700 1 Processor -- start 2023-07-06T20:17:36.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7aebdff700 1 -- start start 2023-07-06T20:17:36.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7aebdff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae40ffc30 0x7f7ae4197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:36.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7aebdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ae4100e20 0x7f7ae4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:36.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7aebdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae41017b0 0x7f7ae419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:36.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7aebdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae419f770 con 0x7f7ae4100e20 2023-07-06T20:17:36.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7aebdff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae419f8e0 con 0x7f7ae40ffc30 2023-07-06T20:17:36.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7aebdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae419faa0 con 0x7f7ae41017b0 2023-07-06T20:17:36.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7ae9b9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae40ffc30 0x7f7ae4197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:36.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7ae9b9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae40ffc30 0x7f7ae4197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:56820/0 (socket says 172.21.15.69:56820) 2023-07-06T20:17:36.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7ae9b9b700 1 -- 172.21.15.69:0/4199872128 learned_addr learned my addr 172.21.15.69:0/4199872128 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:36.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7ae939a700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ae4100e20 0x7f7ae4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:36.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.198+0000 7f7aea39c700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae41017b0 0x7f7ae419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:36.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.199+0000 7f7ae9b9b700 1 -- 172.21.15.69:0/4199872128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae41017b0 msgr2=0x7f7ae419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:36.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.199+0000 7f7ae9b9b700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae41017b0 0x7f7ae419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.199+0000 7f7ae9b9b700 1 -- 172.21.15.69:0/4199872128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ae4100e20 msgr2=0x7f7ae4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:36.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.199+0000 7f7ae9b9b700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ae4100e20 0x7f7ae4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.199+0000 7f7ae9b9b700 1 -- 172.21.15.69:0/4199872128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc009c40 con 0x7f7ae40ffc30 2023-07-06T20:17:36.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.199+0000 7f7ae939a700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ae4100e20 0x7f7ae4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:36.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.199+0000 7f7ae9b9b700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae40ffc30 0x7f7ae4197c30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000e970 tx=0x7f7ae000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:36.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.199+0000 7f7adaffd700 1 -- 172.21.15.69:0/4199872128 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0009790 con 0x7f7ae40ffc30 2023-07-06T20:17:36.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.199+0000 7f7adaffd700 1 -- 172.21.15.69:0/4199872128 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae000eeb0 con 0x7f7ae40ffc30 2023-07-06T20:17:36.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.199+0000 7f7aebdff700 1 -- 172.21.15.69:0/4199872128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae419fd80 con 0x7f7ae40ffc30 2023-07-06T20:17:36.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.199+0000 7f7adaffd700 1 -- 172.21.15.69:0/4199872128 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0003d20 con 0x7f7ae40ffc30 2023-07-06T20:17:36.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.199+0000 7f7aebdff700 1 -- 172.21.15.69:0/4199872128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae41a01d0 con 0x7f7ae40ffc30 2023-07-06T20:17:36.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.200+0000 7f7aebdff700 1 -- 172.21.15.69:0/4199872128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae410b460 con 0x7f7ae40ffc30 2023-07-06T20:17:36.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.202+0000 7f7adaffd700 1 -- 172.21.15.69:0/4199872128 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7ae0004870 con 0x7f7ae40ffc30 2023-07-06T20:17:36.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.203+0000 7f7adaffd700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7ad00a1220 0x7f7ad00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:36.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.204+0000 7f7adaffd700 1 -- 172.21.15.69:0/4199872128 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7ae0014070 con 0x7f7ae40ffc30 2023-07-06T20:17:36.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.204+0000 7f7ae939a700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7ad00a1220 0x7f7ad00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:36.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.204+0000 7f7ae939a700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7ad00a1220 0x7f7ad00a36d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4005fd0 tx=0x7f7ad4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:36.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.206+0000 7f7adaffd700 1 -- 172.21.15.69:0/4199872128 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7ae0096430 con 0x7f7ae40ffc30 2023-07-06T20:17:36.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.405+0000 7f7aebdff700 1 -- 172.21.15.69:0/4199872128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ae410b650 con 0x7f7ae40ffc30 2023-07-06T20:17:36.409 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.407+0000 7f7adaffd700 1 -- 172.21.15.69:0/4199872128 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22065 (secure 0 0 0) 0x7f7ae0099a50 con 0x7f7ae40ffc30 2023-07-06T20:17:36.409 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:36.410 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":83,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:17:29.552171+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:17:36.412 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.409+0000 7f7aebdff700 1 -- 172.21.15.69:0/4199872128 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7ad00a1220 msgr2=0x7f7ad00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:36.412 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.409+0000 7f7aebdff700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7ad00a1220 0x7f7ad00a36d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4005fd0 tx=0x7f7ad4009310 comp rx=0 tx=0).stop 2023-07-06T20:17:36.412 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.409+0000 7f7aebdff700 1 -- 172.21.15.69:0/4199872128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae40ffc30 msgr2=0x7f7ae4197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:36.412 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.409+0000 7f7aebdff700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae40ffc30 0x7f7ae4197c30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000e970 tx=0x7f7ae000ed30 comp rx=0 tx=0).stop 2023-07-06T20:17:36.412 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.410+0000 7f7aebdff700 1 -- 172.21.15.69:0/4199872128 shutdown_connections 2023-07-06T20:17:36.412 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.410+0000 7f7aebdff700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7ad00a1220 0x7f7ad00a36d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.413 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.410+0000 7f7aebdff700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae41017b0 0x7f7ae419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.413 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.410+0000 7f7aebdff700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae40ffc30 0x7f7ae4197c30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.413 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.410+0000 7f7aebdff700 1 --2- 172.21.15.69:0/4199872128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ae4100e20 0x7f7ae4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.413 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.410+0000 7f7aebdff700 1 -- 172.21.15.69:0/4199872128 >> 172.21.15.69:0/4199872128 conn(0x7f7ae40fb210 msgr2=0x7f7ae40fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:36.413 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.410+0000 7f7aebdff700 1 -- 172.21.15.69:0/4199872128 shutdown_connections 2023-07-06T20:17:36.413 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.410+0000 7f7aebdff700 1 -- 172.21.15.69:0/4199872128 wait complete. 2023-07-06T20:17:36.431 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T20:17:36.431 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:17:36.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.629+0000 7fb4a09ff700 1 -- 172.21.15.69:0/609983031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c100e20 msgr2=0x7fb49c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:36.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.629+0000 7fb4a09ff700 1 --2- 172.21.15.69:0/609983031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c100e20 0x7fb49c101270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb490009990 tx=0x7fb490009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.630+0000 7fb4a09ff700 1 -- 172.21.15.69:0/609983031 shutdown_connections 2023-07-06T20:17:36.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.630+0000 7fb4a09ff700 1 --2- 172.21.15.69:0/609983031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb49c1017b0 0x7fb49c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.630+0000 7fb4a09ff700 1 --2- 172.21.15.69:0/609983031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c0ffc30 0x7fb49c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.630+0000 7fb4a09ff700 1 --2- 172.21.15.69:0/609983031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c100e20 0x7fb49c101270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.630+0000 7fb4a09ff700 1 -- 172.21.15.69:0/609983031 >> 172.21.15.69:0/609983031 conn(0x7fb49c0fb210 msgr2=0x7fb49c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:36.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.630+0000 7fb4a09ff700 1 -- 172.21.15.69:0/609983031 shutdown_connections 2023-07-06T20:17:36.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.630+0000 7fb4a09ff700 1 -- 172.21.15.69:0/609983031 wait complete. 2023-07-06T20:17:36.634 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.631+0000 7fb4a09ff700 1 Processor -- start 2023-07-06T20:17:36.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.631+0000 7fb4a09ff700 1 -- start start 2023-07-06T20:17:36.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.631+0000 7fb4a09ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb49c0ffc30 0x7fb49c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:36.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.631+0000 7fb4a09ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c100e20 0x7fb49c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:36.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.631+0000 7fb4a09ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c1017b0 0x7fb49c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:36.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.631+0000 7fb4a09ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c19f6e0 con 0x7fb49c100e20 2023-07-06T20:17:36.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.631+0000 7fb4a09ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c19f820 con 0x7fb49c1017b0 2023-07-06T20:17:36.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.631+0000 7fb4a09ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c19f9b0 con 0x7fb49c0ffc30 2023-07-06T20:17:36.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.631+0000 7fb499d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c100e20 0x7fb49c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:36.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb499d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c100e20 0x7fb49c1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51956/0 (socket says 172.21.15.69:51956) 2023-07-06T20:17:36.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb49ad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c1017b0 0x7fb49c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:36.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb499d9b700 1 -- 172.21.15.69:0/2348882036 learned_addr learned my addr 172.21.15.69:0/2348882036 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:36.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb49a59c700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb49c0ffc30 0x7fb49c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:36.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb49ad9d700 1 -- 172.21.15.69:0/2348882036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb49c0ffc30 msgr2=0x7fb49c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:36.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb49ad9d700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb49c0ffc30 0x7fb49c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.637 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb49ad9d700 1 -- 172.21.15.69:0/2348882036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c100e20 msgr2=0x7fb49c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:36.637 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb49ad9d700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c100e20 0x7fb49c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.637 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb49ad9d700 1 -- 172.21.15.69:0/2348882036 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900095f0 con 0x7fb49c1017b0 2023-07-06T20:17:36.637 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb49ad9d700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c1017b0 0x7fb49c19f110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb49400efc0 tx=0x7fb49400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:36.637 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb48b7fe700 1 -- 172.21.15.69:0/2348882036 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb494018070 con 0x7fb49c1017b0 2023-07-06T20:17:36.637 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb48b7fe700 1 -- 172.21.15.69:0/2348882036 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb49400f040 con 0x7fb49c1017b0 2023-07-06T20:17:36.637 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb4a09ff700 1 -- 172.21.15.69:0/2348882036 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49c19fc30 con 0x7fb49c1017b0 2023-07-06T20:17:36.638 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.632+0000 7fb48b7fe700 1 -- 172.21.15.69:0/2348882036 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49400cd40 con 0x7fb49c1017b0 2023-07-06T20:17:36.638 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.633+0000 7fb4a09ff700 1 -- 172.21.15.69:0/2348882036 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c1a0140 con 0x7fb49c1017b0 2023-07-06T20:17:36.638 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.634+0000 7fb4a09ff700 1 -- 172.21.15.69:0/2348882036 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c10b450 con 0x7fb49c1017b0 2023-07-06T20:17:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.635+0000 7fb48b7fe700 1 -- 172.21.15.69:0/2348882036 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb494003a00 con 0x7fb49c1017b0 2023-07-06T20:17:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.636+0000 7fb48b7fe700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb4840a1220 0x7fb4840a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.637+0000 7fb48b7fe700 1 -- 172.21.15.69:0/2348882036 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb4940cc560 con 0x7fb49c1017b0 2023-07-06T20:17:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.637+0000 7fb49a59c700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb4840a1220 0x7fb4840a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.637+0000 7fb49a59c700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb4840a1220 0x7fb4840a36d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb49c100c80 tx=0x7fb48c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:36.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.641+0000 7fb48b7fe700 1 -- 172.21.15.69:0/2348882036 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb494095c90 con 0x7fb49c1017b0 2023-07-06T20:17:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:36 smithi069 ceph-mon[126399]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:17:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:36 smithi069 ceph-mon[126399]: pgmap v272: 97 pgs: 97 active+clean; 615 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9.1 KiB/s wr, 17 op/s 2023-07-06T20:17:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:36 smithi069 ceph-mon[126399]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-06T20:17:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:36 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2964499470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:36 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/4199872128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:36.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.836+0000 7fb4a09ff700 1 -- 172.21.15.69:0/2348882036 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb49c107dd0 con 0x7fb4840a1220 2023-07-06T20:17:36.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.848+0000 7fb48b7fe700 1 -- 172.21.15.69:0/2348882036 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335525 (secure 0 0 0) 0x7fb49c107dd0 con 0x7fb4840a1220 2023-07-06T20:17:36.855 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:36.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.853+0000 7fb4a09ff700 1 -- 172.21.15.69:0/2348882036 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb4840a1220 msgr2=0x7fb4840a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:36.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.853+0000 7fb4a09ff700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb4840a1220 0x7fb4840a36d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb49c100c80 tx=0x7fb48c009260 comp rx=0 tx=0).stop 2023-07-06T20:17:36.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.853+0000 7fb4a09ff700 1 -- 172.21.15.69:0/2348882036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c1017b0 msgr2=0x7fb49c19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:36.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.853+0000 7fb4a09ff700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c1017b0 0x7fb49c19f110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb49400efc0 tx=0x7fb49400c3a0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.853+0000 7fb4a09ff700 1 -- 172.21.15.69:0/2348882036 shutdown_connections 2023-07-06T20:17:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.853+0000 7fb4a09ff700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb4840a1220 0x7fb4840a36d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.853+0000 7fb4a09ff700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49c1017b0 0x7fb49c19f110 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.853+0000 7fb4a09ff700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb49c0ffc30 0x7fb49c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.853+0000 7fb4a09ff700 1 --2- 172.21.15.69:0/2348882036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c100e20 0x7fb49c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.853+0000 7fb4a09ff700 1 -- 172.21.15.69:0/2348882036 >> 172.21.15.69:0/2348882036 conn(0x7fb49c0fb210 msgr2=0x7fb49c1066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.853+0000 7fb4a09ff700 1 -- 172.21.15.69:0/2348882036 shutdown_connections 2023-07-06T20:17:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:36.853+0000 7fb4a09ff700 1 -- 172.21.15.69:0/2348882036 wait complete. 2023-07-06T20:17:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-07-06T20:17:36.874 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2023-07-06T20:17:35.518855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629668,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":145,"num_read_kb":141,"num_write":297,"num_write_kb":3010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":268,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335756,"kb_used_data":12332,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606068,"statfs":{"total":1151940427776,"available":1151596613632,"internally_reserved":0,"allocated":12627968,"data_stored":6892722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39300,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":19,"num_read_kb":32,"num_write":116,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.932168"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.618149+0000","last_change":"2023-07-06T20:17:28.575520+0000","last_active":"2023-07-06T20:17:31.618149+0000","last_peered":"2023-07-06T20:17:31.618149+0000","last_clean":"2023-07-06T20:17:31.618149+0000","last_became_active":"2023-07-06T20:17:28.575209+0000","last_became_peered":"2023-07-06T20:17:28.575209+0000","last_unstale":"2023-07-06T20:17:31.618149+0000","last_undegraded":"2023-07-06T20:17:31.618149+0000","last_fullsized":"2023-07-06T20:17:31.618149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:32:56.425676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879329+0000","last_change":"2023-07-06T20:17:28.572547+0000","last_active":"2023-07-06T20:17:29.879329+0000","last_peered":"2023-07-06T20:17:29.879329+0000","last_clean":"2023-07-06T20:17:29.879329+0000","last_became_active":"2023-07-06T20:17:28.572150+0000","last_became_peered":"2023-07-06T20:17:28.572150+0000","last_unstale":"2023-07-06T20:17:29.879329+0000","last_undegraded":"2023-07-06T20:17:29.879329+0000","last_fullsized":"2023-07-06T20:17:29.879329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:32:16.930127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879457+0000","last_change":"2023-07-06T20:17:28.574926+0000","last_active":"2023-07-06T20:17:29.879457+0000","last_peered":"2023-07-06T20:17:29.879457+0000","last_clean":"2023-07-06T20:17:29.879457+0000","last_became_active":"2023-07-06T20:17:28.574602+0000","last_became_peered":"2023-07-06T20:17:28.574602+0000","last_unstale":"2023-07-06T20:17:29.879457+0000","last_undegraded":"2023-07-06T20:17:29.879457+0000","last_fullsized":"2023-07-06T20:17:29.879457+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:10:07.997134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.565068+0000","last_change":"2023-07-06T20:17:28.571533+0000","last_active":"2023-07-06T20:17:29.565068+0000","last_peered":"2023-07-06T20:17:29.565068+0000","last_clean":"2023-07-06T20:17:29.565068+0000","last_became_active":"2023-07-06T20:17:28.571099+0000","last_became_peered":"2023-07-06T20:17:28.571099+0000","last_unstale":"2023-07-06T20:17:29.565068+0000","last_undegraded":"2023-07-06T20:17:29.565068+0000","last_fullsized":"2023-07-06T20:17:29.565068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:28:20.866578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556729+0000","last_change":"2023-07-06T20:17:28.572100+0000","last_active":"2023-07-06T20:17:31.556729+0000","last_peered":"2023-07-06T20:17:31.556729+0000","last_clean":"2023-07-06T20:17:31.556729+0000","last_became_active":"2023-07-06T20:17:28.571772+0000","last_became_peered":"2023-07-06T20:17:28.571772+0000","last_unstale":"2023-07-06T20:17:31.556729+0000","last_undegraded":"2023-07-06T20:17:31.556729+0000","last_fullsized":"2023-07-06T20:17:31.556729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:49:44.228120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556089+0000","last_change":"2023-07-06T20:17:28.577816+0000","last_active":"2023-07-06T20:17:31.556089+0000","last_peered":"2023-07-06T20:17:31.556089+0000","last_clean":"2023-07-06T20:17:31.556089+0000","last_became_active":"2023-07-06T20:17:28.577382+0000","last_became_peered":"2023-07-06T20:17:28.577382+0000","last_unstale":"2023-07-06T20:17:31.556089+0000","last_undegraded":"2023-07-06T20:17:31.556089+0000","last_fullsized":"2023-07-06T20:17:31.556089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:43.603856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574702+0000","last_change":"2023-07-06T20:17:28.570415+0000","last_active":"2023-07-06T20:17:28.574702+0000","last_peered":"2023-07-06T20:17:28.574702+0000","last_clean":"2023-07-06T20:17:28.574702+0000","last_became_active":"2023-07-06T20:17:28.570091+0000","last_became_peered":"2023-07-06T20:17:28.570091+0000","last_unstale":"2023-07-06T20:17:28.574702+0000","last_undegraded":"2023-07-06T20:17:28.574702+0000","last_fullsized":"2023-07-06T20:17:28.574702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:06:13.149895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555505+0000","last_change":"2023-07-06T20:17:28.568370+0000","last_active":"2023-07-06T20:17:31.555505+0000","last_peered":"2023-07-06T20:17:31.555505+0000","last_clean":"2023-07-06T20:17:31.555505+0000","last_became_active":"2023-07-06T20:17:28.568032+0000","last_became_peered":"2023-07-06T20:17:28.568032+0000","last_unstale":"2023-07-06T20:17:31.555505+0000","last_undegraded":"2023-07-06T20:17:31.555505+0000","last_fullsized":"2023-07-06T20:17:31.555505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:26:23.998744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:30.075961+0000","last_change":"2023-07-06T20:17:28.563386+0000","last_active":"2023-07-06T20:17:30.075961+0000","last_peered":"2023-07-06T20:17:30.075961+0000","last_clean":"2023-07-06T20:17:30.075961+0000","last_became_active":"2023-07-06T20:17:28.562864+0000","last_became_peered":"2023-07-06T20:17:28.562864+0000","last_unstale":"2023-07-06T20:17:30.075961+0000","last_undegraded":"2023-07-06T20:17:30.075961+0000","last_fullsized":"2023-07-06T20:17:30.075961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:33:27.513610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556379+0000","last_change":"2023-07-06T20:17:28.578016+0000","last_active":"2023-07-06T20:17:31.556379+0000","last_peered":"2023-07-06T20:17:31.556379+0000","last_clean":"2023-07-06T20:17:31.556379+0000","last_became_active":"2023-07-06T20:17:28.577305+0000","last_became_peered":"2023-07-06T20:17:28.577305+0000","last_unstale":"2023-07-06T20:17:31.556379+0000","last_undegraded":"2023-07-06T20:17:31.556379+0000","last_fullsized":"2023-07-06T20:17:31.556379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:59:29.567212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574712+0000","last_change":"2023-07-06T20:17:28.569683+0000","last_active":"2023-07-06T20:17:28.574712+0000","last_peered":"2023-07-06T20:17:28.574712+0000","last_clean":"2023-07-06T20:17:28.574712+0000","last_became_active":"2023-07-06T20:17:28.569070+0000","last_became_peered":"2023-07-06T20:17:28.569070+0000","last_unstale":"2023-07-06T20:17:28.574712+0000","last_undegraded":"2023-07-06T20:17:28.574712+0000","last_fullsized":"2023-07-06T20:17:28.574712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:45:32.404472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.565041+0000","last_change":"2023-07-06T20:17:28.579463+0000","last_active":"2023-07-06T20:17:29.565041+0000","last_peered":"2023-07-06T20:17:29.565041+0000","last_clean":"2023-07-06T20:17:29.565041+0000","last_became_active":"2023-07-06T20:17:28.579141+0000","last_became_peered":"2023-07-06T20:17:28.579141+0000","last_unstale":"2023-07-06T20:17:29.565041+0000","last_undegraded":"2023-07-06T20:17:29.565041+0000","last_fullsized":"2023-07-06T20:17:29.565041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:04:10.705471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.566959+0000","last_change":"2023-07-06T20:17:28.570235+0000","last_active":"2023-07-06T20:17:29.566959+0000","last_peered":"2023-07-06T20:17:29.566959+0000","last_clean":"2023-07-06T20:17:29.566959+0000","last_became_active":"2023-07-06T20:17:28.569780+0000","last_became_peered":"2023-07-06T20:17:28.569780+0000","last_unstale":"2023-07-06T20:17:29.566959+0000","last_undegraded":"2023-07-06T20:17:29.566959+0000","last_fullsized":"2023-07-06T20:17:29.566959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:26:47.226331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879157+0000","last_change":"2023-07-06T20:17:28.572197+0000","last_active":"2023-07-06T20:17:29.879157+0000","last_peered":"2023-07-06T20:17:29.879157+0000","last_clean":"2023-07-06T20:17:29.879157+0000","last_became_active":"2023-07-06T20:17:28.571816+0000","last_became_peered":"2023-07-06T20:17:28.571816+0000","last_unstale":"2023-07-06T20:17:29.879157+0000","last_undegraded":"2023-07-06T20:17:29.879157+0000","last_fullsized":"2023-07-06T20:17:29.879157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:56:16.289124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879101+0000","last_change":"2023-07-06T20:17:28.574250+0000","last_active":"2023-07-06T20:17:29.879101+0000","last_peered":"2023-07-06T20:17:29.879101+0000","last_clean":"2023-07-06T20:17:29.879101+0000","last_became_active":"2023-07-06T20:17:28.573916+0000","last_became_peered":"2023-07-06T20:17:28.573916+0000","last_unstale":"2023-07-06T20:17:29.879101+0000","last_undegraded":"2023-07-06T20:17:29.879101+0000","last_fullsized":"2023-07-06T20:17:29.879101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:55:31.687080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:30.075428+0000","last_change":"2023-07-06T20:17:28.573552+0000","last_active":"2023-07-06T20:17:30.075428+0000","last_peered":"2023-07-06T20:17:30.075428+0000","last_clean":"2023-07-06T20:17:30.075428+0000","last_became_active":"2023-07-06T20:17:28.573253+0000","last_became_peered":"2023-07-06T20:17:28.573253+0000","last_unstale":"2023-07-06T20:17:30.075428+0000","last_undegraded":"2023-07-06T20:17:30.075428+0000","last_fullsized":"2023-07-06T20:17:30.075428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:51.333395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.618223+0000","last_change":"2023-07-06T20:17:28.572425+0000","last_active":"2023-07-06T20:17:31.618223+0000","last_peered":"2023-07-06T20:17:31.618223+0000","last_clean":"2023-07-06T20:17:31.618223+0000","last_became_active":"2023-07-06T20:17:28.572104+0000","last_became_peered":"2023-07-06T20:17:28.572104+0000","last_unstale":"2023-07-06T20:17:31.618223+0000","last_undegraded":"2023-07-06T20:17:31.618223+0000","last_fullsized":"2023-07-06T20:17:31.618223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:08:34.890550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555809+0000","last_change":"2023-07-06T20:17:28.573814+0000","last_active":"2023-07-06T20:17:31.555809+0000","last_peered":"2023-07-06T20:17:31.555809+0000","last_clean":"2023-07-06T20:17:31.555809+0000","last_became_active":"2023-07-06T20:17:28.573470+0000","last_became_peered":"2023-07-06T20:17:28.573470+0000","last_unstale":"2023-07-06T20:17:31.555809+0000","last_undegraded":"2023-07-06T20:17:31.555809+0000","last_fullsized":"2023-07-06T20:17:31.555809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:49:01.038082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.564943+0000","last_change":"2023-07-06T20:17:28.571639+0000","last_active":"2023-07-06T20:17:29.564943+0000","last_peered":"2023-07-06T20:17:29.564943+0000","last_clean":"2023-07-06T20:17:29.564943+0000","last_became_active":"2023-07-06T20:17:28.571356+0000","last_became_peered":"2023-07-06T20:17:28.571356+0000","last_unstale":"2023-07-06T20:17:29.564943+0000","last_undegraded":"2023-07-06T20:17:29.564943+0000","last_fullsized":"2023-07-06T20:17:29.564943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:20:33.572044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556007+0000","last_change":"2023-07-06T20:17:27.603771+0000","last_active":"2023-07-06T20:17:31.556007+0000","last_peered":"2023-07-06T20:17:31.556007+0000","last_clean":"2023-07-06T20:17:31.556007+0000","last_became_active":"2023-07-06T20:17:27.603404+0000","last_became_peered":"2023-07-06T20:17:27.603404+0000","last_unstale":"2023-07-06T20:17:31.556007+0000","last_undegraded":"2023-07-06T20:17:31.556007+0000","last_fullsized":"2023-07-06T20:17:31.556007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:14:29.214268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555982+0000","last_change":"2023-07-06T20:17:28.568371+0000","last_active":"2023-07-06T20:17:31.555982+0000","last_peered":"2023-07-06T20:17:31.555982+0000","last_clean":"2023-07-06T20:17:31.555982+0000","last_became_active":"2023-07-06T20:17:28.568034+0000","last_became_peered":"2023-07-06T20:17:28.568034+0000","last_unstale":"2023-07-06T20:17:31.555982+0000","last_undegraded":"2023-07-06T20:17:31.555982+0000","last_fullsized":"2023-07-06T20:17:31.555982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:16:18.752577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556266+0000","last_change":"2023-07-06T20:17:27.609833+0000","last_active":"2023-07-06T20:17:31.556266+0000","last_peered":"2023-07-06T20:17:31.556266+0000","last_clean":"2023-07-06T20:17:31.556266+0000","last_became_active":"2023-07-06T20:17:27.609520+0000","last_became_peered":"2023-07-06T20:17:27.609520+0000","last_unstale":"2023-07-06T20:17:31.556266+0000","last_undegraded":"2023-07-06T20:17:31.556266+0000","last_fullsized":"2023-07-06T20:17:31.556266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:40:41.429428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879431+0000","last_change":"2023-07-06T20:17:28.574520+0000","last_active":"2023-07-06T20:17:29.879431+0000","last_peered":"2023-07-06T20:17:29.879431+0000","last_clean":"2023-07-06T20:17:29.879431+0000","last_became_active":"2023-07-06T20:17:28.574158+0000","last_became_peered":"2023-07-06T20:17:28.574158+0000","last_unstale":"2023-07-06T20:17:29.879431+0000","last_undegraded":"2023-07-06T20:17:29.879431+0000","last_fullsized":"2023-07-06T20:17:29.879431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:42:31.557796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.562363+0000","last_change":"2023-07-06T20:17:27.601612+0000","last_active":"2023-07-06T20:17:32.562363+0000","last_peered":"2023-07-06T20:17:32.562363+0000","last_clean":"2023-07-06T20:17:32.562363+0000","last_became_active":"2023-07-06T20:17:27.601313+0000","last_became_peered":"2023-07-06T20:17:27.601313+0000","last_unstale":"2023-07-06T20:17:32.562363+0000","last_undegraded":"2023-07-06T20:17:32.562363+0000","last_fullsized":"2023-07-06T20:17:32.562363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:45:02.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:30.075593+0000","last_change":"2023-07-06T20:17:28.566389+0000","last_active":"2023-07-06T20:17:30.075593+0000","last_peered":"2023-07-06T20:17:30.075593+0000","last_clean":"2023-07-06T20:17:30.075593+0000","last_became_active":"2023-07-06T20:17:28.566094+0000","last_became_peered":"2023-07-06T20:17:28.566094+0000","last_unstale":"2023-07-06T20:17:30.075593+0000","last_undegraded":"2023-07-06T20:17:30.075593+0000","last_fullsized":"2023-07-06T20:17:30.075593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:47:12.929391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556276+0000","last_change":"2023-07-06T20:17:27.599893+0000","last_active":"2023-07-06T20:17:31.556276+0000","last_peered":"2023-07-06T20:17:31.556276+0000","last_clean":"2023-07-06T20:17:31.556276+0000","last_became_active":"2023-07-06T20:17:27.599544+0000","last_became_peered":"2023-07-06T20:17:27.599544+0000","last_unstale":"2023-07-06T20:17:31.556276+0000","last_undegraded":"2023-07-06T20:17:31.556276+0000","last_fullsized":"2023-07-06T20:17:31.556276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:03:13.210197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556719+0000","last_change":"2023-07-06T20:17:28.575794+0000","last_active":"2023-07-06T20:17:31.556719+0000","last_peered":"2023-07-06T20:17:31.556719+0000","last_clean":"2023-07-06T20:17:31.556719+0000","last_became_active":"2023-07-06T20:17:28.575456+0000","last_became_peered":"2023-07-06T20:17:28.575456+0000","last_unstale":"2023-07-06T20:17:31.556719+0000","last_undegraded":"2023-07-06T20:17:31.556719+0000","last_fullsized":"2023-07-06T20:17:31.556719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:57.974475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.620919+0000","last_change":"2023-07-06T20:17:27.596351+0000","last_active":"2023-07-06T20:17:31.620919+0000","last_peered":"2023-07-06T20:17:31.620919+0000","last_clean":"2023-07-06T20:17:31.620919+0000","last_became_active":"2023-07-06T20:17:27.595982+0000","last_became_peered":"2023-07-06T20:17:27.595982+0000","last_unstale":"2023-07-06T20:17:31.620919+0000","last_undegraded":"2023-07-06T20:17:31.620919+0000","last_fullsized":"2023-07-06T20:17:31.620919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:13:06.029344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555937+0000","last_change":"2023-07-06T20:17:28.570018+0000","last_active":"2023-07-06T20:17:31.555937+0000","last_peered":"2023-07-06T20:17:31.555937+0000","last_clean":"2023-07-06T20:17:31.555937+0000","last_became_active":"2023-07-06T20:17:28.569674+0000","last_became_peered":"2023-07-06T20:17:28.569674+0000","last_unstale":"2023-07-06T20:17:31.555937+0000","last_undegraded":"2023-07-06T20:17:31.555937+0000","last_fullsized":"2023-07-06T20:17:31.555937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:59:52.145494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.563000+0000","last_change":"2023-07-06T20:17:27.602792+0000","last_active":"2023-07-06T20:17:32.563000+0000","last_peered":"2023-07-06T20:17:32.563000+0000","last_clean":"2023-07-06T20:17:32.563000+0000","last_became_active":"2023-07-06T20:17:27.602373+0000","last_became_peered":"2023-07-06T20:17:27.602373+0000","last_unstale":"2023-07-06T20:17:32.563000+0000","last_undegraded":"2023-07-06T20:17:32.563000+0000","last_fullsized":"2023-07-06T20:17:32.563000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:55:12.560893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556254+0000","last_change":"2023-07-06T20:17:28.575249+0000","last_active":"2023-07-06T20:17:31.556254+0000","last_peered":"2023-07-06T20:17:31.556254+0000","last_clean":"2023-07-06T20:17:31.556254+0000","last_became_active":"2023-07-06T20:17:28.574954+0000","last_became_peered":"2023-07-06T20:17:28.574954+0000","last_unstale":"2023-07-06T20:17:31.556254+0000","last_undegraded":"2023-07-06T20:17:31.556254+0000","last_fullsized":"2023-07-06T20:17:31.556254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:36:50.592568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.563699+0000","last_change":"2023-07-06T20:17:27.600552+0000","last_active":"2023-07-06T20:17:32.563699+0000","last_peered":"2023-07-06T20:17:32.563699+0000","last_clean":"2023-07-06T20:17:32.563699+0000","last_became_active":"2023-07-06T20:17:27.600016+0000","last_became_peered":"2023-07-06T20:17:27.600016+0000","last_unstale":"2023-07-06T20:17:32.563699+0000","last_undegraded":"2023-07-06T20:17:32.563699+0000","last_fullsized":"2023-07-06T20:17:32.563699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:42:56.520957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555603+0000","last_change":"2023-07-06T20:17:28.566364+0000","last_active":"2023-07-06T20:17:31.555603+0000","last_peered":"2023-07-06T20:17:31.555603+0000","last_clean":"2023-07-06T20:17:31.555603+0000","last_became_active":"2023-07-06T20:17:28.565474+0000","last_became_peered":"2023-07-06T20:17:28.565474+0000","last_unstale":"2023-07-06T20:17:31.555603+0000","last_undegraded":"2023-07-06T20:17:31.555603+0000","last_fullsized":"2023-07-06T20:17:31.555603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:20:32.320775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555897+0000","last_change":"2023-07-06T20:17:27.598606+0000","last_active":"2023-07-06T20:17:31.555897+0000","last_peered":"2023-07-06T20:17:31.555897+0000","last_clean":"2023-07-06T20:17:31.555897+0000","last_became_active":"2023-07-06T20:17:27.598292+0000","last_became_peered":"2023-07-06T20:17:27.598292+0000","last_unstale":"2023-07-06T20:17:31.555897+0000","last_undegraded":"2023-07-06T20:17:31.555897+0000","last_fullsized":"2023-07-06T20:17:31.555897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:32:06.168848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555943+0000","last_change":"2023-07-06T20:17:28.571246+0000","last_active":"2023-07-06T20:17:31.555943+0000","last_peered":"2023-07-06T20:17:31.555943+0000","last_clean":"2023-07-06T20:17:31.555943+0000","last_became_active":"2023-07-06T20:17:28.570846+0000","last_became_peered":"2023-07-06T20:17:28.570846+0000","last_unstale":"2023-07-06T20:17:31.555943+0000","last_undegraded":"2023-07-06T20:17:31.555943+0000","last_fullsized":"2023-07-06T20:17:31.555943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:41:17.390535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:33.558723+0000","last_change":"2023-07-06T20:17:27.595879+0000","last_active":"2023-07-06T20:17:33.558723+0000","last_peered":"2023-07-06T20:17:33.558723+0000","last_clean":"2023-07-06T20:17:33.558723+0000","last_became_active":"2023-07-06T20:17:27.595553+0000","last_became_peered":"2023-07-06T20:17:27.595553+0000","last_unstale":"2023-07-06T20:17:33.558723+0000","last_undegraded":"2023-07-06T20:17:33.558723+0000","last_fullsized":"2023-07-06T20:17:33.558723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:29:30.880922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:30.075914+0000","last_change":"2023-07-06T20:17:28.566816+0000","last_active":"2023-07-06T20:17:30.075914+0000","last_peered":"2023-07-06T20:17:30.075914+0000","last_clean":"2023-07-06T20:17:30.075914+0000","last_became_active":"2023-07-06T20:17:28.566565+0000","last_became_peered":"2023-07-06T20:17:28.566565+0000","last_unstale":"2023-07-06T20:17:30.075914+0000","last_undegraded":"2023-07-06T20:17:30.075914+0000","last_fullsized":"2023-07-06T20:17:30.075914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:52:54.354015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.560374+0000","last_change":"2023-07-06T20:17:27.598572+0000","last_active":"2023-07-06T20:17:28.560374+0000","last_peered":"2023-07-06T20:17:28.560374+0000","last_clean":"2023-07-06T20:17:28.560374+0000","last_became_active":"2023-07-06T20:17:27.598243+0000","last_became_peered":"2023-07-06T20:17:27.598243+0000","last_unstale":"2023-07-06T20:17:28.560374+0000","last_undegraded":"2023-07-06T20:17:28.560374+0000","last_fullsized":"2023-07-06T20:17:28.560374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:29:09.506737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879451+0000","last_change":"2023-07-06T20:17:28.572400+0000","last_active":"2023-07-06T20:17:29.879451+0000","last_peered":"2023-07-06T20:17:29.879451+0000","last_clean":"2023-07-06T20:17:29.879451+0000","last_became_active":"2023-07-06T20:17:28.572047+0000","last_became_peered":"2023-07-06T20:17:28.572047+0000","last_unstale":"2023-07-06T20:17:29.879451+0000","last_undegraded":"2023-07-06T20:17:29.879451+0000","last_fullsized":"2023-07-06T20:17:29.879451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:22:53.203510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556727+0000","last_change":"2023-07-06T20:17:27.603224+0000","last_active":"2023-07-06T20:17:31.556727+0000","last_peered":"2023-07-06T20:17:31.556727+0000","last_clean":"2023-07-06T20:17:31.556727+0000","last_became_active":"2023-07-06T20:17:27.602864+0000","last_became_peered":"2023-07-06T20:17:27.602864+0000","last_unstale":"2023-07-06T20:17:31.556727+0000","last_undegraded":"2023-07-06T20:17:31.556727+0000","last_fullsized":"2023-07-06T20:17:31.556727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:59:59.044611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:30.075651+0000","last_change":"2023-07-06T20:17:28.572683+0000","last_active":"2023-07-06T20:17:30.075651+0000","last_peered":"2023-07-06T20:17:30.075651+0000","last_clean":"2023-07-06T20:17:30.075651+0000","last_became_active":"2023-07-06T20:17:28.572352+0000","last_became_peered":"2023-07-06T20:17:28.572352+0000","last_unstale":"2023-07-06T20:17:30.075651+0000","last_undegraded":"2023-07-06T20:17:30.075651+0000","last_fullsized":"2023-07-06T20:17:30.075651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:03:04.106335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'18","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:35.029036+0000","last_change":"2023-07-06T20:17:27.597285+0000","last_active":"2023-07-06T20:17:35.029036+0000","last_peered":"2023-07-06T20:17:35.029036+0000","last_clean":"2023-07-06T20:17:35.029036+0000","last_became_active":"2023-07-06T20:17:27.596850+0000","last_became_peered":"2023-07-06T20:17:27.596850+0000","last_unstale":"2023-07-06T20:17:35.029036+0000","last_undegraded":"2023-07-06T20:17:35.029036+0000","last_fullsized":"2023-07-06T20:17:35.029036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:51:25.444929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36237,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":22,"num_write":19,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556263+0000","last_change":"2023-07-06T20:17:28.569855+0000","last_active":"2023-07-06T20:17:31.556263+0000","last_peered":"2023-07-06T20:17:31.556263+0000","last_clean":"2023-07-06T20:17:31.556263+0000","last_became_active":"2023-07-06T20:17:28.569495+0000","last_became_peered":"2023-07-06T20:17:28.569495+0000","last_unstale":"2023-07-06T20:17:31.556263+0000","last_undegraded":"2023-07-06T20:17:31.556263+0000","last_fullsized":"2023-07-06T20:17:31.556263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:37:13.491582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.562082+0000","last_change":"2023-07-06T20:17:27.605898+0000","last_active":"2023-07-06T20:17:32.562082+0000","last_peered":"2023-07-06T20:17:32.562082+0000","last_clean":"2023-07-06T20:17:32.562082+0000","last_became_active":"2023-07-06T20:17:27.605605+0000","last_became_peered":"2023-07-06T20:17:27.605605+0000","last_unstale":"2023-07-06T20:17:32.562082+0000","last_undegraded":"2023-07-06T20:17:32.562082+0000","last_fullsized":"2023-07-06T20:17:32.562082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:19:04.240869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.566733+0000","last_change":"2023-07-06T20:17:28.579236+0000","last_active":"2023-07-06T20:17:29.566733+0000","last_peered":"2023-07-06T20:17:29.566733+0000","last_clean":"2023-07-06T20:17:29.566733+0000","last_became_active":"2023-07-06T20:17:28.578854+0000","last_became_peered":"2023-07-06T20:17:28.578854+0000","last_unstale":"2023-07-06T20:17:29.566733+0000","last_undegraded":"2023-07-06T20:17:29.566733+0000","last_fullsized":"2023-07-06T20:17:29.566733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:25:51.789427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.620762+0000","last_change":"2023-07-06T20:17:27.595724+0000","last_active":"2023-07-06T20:17:31.620762+0000","last_peered":"2023-07-06T20:17:31.620762+0000","last_clean":"2023-07-06T20:17:31.620762+0000","last_became_active":"2023-07-06T20:17:27.595360+0000","last_became_peered":"2023-07-06T20:17:27.595360+0000","last_unstale":"2023-07-06T20:17:31.620762+0000","last_undegraded":"2023-07-06T20:17:31.620762+0000","last_fullsized":"2023-07-06T20:17:31.620762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:25:06.238847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574678+0000","last_change":"2023-07-06T20:17:28.573447+0000","last_active":"2023-07-06T20:17:28.574678+0000","last_peered":"2023-07-06T20:17:28.574678+0000","last_clean":"2023-07-06T20:17:28.574678+0000","last_became_active":"2023-07-06T20:17:28.573161+0000","last_became_peered":"2023-07-06T20:17:28.573161+0000","last_unstale":"2023-07-06T20:17:28.574678+0000","last_undegraded":"2023-07-06T20:17:28.574678+0000","last_fullsized":"2023-07-06T20:17:28.574678+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:08:06.240576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.564909+0000","last_change":"2023-07-06T20:17:27.596254+0000","last_active":"2023-07-06T20:17:29.564909+0000","last_peered":"2023-07-06T20:17:29.564909+0000","last_clean":"2023-07-06T20:17:29.564909+0000","last_became_active":"2023-07-06T20:17:27.595178+0000","last_became_peered":"2023-07-06T20:17:27.595178+0000","last_unstale":"2023-07-06T20:17:29.564909+0000","last_undegraded":"2023-07-06T20:17:29.564909+0000","last_fullsized":"2023-07-06T20:17:29.564909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:49.788828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574738+0000","last_change":"2023-07-06T20:17:28.569745+0000","last_active":"2023-07-06T20:17:28.574738+0000","last_peered":"2023-07-06T20:17:28.574738+0000","last_clean":"2023-07-06T20:17:28.574738+0000","last_became_active":"2023-07-06T20:17:28.569468+0000","last_became_peered":"2023-07-06T20:17:28.569468+0000","last_unstale":"2023-07-06T20:17:28.574738+0000","last_undegraded":"2023-07-06T20:17:28.574738+0000","last_fullsized":"2023-07-06T20:17:28.574738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:47:12.397526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.564783+0000","last_change":"2023-07-06T20:17:27.608166+0000","last_active":"2023-07-06T20:17:32.564783+0000","last_peered":"2023-07-06T20:17:32.564783+0000","last_clean":"2023-07-06T20:17:32.564783+0000","last_became_active":"2023-07-06T20:17:27.607904+0000","last_became_peered":"2023-07-06T20:17:27.607904+0000","last_unstale":"2023-07-06T20:17:32.564783+0000","last_undegraded":"2023-07-06T20:17:32.564783+0000","last_fullsized":"2023-07-06T20:17:32.564783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:19:42.369761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556531+0000","last_change":"2023-07-06T20:17:28.575390+0000","last_active":"2023-07-06T20:17:31.556531+0000","last_peered":"2023-07-06T20:17:31.556531+0000","last_clean":"2023-07-06T20:17:31.556531+0000","last_became_active":"2023-07-06T20:17:28.575083+0000","last_became_peered":"2023-07-06T20:17:28.575083+0000","last_unstale":"2023-07-06T20:17:31.556531+0000","last_undegraded":"2023-07-06T20:17:31.556531+0000","last_fullsized":"2023-07-06T20:17:31.556531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:08:34.646494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556326+0000","last_change":"2023-07-06T20:17:27.600891+0000","last_active":"2023-07-06T20:17:31.556326+0000","last_peered":"2023-07-06T20:17:31.556326+0000","last_clean":"2023-07-06T20:17:31.556326+0000","last_became_active":"2023-07-06T20:17:27.600533+0000","last_became_peered":"2023-07-06T20:17:27.600533+0000","last_unstale":"2023-07-06T20:17:31.556326+0000","last_undegraded":"2023-07-06T20:17:31.556326+0000","last_fullsized":"2023-07-06T20:17:31.556326+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:22:04.466952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.617546+0000","last_change":"2023-07-06T20:17:28.574205+0000","last_active":"2023-07-06T20:17:31.617546+0000","last_peered":"2023-07-06T20:17:31.617546+0000","last_clean":"2023-07-06T20:17:31.617546+0000","last_became_active":"2023-07-06T20:17:28.573699+0000","last_became_peered":"2023-07-06T20:17:28.573699+0000","last_unstale":"2023-07-06T20:17:31.617546+0000","last_undegraded":"2023-07-06T20:17:31.617546+0000","last_fullsized":"2023-07-06T20:17:31.617546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:58:45.139117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555888+0000","last_change":"2023-07-06T20:17:27.604464+0000","last_active":"2023-07-06T20:17:31.555888+0000","last_peered":"2023-07-06T20:17:31.555888+0000","last_clean":"2023-07-06T20:17:31.555888+0000","last_became_active":"2023-07-06T20:17:27.604036+0000","last_became_peered":"2023-07-06T20:17:27.604036+0000","last_unstale":"2023-07-06T20:17:31.555888+0000","last_undegraded":"2023-07-06T20:17:31.555888+0000","last_fullsized":"2023-07-06T20:17:31.555888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:08:32.646765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556251+0000","last_change":"2023-07-06T20:17:28.577302+0000","last_active":"2023-07-06T20:17:31.556251+0000","last_peered":"2023-07-06T20:17:31.556251+0000","last_clean":"2023-07-06T20:17:31.556251+0000","last_became_active":"2023-07-06T20:17:28.576998+0000","last_became_peered":"2023-07-06T20:17:28.576998+0000","last_unstale":"2023-07-06T20:17:31.556251+0000","last_undegraded":"2023-07-06T20:17:31.556251+0000","last_fullsized":"2023-07-06T20:17:31.556251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:55:59.882268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.560306+0000","last_change":"2023-07-06T20:17:27.608391+0000","last_active":"2023-07-06T20:17:28.560306+0000","last_peered":"2023-07-06T20:17:28.560306+0000","last_clean":"2023-07-06T20:17:28.560306+0000","last_became_active":"2023-07-06T20:17:27.608073+0000","last_became_peered":"2023-07-06T20:17:27.608073+0000","last_unstale":"2023-07-06T20:17:28.560306+0000","last_undegraded":"2023-07-06T20:17:28.560306+0000","last_fullsized":"2023-07-06T20:17:28.560306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:54:19.558084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556268+0000","last_change":"2023-07-06T20:17:28.577554+0000","last_active":"2023-07-06T20:17:31.556268+0000","last_peered":"2023-07-06T20:17:31.556268+0000","last_clean":"2023-07-06T20:17:31.556268+0000","last_became_active":"2023-07-06T20:17:28.577208+0000","last_became_peered":"2023-07-06T20:17:28.577208+0000","last_unstale":"2023-07-06T20:17:31.556268+0000","last_undegraded":"2023-07-06T20:17:31.556268+0000","last_fullsized":"2023-07-06T20:17:31.556268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:29:19.435901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556713+0000","last_change":"2023-07-06T20:17:28.571678+0000","last_active":"2023-07-06T20:17:31.556713+0000","last_peered":"2023-07-06T20:17:31.556713+0000","last_clean":"2023-07-06T20:17:31.556713+0000","last_became_active":"2023-07-06T20:17:28.570974+0000","last_became_peered":"2023-07-06T20:17:28.570974+0000","last_unstale":"2023-07-06T20:17:31.556713+0000","last_undegraded":"2023-07-06T20:17:31.556713+0000","last_fullsized":"2023-07-06T20:17:31.556713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:14.827580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.564171+0000","last_change":"2023-07-06T20:17:27.607172+0000","last_active":"2023-07-06T20:17:32.564171+0000","last_peered":"2023-07-06T20:17:32.564171+0000","last_clean":"2023-07-06T20:17:32.564171+0000","last_became_active":"2023-07-06T20:17:27.606841+0000","last_became_peered":"2023-07-06T20:17:27.606841+0000","last_unstale":"2023-07-06T20:17:32.564171+0000","last_undegraded":"2023-07-06T20:17:32.564171+0000","last_fullsized":"2023-07-06T20:17:32.564171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:10:16.491664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556335+0000","last_change":"2023-07-06T20:17:28.567707+0000","last_active":"2023-07-06T20:17:31.556335+0000","last_peered":"2023-07-06T20:17:31.556335+0000","last_clean":"2023-07-06T20:17:31.556335+0000","last_became_active":"2023-07-06T20:17:28.567394+0000","last_became_peered":"2023-07-06T20:17:28.567394+0000","last_unstale":"2023-07-06T20:17:31.556335+0000","last_undegraded":"2023-07-06T20:17:31.556335+0000","last_fullsized":"2023-07-06T20:17:31.556335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:09:58.514942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.619257+0000","last_change":"2023-07-06T20:17:27.595746+0000","last_active":"2023-07-06T20:17:31.619257+0000","last_peered":"2023-07-06T20:17:31.619257+0000","last_clean":"2023-07-06T20:17:31.619257+0000","last_became_active":"2023-07-06T20:17:27.595039+0000","last_became_peered":"2023-07-06T20:17:27.595039+0000","last_unstale":"2023-07-06T20:17:31.619257+0000","last_undegraded":"2023-07-06T20:17:31.619257+0000","last_fullsized":"2023-07-06T20:17:31.619257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:00:16.991853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":331,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.560399+0000","last_change":"2023-07-06T20:16:23.542324+0000","last_active":"2023-07-06T20:17:28.560399+0000","last_peered":"2023-07-06T20:17:28.560399+0000","last_clean":"2023-07-06T20:17:28.560399+0000","last_became_active":"2023-07-06T20:16:23.541832+0000","last_became_peered":"2023-07-06T20:16:23.541832+0000","last_unstale":"2023-07-06T20:17:28.560399+0000","last_undegraded":"2023-07-06T20:17:28.560399+0000","last_fullsized":"2023-07-06T20:17:28.560399+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_clean_scrub_stamp":"2023-07-06T20:13:14.072924+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:39:31.829270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.566606+0000","last_change":"2023-07-06T20:17:28.577309+0000","last_active":"2023-07-06T20:17:29.566606+0000","last_peered":"2023-07-06T20:17:29.566606+0000","last_clean":"2023-07-06T20:17:29.566606+0000","last_became_active":"2023-07-06T20:17:28.576962+0000","last_became_peered":"2023-07-06T20:17:28.576962+0000","last_unstale":"2023-07-06T20:17:29.566606+0000","last_undegraded":"2023-07-06T20:17:29.566606+0000","last_fullsized":"2023-07-06T20:17:29.566606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:58:48.351245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.561010+0000","last_change":"2023-07-06T20:17:27.601487+0000","last_active":"2023-07-06T20:17:28.561010+0000","last_peered":"2023-07-06T20:17:28.561010+0000","last_clean":"2023-07-06T20:17:28.561010+0000","last_became_active":"2023-07-06T20:17:27.601195+0000","last_became_peered":"2023-07-06T20:17:27.601195+0000","last_unstale":"2023-07-06T20:17:28.561010+0000","last_undegraded":"2023-07-06T20:17:28.561010+0000","last_fullsized":"2023-07-06T20:17:28.561010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:56:39.336345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.619412+0000","last_change":"2023-07-06T20:17:27.610374+0000","last_active":"2023-07-06T20:17:31.619412+0000","last_peered":"2023-07-06T20:17:31.619412+0000","last_clean":"2023-07-06T20:17:31.619412+0000","last_became_active":"2023-07-06T20:17:27.610104+0000","last_became_peered":"2023-07-06T20:17:27.610104+0000","last_unstale":"2023-07-06T20:17:31.619412+0000","last_undegraded":"2023-07-06T20:17:31.619412+0000","last_fullsized":"2023-07-06T20:17:31.619412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:41:02.053169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.567031+0000","last_change":"2023-07-06T20:17:28.570146+0000","last_active":"2023-07-06T20:17:29.567031+0000","last_peered":"2023-07-06T20:17:29.567031+0000","last_clean":"2023-07-06T20:17:29.567031+0000","last_became_active":"2023-07-06T20:17:28.569341+0000","last_became_peered":"2023-07-06T20:17:28.569341+0000","last_unstale":"2023-07-06T20:17:29.567031+0000","last_undegraded":"2023-07-06T20:17:29.567031+0000","last_fullsized":"2023-07-06T20:17:29.567031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:08:42.339199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555949+0000","last_change":"2023-07-06T20:17:27.604630+0000","last_active":"2023-07-06T20:17:31.555949+0000","last_peered":"2023-07-06T20:17:31.555949+0000","last_clean":"2023-07-06T20:17:31.555949+0000","last_became_active":"2023-07-06T20:17:27.604118+0000","last_became_peered":"2023-07-06T20:17:27.604118+0000","last_unstale":"2023-07-06T20:17:31.555949+0000","last_undegraded":"2023-07-06T20:17:31.555949+0000","last_fullsized":"2023-07-06T20:17:31.555949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:59:34.337697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556287+0000","last_change":"2023-07-06T20:17:28.571753+0000","last_active":"2023-07-06T20:17:31.556287+0000","last_peered":"2023-07-06T20:17:31.556287+0000","last_clean":"2023-07-06T20:17:31.556287+0000","last_became_active":"2023-07-06T20:17:28.571073+0000","last_became_peered":"2023-07-06T20:17:28.571073+0000","last_unstale":"2023-07-06T20:17:31.556287+0000","last_undegraded":"2023-07-06T20:17:31.556287+0000","last_fullsized":"2023-07-06T20:17:31.556287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:39:21.241942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.617659+0000","last_change":"2023-07-06T20:17:27.594354+0000","last_active":"2023-07-06T20:17:31.617659+0000","last_peered":"2023-07-06T20:17:31.617659+0000","last_clean":"2023-07-06T20:17:31.617659+0000","last_became_active":"2023-07-06T20:17:27.594042+0000","last_became_peered":"2023-07-06T20:17:27.594042+0000","last_unstale":"2023-07-06T20:17:31.617659+0000","last_undegraded":"2023-07-06T20:17:31.617659+0000","last_fullsized":"2023-07-06T20:17:31.617659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:26:46.689491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879415+0000","last_change":"2023-07-06T20:17:28.576170+0000","last_active":"2023-07-06T20:17:29.879415+0000","last_peered":"2023-07-06T20:17:29.879415+0000","last_clean":"2023-07-06T20:17:29.879415+0000","last_became_active":"2023-07-06T20:17:28.575493+0000","last_became_peered":"2023-07-06T20:17:28.575493+0000","last_unstale":"2023-07-06T20:17:29.879415+0000","last_undegraded":"2023-07-06T20:17:29.879415+0000","last_fullsized":"2023-07-06T20:17:29.879415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:52:28.276995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.619859+0000","last_change":"2023-07-06T20:17:27.602984+0000","last_active":"2023-07-06T20:17:31.619859+0000","last_peered":"2023-07-06T20:17:31.619859+0000","last_clean":"2023-07-06T20:17:31.619859+0000","last_became_active":"2023-07-06T20:17:27.602720+0000","last_became_peered":"2023-07-06T20:17:27.602720+0000","last_unstale":"2023-07-06T20:17:31.619859+0000","last_undegraded":"2023-07-06T20:17:31.619859+0000","last_fullsized":"2023-07-06T20:17:31.619859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:51:09.808812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555911+0000","last_change":"2023-07-06T20:17:28.565954+0000","last_active":"2023-07-06T20:17:31.555911+0000","last_peered":"2023-07-06T20:17:31.555911+0000","last_clean":"2023-07-06T20:17:31.555911+0000","last_became_active":"2023-07-06T20:17:28.565300+0000","last_became_peered":"2023-07-06T20:17:28.565300+0000","last_unstale":"2023-07-06T20:17:31.555911+0000","last_undegraded":"2023-07-06T20:17:31.555911+0000","last_fullsized":"2023-07-06T20:17:31.555911+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:06:35.879942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.618718+0000","last_change":"2023-07-06T20:17:27.597859+0000","last_active":"2023-07-06T20:17:31.618718+0000","last_peered":"2023-07-06T20:17:31.618718+0000","last_clean":"2023-07-06T20:17:31.618718+0000","last_became_active":"2023-07-06T20:17:27.597479+0000","last_became_peered":"2023-07-06T20:17:27.597479+0000","last_unstale":"2023-07-06T20:17:31.618718+0000","last_undegraded":"2023-07-06T20:17:31.618718+0000","last_fullsized":"2023-07-06T20:17:31.618718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:18:38.533163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.617499+0000","last_change":"2023-07-06T20:17:28.573069+0000","last_active":"2023-07-06T20:17:31.617499+0000","last_peered":"2023-07-06T20:17:31.617499+0000","last_clean":"2023-07-06T20:17:31.617499+0000","last_became_active":"2023-07-06T20:17:28.572742+0000","last_became_peered":"2023-07-06T20:17:28.572742+0000","last_unstale":"2023-07-06T20:17:31.617499+0000","last_undegraded":"2023-07-06T20:17:31.617499+0000","last_fullsized":"2023-07-06T20:17:31.617499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:29:10.236041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.564683+0000","last_change":"2023-07-06T20:17:27.599259+0000","last_active":"2023-07-06T20:17:29.564683+0000","last_peered":"2023-07-06T20:17:29.564683+0000","last_clean":"2023-07-06T20:17:29.564683+0000","last_became_active":"2023-07-06T20:17:27.598883+0000","last_became_peered":"2023-07-06T20:17:27.598883+0000","last_unstale":"2023-07-06T20:17:29.564683+0000","last_undegraded":"2023-07-06T20:17:29.564683+0000","last_fullsized":"2023-07-06T20:17:29.564683+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:09:03.059986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555763+0000","last_change":"2023-07-06T20:17:28.573874+0000","last_active":"2023-07-06T20:17:31.555763+0000","last_peered":"2023-07-06T20:17:31.555763+0000","last_clean":"2023-07-06T20:17:31.555763+0000","last_became_active":"2023-07-06T20:17:28.573379+0000","last_became_peered":"2023-07-06T20:17:28.573379+0000","last_unstale":"2023-07-06T20:17:31.555763+0000","last_undegraded":"2023-07-06T20:17:31.555763+0000","last_fullsized":"2023-07-06T20:17:31.555763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:53:53.176147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.620120+0000","last_change":"2023-07-06T20:17:27.601474+0000","last_active":"2023-07-06T20:17:31.620120+0000","last_peered":"2023-07-06T20:17:31.620120+0000","last_clean":"2023-07-06T20:17:31.620120+0000","last_became_active":"2023-07-06T20:17:27.601099+0000","last_became_peered":"2023-07-06T20:17:27.601099+0000","last_unstale":"2023-07-06T20:17:31.620120+0000","last_undegraded":"2023-07-06T20:17:31.620120+0000","last_fullsized":"2023-07-06T20:17:31.620120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:54:48.745926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574411+0000","last_change":"2023-07-06T20:17:28.571170+0000","last_active":"2023-07-06T20:17:28.574411+0000","last_peered":"2023-07-06T20:17:28.574411+0000","last_clean":"2023-07-06T20:17:28.574411+0000","last_became_active":"2023-07-06T20:17:28.570874+0000","last_became_peered":"2023-07-06T20:17:28.570874+0000","last_unstale":"2023-07-06T20:17:28.574411+0000","last_undegraded":"2023-07-06T20:17:28.574411+0000","last_fullsized":"2023-07-06T20:17:28.574411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:47:44.919469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.564205+0000","last_change":"2023-07-06T20:17:27.603604+0000","last_active":"2023-07-06T20:17:32.564205+0000","last_peered":"2023-07-06T20:17:32.564205+0000","last_clean":"2023-07-06T20:17:32.564205+0000","last_became_active":"2023-07-06T20:17:27.603015+0000","last_became_peered":"2023-07-06T20:17:27.603015+0000","last_unstale":"2023-07-06T20:17:32.564205+0000","last_undegraded":"2023-07-06T20:17:32.564205+0000","last_fullsized":"2023-07-06T20:17:32.564205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:36:14.981024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574692+0000","last_change":"2023-07-06T20:17:28.567906+0000","last_active":"2023-07-06T20:17:28.574692+0000","last_peered":"2023-07-06T20:17:28.574692+0000","last_clean":"2023-07-06T20:17:28.574692+0000","last_became_active":"2023-07-06T20:17:28.567509+0000","last_became_peered":"2023-07-06T20:17:28.567509+0000","last_unstale":"2023-07-06T20:17:28.574692+0000","last_undegraded":"2023-07-06T20:17:28.574692+0000","last_fullsized":"2023-07-06T20:17:28.574692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:30:16.287812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556464+0000","last_change":"2023-07-06T20:17:27.598323+0000","last_active":"2023-07-06T20:17:31.556464+0000","last_peered":"2023-07-06T20:17:31.556464+0000","last_clean":"2023-07-06T20:17:31.556464+0000","last_became_active":"2023-07-06T20:17:27.598032+0000","last_became_peered":"2023-07-06T20:17:27.598032+0000","last_unstale":"2023-07-06T20:17:31.556464+0000","last_undegraded":"2023-07-06T20:17:31.556464+0000","last_fullsized":"2023-07-06T20:17:31.556464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:47:59.236059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574391+0000","last_change":"2023-07-06T20:17:28.571756+0000","last_active":"2023-07-06T20:17:28.574391+0000","last_peered":"2023-07-06T20:17:28.574391+0000","last_clean":"2023-07-06T20:17:28.574391+0000","last_became_active":"2023-07-06T20:17:28.571440+0000","last_became_peered":"2023-07-06T20:17:28.571440+0000","last_unstale":"2023-07-06T20:17:28.574391+0000","last_undegraded":"2023-07-06T20:17:28.574391+0000","last_fullsized":"2023-07-06T20:17:28.574391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:06.531688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.567014+0000","last_change":"2023-07-06T20:17:27.602620+0000","last_active":"2023-07-06T20:17:29.567014+0000","last_peered":"2023-07-06T20:17:29.567014+0000","last_clean":"2023-07-06T20:17:29.567014+0000","last_became_active":"2023-07-06T20:17:27.602224+0000","last_became_peered":"2023-07-06T20:17:27.602224+0000","last_unstale":"2023-07-06T20:17:29.567014+0000","last_undegraded":"2023-07-06T20:17:29.567014+0000","last_fullsized":"2023-07-06T20:17:29.567014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:16:04.139943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574255+0000","last_change":"2023-07-06T20:17:28.569203+0000","last_active":"2023-07-06T20:17:28.574255+0000","last_peered":"2023-07-06T20:17:28.574255+0000","last_clean":"2023-07-06T20:17:28.574255+0000","last_became_active":"2023-07-06T20:17:28.568948+0000","last_became_peered":"2023-07-06T20:17:28.568948+0000","last_unstale":"2023-07-06T20:17:28.574255+0000","last_undegraded":"2023-07-06T20:17:28.574255+0000","last_fullsized":"2023-07-06T20:17:28.574255+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:30:21.073327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.617865+0000","last_change":"2023-07-06T20:17:28.568829+0000","last_active":"2023-07-06T20:17:31.617865+0000","last_peered":"2023-07-06T20:17:31.617865+0000","last_clean":"2023-07-06T20:17:31.617865+0000","last_became_active":"2023-07-06T20:17:28.568511+0000","last_became_peered":"2023-07-06T20:17:28.568511+0000","last_unstale":"2023-07-06T20:17:31.617865+0000","last_undegraded":"2023-07-06T20:17:31.617865+0000","last_fullsized":"2023-07-06T20:17:31.617865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:02:33.443179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.564861+0000","last_change":"2023-07-06T20:17:28.574919+0000","last_active":"2023-07-06T20:17:29.564861+0000","last_peered":"2023-07-06T20:17:29.564861+0000","last_clean":"2023-07-06T20:17:29.564861+0000","last_became_active":"2023-07-06T20:17:28.574545+0000","last_became_peered":"2023-07-06T20:17:28.574545+0000","last_unstale":"2023-07-06T20:17:29.564861+0000","last_undegraded":"2023-07-06T20:17:29.564861+0000","last_fullsized":"2023-07-06T20:17:29.564861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:45:50.951066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.564636+0000","last_change":"2023-07-06T20:17:28.572264+0000","last_active":"2023-07-06T20:17:29.564636+0000","last_peered":"2023-07-06T20:17:29.564636+0000","last_clean":"2023-07-06T20:17:29.564636+0000","last_became_active":"2023-07-06T20:17:28.571914+0000","last_became_peered":"2023-07-06T20:17:28.571914+0000","last_unstale":"2023-07-06T20:17:29.564636+0000","last_undegraded":"2023-07-06T20:17:29.564636+0000","last_fullsized":"2023-07-06T20:17:29.564636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:23:40.059497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879427+0000","last_change":"2023-07-06T20:17:28.581374+0000","last_active":"2023-07-06T20:17:29.879427+0000","last_peered":"2023-07-06T20:17:29.879427+0000","last_clean":"2023-07-06T20:17:29.879427+0000","last_became_active":"2023-07-06T20:17:28.580995+0000","last_became_peered":"2023-07-06T20:17:28.580995+0000","last_unstale":"2023-07-06T20:17:29.879427+0000","last_undegraded":"2023-07-06T20:17:29.879427+0000","last_fullsized":"2023-07-06T20:17:29.879427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:00:43.859988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.567118+0000","last_change":"2023-07-06T20:17:28.578926+0000","last_active":"2023-07-06T20:17:29.567118+0000","last_peered":"2023-07-06T20:17:29.567118+0000","last_clean":"2023-07-06T20:17:29.567118+0000","last_became_active":"2023-07-06T20:17:28.578620+0000","last_became_peered":"2023-07-06T20:17:28.578620+0000","last_unstale":"2023-07-06T20:17:29.567118+0000","last_undegraded":"2023-07-06T20:17:29.567118+0000","last_fullsized":"2023-07-06T20:17:29.567118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:28:51.567310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.617809+0000","last_change":"2023-07-06T20:17:28.566612+0000","last_active":"2023-07-06T20:17:31.617809+0000","last_peered":"2023-07-06T20:17:31.617809+0000","last_clean":"2023-07-06T20:17:31.617809+0000","last_became_active":"2023-07-06T20:17:28.566288+0000","last_became_peered":"2023-07-06T20:17:28.566288+0000","last_unstale":"2023-07-06T20:17:31.617809+0000","last_undegraded":"2023-07-06T20:17:31.617809+0000","last_fullsized":"2023-07-06T20:17:31.617809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:58:57.543076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.565042+0000","last_change":"2023-07-06T20:17:28.567965+0000","last_active":"2023-07-06T20:17:29.565042+0000","last_peered":"2023-07-06T20:17:29.565042+0000","last_clean":"2023-07-06T20:17:29.565042+0000","last_became_active":"2023-07-06T20:17:28.567618+0000","last_became_peered":"2023-07-06T20:17:28.567618+0000","last_unstale":"2023-07-06T20:17:29.565042+0000","last_undegraded":"2023-07-06T20:17:29.565042+0000","last_fullsized":"2023-07-06T20:17:29.565042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:25:08.361862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556395+0000","last_change":"2023-07-06T20:17:28.571934+0000","last_active":"2023-07-06T20:17:31.556395+0000","last_peered":"2023-07-06T20:17:31.556395+0000","last_clean":"2023-07-06T20:17:31.556395+0000","last_became_active":"2023-07-06T20:17:28.571472+0000","last_became_peered":"2023-07-06T20:17:28.571472+0000","last_unstale":"2023-07-06T20:17:31.556395+0000","last_undegraded":"2023-07-06T20:17:31.556395+0000","last_fullsized":"2023-07-06T20:17:31.556395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:21:18.002079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556069+0000","last_change":"2023-07-06T20:17:28.571904+0000","last_active":"2023-07-06T20:17:31.556069+0000","last_peered":"2023-07-06T20:17:31.556069+0000","last_clean":"2023-07-06T20:17:31.556069+0000","last_became_active":"2023-07-06T20:17:28.571375+0000","last_became_peered":"2023-07-06T20:17:28.571375+0000","last_unstale":"2023-07-06T20:17:31.556069+0000","last_undegraded":"2023-07-06T20:17:31.556069+0000","last_fullsized":"2023-07-06T20:17:31.556069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:21:22.902205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555529+0000","last_change":"2023-07-06T20:17:28.569139+0000","last_active":"2023-07-06T20:17:31.555529+0000","last_peered":"2023-07-06T20:17:31.555529+0000","last_clean":"2023-07-06T20:17:31.555529+0000","last_became_active":"2023-07-06T20:17:28.568828+0000","last_became_peered":"2023-07-06T20:17:28.568828+0000","last_unstale":"2023-07-06T20:17:31.555529+0000","last_undegraded":"2023-07-06T20:17:31.555529+0000","last_fullsized":"2023-07-06T20:17:31.555529+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:47:40.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:30.075587+0000","last_change":"2023-07-06T20:17:28.562352+0000","last_active":"2023-07-06T20:17:30.075587+0000","last_peered":"2023-07-06T20:17:30.075587+0000","last_clean":"2023-07-06T20:17:30.075587+0000","last_became_active":"2023-07-06T20:17:28.562128+0000","last_became_peered":"2023-07-06T20:17:28.562128+0000","last_unstale":"2023-07-06T20:17:30.075587+0000","last_undegraded":"2023-07-06T20:17:30.075587+0000","last_fullsized":"2023-07-06T20:17:30.075587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:48:50.486178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.566941+0000","last_change":"2023-07-06T20:17:28.578940+0000","last_active":"2023-07-06T20:17:29.566941+0000","last_peered":"2023-07-06T20:17:29.566941+0000","last_clean":"2023-07-06T20:17:29.566941+0000","last_became_active":"2023-07-06T20:17:28.578660+0000","last_became_peered":"2023-07-06T20:17:28.578660+0000","last_unstale":"2023-07-06T20:17:29.566941+0000","last_undegraded":"2023-07-06T20:17:29.566941+0000","last_fullsized":"2023-07-06T20:17:29.566941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:51:32.527059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.617356+0000","last_change":"2023-07-06T20:17:28.573964+0000","last_active":"2023-07-06T20:17:31.617356+0000","last_peered":"2023-07-06T20:17:31.617356+0000","last_clean":"2023-07-06T20:17:31.617356+0000","last_became_active":"2023-07-06T20:17:28.573618+0000","last_became_peered":"2023-07-06T20:17:28.573618+0000","last_unstale":"2023-07-06T20:17:31.617356+0000","last_undegraded":"2023-07-06T20:17:31.617356+0000","last_fullsized":"2023-07-06T20:17:31.617356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:42:04.680868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":39177,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":19,"num_read_kb":32,"num_write":115,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":46605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449101,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":417248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]}]},{"osd":10,"up_from":71,"seq":304942678033,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1548288,"data_stored":1037158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":1,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":1.131},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.02}]},{"osd":7,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.71999999999999997}]},{"osd":9,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":5.4219999999999997,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.85499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":9,"up_from":66,"seq":283467841556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":426602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60199999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.66400000000000003}]},{"osd":8,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.90600000000000003}]},{"osd":10,"last update":"Thu Jul 6 20:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.93700000000000006}]}]},{"osd":8,"up_from":58,"seq":249108103193,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":426163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.441}]},{"osd":1,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.498},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.622},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.9139999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.64800000000000002}]},{"osd":9,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.75},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.73499999999999999}]},{"osd":10,"last update":"Thu Jul 6 20:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.70399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332126,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28392,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716760,"statfs":{"total":95995035648,"available":95965962240,"internally_reserved":0,"allocated":1474560,"data_stored":1008198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51300000000000001,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85299999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.38100000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.41799999999999998,"15min":0.40899999999999997},"min":{"1min":0.29499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.437},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77900000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83799999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.57799999999999996,"15min":0.56399999999999995},"min":{"1min":0.54000000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68700000000000006,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85099999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.378}]},{"osd":5,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84699999999999998,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.498,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.46700000000000003}]},{"osd":6,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.53400000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.54900000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88300000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.55900000000000005}]},{"osd":8,"last update":"Thu Jul 6 20:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.5}]},{"osd":9,"last update":"Thu Jul 6 20:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58199999999999996}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54200000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495651,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.48599999999999999,"15min":0.47199999999999998},"min":{"1min":0.40799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.66700000000000004,"15min":0.65200000000000002},"min":{"1min":0.57299999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.504,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.85899999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45200000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.497},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0509999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.74299999999999999,"15min":0.72699999999999998},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.181,"5min":1.393,"15min":1.393},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.77100000000000002,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jul 6 20:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.78500000000000003,"15min":0.77500000000000002},"min":{"1min":0.60699999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.78800000000000003},"min":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.55600000000000005}]},{"osd":8,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.93799999999999994}]},{"osd":9,"last update":"Thu Jul 6 20:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.66900000000000004}]},{"osd":10,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.752},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.63900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":427382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.504,"15min":0.47599999999999998},"min":{"1min":0.34799999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.78600000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.53900000000000003,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.248,"15min":0.248},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.52900000000000003,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.95699999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.53700000000000003,"15min":0.501},"min":{"1min":0.43099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.871,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.875}]},{"osd":3,"last update":"Thu Jul 6 20:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.35499999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.85299999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57899999999999996,"15min":0.55100000000000005},"min":{"1min":0.44900000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.876,"15min":0.876},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jul 6 20:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.40400000000000003,"15min":0.38400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.433,"15min":0.41299999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41299999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.52700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0169999999999999,"5min":1.042,"15min":1.042},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jul 6 20:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.438,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.43099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90500000000000003,"5min":1.026,"15min":1.026},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jul 6 20:17:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.32600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.39900000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.59699999999999998}]},{"osd":8,"last update":"Thu Jul 6 20:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.90500000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66600000000000004}]},{"osd":10,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.67300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738431,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.0229999999999999,"5min":1.778,"15min":1.778},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60699999999999998,"15min":0.57199999999999995},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85399999999999998,"5min":4.1379999999999999,"15min":4.1379999999999999},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61399999999999999,"15min":0.58199999999999996},"min":{"1min":0.45100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85399999999999998,"5min":1.085,"15min":1.085},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.622,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":1.258,"15min":1.258},"last":0.84399999999999997}]},{"osd":3,"last update":"Thu Jul 6 20:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72199999999999998,"15min":0.74399999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.046,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":0.755}]},{"osd":5,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65400000000000003,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0389999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0069999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.79400000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.103,"5min":1.319,"15min":1.319},"last":0.77100000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.03,"5min":1.131,"15min":1.131},"last":0.505},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79700000000000004,"15min":0.79600000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.179,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.84399999999999997}]},{"osd":8,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70399999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.83099999999999996}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.93799999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":428684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55800000000000005,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.88700000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.56000000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69399999999999995,"5min":0.876,"15min":0.876},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.32100000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:16:45 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.995,"15min":0.995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.626,"15min":0.626},"min":{"1min":0.55900000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84499999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47199999999999998,"15min":0.45200000000000001},"min":{"1min":0.33500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.47299999999999998,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.34499999999999997}]},{"osd":6,"last update":"Thu Jul 6 20:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.64900000000000002,"15min":0.63800000000000001},"min":{"1min":0.41899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.65900000000000003,"15min":0.64500000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.621}]},{"osd":7,"last update":"Thu Jul 6 20:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.624},"min":{"1min":0.435,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81899999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.40100000000000002}]},{"osd":9,"last update":"Thu Jul 6 20:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.58099999999999996}]},{"osd":10,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.75},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.77000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":426348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.313,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.41999999999999998,"15min":0.39800000000000002},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.60899999999999999,"15min":0.56899999999999995},"min":{"1min":0.45600000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.6360000000000001,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.752},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.83699999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.65000000000000002,"15min":0.61599999999999999},"min":{"1min":0.48299999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.6720000000000002,"5min":3.6720000000000002,"15min":3.6720000000000002},"last":0.79000000000000004}]},{"osd":4,"last update":"Thu Jul 6 20:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.64600000000000002,"15min":0.58899999999999997},"min":{"1min":0.442,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99199999999999999,"5min":3.6179999999999999,"15min":3.6179999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.60899999999999999,"15min":0.57799999999999996},"min":{"1min":0.52300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.97799999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.65000000000000002,"15min":0.624},"min":{"1min":0.56899999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.501,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.69199999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.876}]},{"osd":8,"last update":"Thu Jul 6 20:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.58299999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:17:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.69999999999999996}]},{"osd":10,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.96199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920812,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.55200000000000005,"15min":0.501},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":6.1539999999999999,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.51900000000000002,"15min":0.49299999999999999},"min":{"1min":0.45700000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.35099999999999998},"min":{"1min":0.255,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63500000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.501},{"interface":"front","average":{"1min":0.434,"5min":0.376,"15min":0.35099999999999998},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.71199999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.38500000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85899999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.377,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88300000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.58699999999999997,"15min":0.52500000000000002},"min":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":5.593,"15min":5.593},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59599999999999997,"15min":0.54500000000000004},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":5.5529999999999999,"15min":5.5529999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jul 6 20:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.55600000000000005},"min":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89800000000000002,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.90900000000000003}]},{"osd":6,"last update":"Thu Jul 6 20:16:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64100000000000001,"15min":0.63700000000000001},"min":{"1min":0.51200000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0149999999999999,"5min":1.109,"15min":1.109},"last":0.628},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.109,"5min":1.163,"15min":1.163},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.55100000000000005,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.371,"5min":0.314,"15min":0.314},"max":{"1min":0.89100000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.84199999999999997}]},{"osd":8,"last update":"Thu Jul 6 20:16:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.93700000000000006}]},{"osd":9,"last update":"Thu Jul 6 20:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.87},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.88100000000000001}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.89700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757288,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1507328,"data_stored":1017283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80600000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69399999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.309}]},{"osd":1,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78100000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40300000000000002,"15min":0.39800000000000002},"min":{"1min":0.29299999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.436,"15min":0.42999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.26000000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76200000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.434},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83299999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.32900000000000001}]},{"osd":6,"last update":"Thu Jul 6 20:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.622,"15min":0.622},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64300000000000002,"15min":0.63500000000000001},"min":{"1min":0.47399999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.45600000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70099999999999996,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.498}]},{"osd":8,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.47199999999999998}]},{"osd":9,"last update":"Thu Jul 6 20:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.54100000000000004}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.58399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":38157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:17:36.883 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:17:36.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:36 smithi098 ceph-mon[128968]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:17:36.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:36 smithi098 ceph-mon[128968]: pgmap v272: 97 pgs: 97 active+clean; 615 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9.1 KiB/s wr, 17 op/s 2023-07-06T20:17:36.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:36 smithi098 ceph-mon[128968]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-06T20:17:36.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:36 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2964499470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:36.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:36 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/4199872128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:37.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:36 smithi156 ceph-mon[129280]: from='client.24676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:17:37.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:36 smithi156 ceph-mon[129280]: pgmap v272: 97 pgs: 97 active+clean; 615 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9.1 KiB/s wr, 17 op/s 2023-07-06T20:17:37.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:36 smithi156 ceph-mon[129280]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-06T20:17:37.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:36 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2964499470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:17:37.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:36 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/4199872128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:37.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.107+0000 7f850a9ff700 1 -- 172.21.15.69:0/12730169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85041058c0 msgr2=0x7f8504107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:37.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.107+0000 7f850a9ff700 1 --2- 172.21.15.69:0/12730169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85041058c0 0x7f8504107d20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00b0a0 tx=0x7f84fc00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:17:37.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.107+0000 7f850a9ff700 1 -- 172.21.15.69:0/12730169 shutdown_connections 2023-07-06T20:17:37.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.107+0000 7f850a9ff700 1 --2- 172.21.15.69:0/12730169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85041058c0 0x7f8504107d20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:37.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.107+0000 7f850a9ff700 1 --2- 172.21.15.69:0/12730169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8504100680 0x7f8504102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:37.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.107+0000 7f850a9ff700 1 --2- 172.21.15.69:0/12730169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8504102fa0 0x7f8504105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:37.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.107+0000 7f850a9ff700 1 -- 172.21.15.69:0/12730169 >> 172.21.15.69:0/12730169 conn(0x7f85040fa100 msgr2=0x7f85040fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:37.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.107+0000 7f850a9ff700 1 -- 172.21.15.69:0/12730169 shutdown_connections 2023-07-06T20:17:37.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.107+0000 7f850a9ff700 1 -- 172.21.15.69:0/12730169 wait complete. 2023-07-06T20:17:37.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.108+0000 7f850a9ff700 1 Processor -- start 2023-07-06T20:17:37.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.108+0000 7f850a9ff700 1 -- start start 2023-07-06T20:17:37.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.108+0000 7f850a9ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8504100680 0x7f8504197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:37.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.108+0000 7f850a9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8504102fa0 0x7f8504198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:37.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.108+0000 7f850a9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85041058c0 0x7f850419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:37.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.108+0000 7f850a9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850419f760 con 0x7f85041058c0 2023-07-06T20:17:37.115 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.108+0000 7f850a9ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850419f8d0 con 0x7f8504100680 2023-07-06T20:17:37.115 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.108+0000 7f850a9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850419fa90 con 0x7f8504102fa0 2023-07-06T20:17:37.115 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f8508f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85041058c0 0x7f850419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:37.115 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f8508f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85041058c0 0x7f850419f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51974/0 (socket says 172.21.15.69:51974) 2023-07-06T20:17:37.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f8508f9c700 1 -- 172.21.15.69:0/3539899865 learned_addr learned my addr 172.21.15.69:0/3539899865 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:37.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f8503fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8504100680 0x7f8504197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:37.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f84fbfff700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8504102fa0 0x7f8504198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:37.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f8508f9c700 1 -- 172.21.15.69:0/3539899865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8504102fa0 msgr2=0x7f8504198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:37.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f8508f9c700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8504102fa0 0x7f8504198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:37.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f8508f9c700 1 -- 172.21.15.69:0/3539899865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8504100680 msgr2=0x7f8504197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:37.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f8508f9c700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8504100680 0x7f8504197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:37.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f8508f9c700 1 -- 172.21.15.69:0/3539899865 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc009d00 con 0x7f85041058c0 2023-07-06T20:17:37.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f8508f9c700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85041058c0 0x7f850419f190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009fd0 tx=0x7f84fc00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:37.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f8501ffb700 1 -- 172.21.15.69:0/3539899865 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00e040 con 0x7f85041058c0 2023-07-06T20:17:37.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f8501ffb700 1 -- 172.21.15.69:0/3539899865 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84fc0094a0 con 0x7f85041058c0 2023-07-06T20:17:37.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.109+0000 7f850a9ff700 1 -- 172.21.15.69:0/3539899865 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850419fd10 con 0x7f85041058c0 2023-07-06T20:17:37.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.110+0000 7f8501ffb700 1 -- 172.21.15.69:0/3539899865 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc01b6a0 con 0x7f85041058c0 2023-07-06T20:17:37.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.110+0000 7f850a9ff700 1 -- 172.21.15.69:0/3539899865 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85041a0130 con 0x7f85041058c0 2023-07-06T20:17:37.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.111+0000 7f850a9ff700 1 -- 172.21.15.69:0/3539899865 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8504107430 con 0x7f85041058c0 2023-07-06T20:17:37.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.112+0000 7f8501ffb700 1 -- 172.21.15.69:0/3539899865 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f84fc01d470 con 0x7f85041058c0 2023-07-06T20:17:37.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.115+0000 7f8501ffb700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f84f00a12a0 0x7f84f00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:37.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.115+0000 7f8501ffb700 1 -- 172.21.15.69:0/3539899865 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f84fc0cdc20 con 0x7f85041058c0 2023-07-06T20:17:37.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.115+0000 7f8503fff700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f84f00a12a0 0x7f84f00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:37.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.115+0000 7f8503fff700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f84f00a12a0 0x7f84f00a3750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f85041990f0 tx=0x7f84f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:37.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.118+0000 7f8501ffb700 1 -- 172.21.15.69:0/3539899865 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f84fc097450 con 0x7f85041058c0 2023-07-06T20:17:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.314+0000 7f850a9ff700 1 -- 172.21.15.69:0/3539899865 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8504060a80 con 0x7f84f00a12a0 2023-07-06T20:17:37.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.324+0000 7f8501ffb700 1 -- 172.21.15.69:0/3539899865 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335525 (secure 0 0 0) 0x7f8504060a80 con 0x7f84f00a12a0 2023-07-06T20:17:37.330 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:37.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.328+0000 7f850a9ff700 1 -- 172.21.15.69:0/3539899865 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f84f00a12a0 msgr2=0x7f84f00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:37.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.328+0000 7f850a9ff700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f84f00a12a0 0x7f84f00a3750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f85041990f0 tx=0x7f84f4009260 comp rx=0 tx=0).stop 2023-07-06T20:17:37.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.328+0000 7f850a9ff700 1 -- 172.21.15.69:0/3539899865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85041058c0 msgr2=0x7f850419f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:37.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.328+0000 7f850a9ff700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85041058c0 0x7f850419f190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009fd0 tx=0x7f84fc00bce0 comp rx=0 tx=0).stop 2023-07-06T20:17:37.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.328+0000 7f850a9ff700 1 -- 172.21.15.69:0/3539899865 shutdown_connections 2023-07-06T20:17:37.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.328+0000 7f850a9ff700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f84f00a12a0 0x7f84f00a3750 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:37.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.328+0000 7f850a9ff700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85041058c0 0x7f850419f190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:37.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.328+0000 7f850a9ff700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8504100680 0x7f8504197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:37.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.328+0000 7f850a9ff700 1 --2- 172.21.15.69:0/3539899865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8504102fa0 0x7f8504198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:37.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.328+0000 7f850a9ff700 1 -- 172.21.15.69:0/3539899865 >> 172.21.15.69:0/3539899865 conn(0x7f85040fa100 msgr2=0x7f85040fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:37.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.328+0000 7f850a9ff700 1 -- 172.21.15.69:0/3539899865 shutdown_connections 2023-07-06T20:17:37.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:37.329+0000 7f850a9ff700 1 -- 172.21.15.69:0/3539899865 wait complete. 2023-07-06T20:17:37.333 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-07-06T20:17:37.349 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2023-07-06T20:17:35.518855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629668,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":145,"num_read_kb":141,"num_write":297,"num_write_kb":3010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":268,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335756,"kb_used_data":12332,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606068,"statfs":{"total":1151940427776,"available":1151596613632,"internally_reserved":0,"allocated":12627968,"data_stored":6892722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39300,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":19,"num_read_kb":32,"num_write":116,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.932168"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.618149+0000","last_change":"2023-07-06T20:17:28.575520+0000","last_active":"2023-07-06T20:17:31.618149+0000","last_peered":"2023-07-06T20:17:31.618149+0000","last_clean":"2023-07-06T20:17:31.618149+0000","last_became_active":"2023-07-06T20:17:28.575209+0000","last_became_peered":"2023-07-06T20:17:28.575209+0000","last_unstale":"2023-07-06T20:17:31.618149+0000","last_undegraded":"2023-07-06T20:17:31.618149+0000","last_fullsized":"2023-07-06T20:17:31.618149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:32:56.425676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879329+0000","last_change":"2023-07-06T20:17:28.572547+0000","last_active":"2023-07-06T20:17:29.879329+0000","last_peered":"2023-07-06T20:17:29.879329+0000","last_clean":"2023-07-06T20:17:29.879329+0000","last_became_active":"2023-07-06T20:17:28.572150+0000","last_became_peered":"2023-07-06T20:17:28.572150+0000","last_unstale":"2023-07-06T20:17:29.879329+0000","last_undegraded":"2023-07-06T20:17:29.879329+0000","last_fullsized":"2023-07-06T20:17:29.879329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:32:16.930127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879457+0000","last_change":"2023-07-06T20:17:28.574926+0000","last_active":"2023-07-06T20:17:29.879457+0000","last_peered":"2023-07-06T20:17:29.879457+0000","last_clean":"2023-07-06T20:17:29.879457+0000","last_became_active":"2023-07-06T20:17:28.574602+0000","last_became_peered":"2023-07-06T20:17:28.574602+0000","last_unstale":"2023-07-06T20:17:29.879457+0000","last_undegraded":"2023-07-06T20:17:29.879457+0000","last_fullsized":"2023-07-06T20:17:29.879457+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:10:07.997134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.565068+0000","last_change":"2023-07-06T20:17:28.571533+0000","last_active":"2023-07-06T20:17:29.565068+0000","last_peered":"2023-07-06T20:17:29.565068+0000","last_clean":"2023-07-06T20:17:29.565068+0000","last_became_active":"2023-07-06T20:17:28.571099+0000","last_became_peered":"2023-07-06T20:17:28.571099+0000","last_unstale":"2023-07-06T20:17:29.565068+0000","last_undegraded":"2023-07-06T20:17:29.565068+0000","last_fullsized":"2023-07-06T20:17:29.565068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:28:20.866578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556729+0000","last_change":"2023-07-06T20:17:28.572100+0000","last_active":"2023-07-06T20:17:31.556729+0000","last_peered":"2023-07-06T20:17:31.556729+0000","last_clean":"2023-07-06T20:17:31.556729+0000","last_became_active":"2023-07-06T20:17:28.571772+0000","last_became_peered":"2023-07-06T20:17:28.571772+0000","last_unstale":"2023-07-06T20:17:31.556729+0000","last_undegraded":"2023-07-06T20:17:31.556729+0000","last_fullsized":"2023-07-06T20:17:31.556729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:49:44.228120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556089+0000","last_change":"2023-07-06T20:17:28.577816+0000","last_active":"2023-07-06T20:17:31.556089+0000","last_peered":"2023-07-06T20:17:31.556089+0000","last_clean":"2023-07-06T20:17:31.556089+0000","last_became_active":"2023-07-06T20:17:28.577382+0000","last_became_peered":"2023-07-06T20:17:28.577382+0000","last_unstale":"2023-07-06T20:17:31.556089+0000","last_undegraded":"2023-07-06T20:17:31.556089+0000","last_fullsized":"2023-07-06T20:17:31.556089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:13:43.603856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574702+0000","last_change":"2023-07-06T20:17:28.570415+0000","last_active":"2023-07-06T20:17:28.574702+0000","last_peered":"2023-07-06T20:17:28.574702+0000","last_clean":"2023-07-06T20:17:28.574702+0000","last_became_active":"2023-07-06T20:17:28.570091+0000","last_became_peered":"2023-07-06T20:17:28.570091+0000","last_unstale":"2023-07-06T20:17:28.574702+0000","last_undegraded":"2023-07-06T20:17:28.574702+0000","last_fullsized":"2023-07-06T20:17:28.574702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:06:13.149895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555505+0000","last_change":"2023-07-06T20:17:28.568370+0000","last_active":"2023-07-06T20:17:31.555505+0000","last_peered":"2023-07-06T20:17:31.555505+0000","last_clean":"2023-07-06T20:17:31.555505+0000","last_became_active":"2023-07-06T20:17:28.568032+0000","last_became_peered":"2023-07-06T20:17:28.568032+0000","last_unstale":"2023-07-06T20:17:31.555505+0000","last_undegraded":"2023-07-06T20:17:31.555505+0000","last_fullsized":"2023-07-06T20:17:31.555505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:26:23.998744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:30.075961+0000","last_change":"2023-07-06T20:17:28.563386+0000","last_active":"2023-07-06T20:17:30.075961+0000","last_peered":"2023-07-06T20:17:30.075961+0000","last_clean":"2023-07-06T20:17:30.075961+0000","last_became_active":"2023-07-06T20:17:28.562864+0000","last_became_peered":"2023-07-06T20:17:28.562864+0000","last_unstale":"2023-07-06T20:17:30.075961+0000","last_undegraded":"2023-07-06T20:17:30.075961+0000","last_fullsized":"2023-07-06T20:17:30.075961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:33:27.513610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556379+0000","last_change":"2023-07-06T20:17:28.578016+0000","last_active":"2023-07-06T20:17:31.556379+0000","last_peered":"2023-07-06T20:17:31.556379+0000","last_clean":"2023-07-06T20:17:31.556379+0000","last_became_active":"2023-07-06T20:17:28.577305+0000","last_became_peered":"2023-07-06T20:17:28.577305+0000","last_unstale":"2023-07-06T20:17:31.556379+0000","last_undegraded":"2023-07-06T20:17:31.556379+0000","last_fullsized":"2023-07-06T20:17:31.556379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:59:29.567212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574712+0000","last_change":"2023-07-06T20:17:28.569683+0000","last_active":"2023-07-06T20:17:28.574712+0000","last_peered":"2023-07-06T20:17:28.574712+0000","last_clean":"2023-07-06T20:17:28.574712+0000","last_became_active":"2023-07-06T20:17:28.569070+0000","last_became_peered":"2023-07-06T20:17:28.569070+0000","last_unstale":"2023-07-06T20:17:28.574712+0000","last_undegraded":"2023-07-06T20:17:28.574712+0000","last_fullsized":"2023-07-06T20:17:28.574712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:45:32.404472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.565041+0000","last_change":"2023-07-06T20:17:28.579463+0000","last_active":"2023-07-06T20:17:29.565041+0000","last_peered":"2023-07-06T20:17:29.565041+0000","last_clean":"2023-07-06T20:17:29.565041+0000","last_became_active":"2023-07-06T20:17:28.579141+0000","last_became_peered":"2023-07-06T20:17:28.579141+0000","last_unstale":"2023-07-06T20:17:29.565041+0000","last_undegraded":"2023-07-06T20:17:29.565041+0000","last_fullsized":"2023-07-06T20:17:29.565041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:04:10.705471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.566959+0000","last_change":"2023-07-06T20:17:28.570235+0000","last_active":"2023-07-06T20:17:29.566959+0000","last_peered":"2023-07-06T20:17:29.566959+0000","last_clean":"2023-07-06T20:17:29.566959+0000","last_became_active":"2023-07-06T20:17:28.569780+0000","last_became_peered":"2023-07-06T20:17:28.569780+0000","last_unstale":"2023-07-06T20:17:29.566959+0000","last_undegraded":"2023-07-06T20:17:29.566959+0000","last_fullsized":"2023-07-06T20:17:29.566959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:26:47.226331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879157+0000","last_change":"2023-07-06T20:17:28.572197+0000","last_active":"2023-07-06T20:17:29.879157+0000","last_peered":"2023-07-06T20:17:29.879157+0000","last_clean":"2023-07-06T20:17:29.879157+0000","last_became_active":"2023-07-06T20:17:28.571816+0000","last_became_peered":"2023-07-06T20:17:28.571816+0000","last_unstale":"2023-07-06T20:17:29.879157+0000","last_undegraded":"2023-07-06T20:17:29.879157+0000","last_fullsized":"2023-07-06T20:17:29.879157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:56:16.289124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879101+0000","last_change":"2023-07-06T20:17:28.574250+0000","last_active":"2023-07-06T20:17:29.879101+0000","last_peered":"2023-07-06T20:17:29.879101+0000","last_clean":"2023-07-06T20:17:29.879101+0000","last_became_active":"2023-07-06T20:17:28.573916+0000","last_became_peered":"2023-07-06T20:17:28.573916+0000","last_unstale":"2023-07-06T20:17:29.879101+0000","last_undegraded":"2023-07-06T20:17:29.879101+0000","last_fullsized":"2023-07-06T20:17:29.879101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:55:31.687080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:30.075428+0000","last_change":"2023-07-06T20:17:28.573552+0000","last_active":"2023-07-06T20:17:30.075428+0000","last_peered":"2023-07-06T20:17:30.075428+0000","last_clean":"2023-07-06T20:17:30.075428+0000","last_became_active":"2023-07-06T20:17:28.573253+0000","last_became_peered":"2023-07-06T20:17:28.573253+0000","last_unstale":"2023-07-06T20:17:30.075428+0000","last_undegraded":"2023-07-06T20:17:30.075428+0000","last_fullsized":"2023-07-06T20:17:30.075428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:26:51.333395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.618223+0000","last_change":"2023-07-06T20:17:28.572425+0000","last_active":"2023-07-06T20:17:31.618223+0000","last_peered":"2023-07-06T20:17:31.618223+0000","last_clean":"2023-07-06T20:17:31.618223+0000","last_became_active":"2023-07-06T20:17:28.572104+0000","last_became_peered":"2023-07-06T20:17:28.572104+0000","last_unstale":"2023-07-06T20:17:31.618223+0000","last_undegraded":"2023-07-06T20:17:31.618223+0000","last_fullsized":"2023-07-06T20:17:31.618223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:08:34.890550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555809+0000","last_change":"2023-07-06T20:17:28.573814+0000","last_active":"2023-07-06T20:17:31.555809+0000","last_peered":"2023-07-06T20:17:31.555809+0000","last_clean":"2023-07-06T20:17:31.555809+0000","last_became_active":"2023-07-06T20:17:28.573470+0000","last_became_peered":"2023-07-06T20:17:28.573470+0000","last_unstale":"2023-07-06T20:17:31.555809+0000","last_undegraded":"2023-07-06T20:17:31.555809+0000","last_fullsized":"2023-07-06T20:17:31.555809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:49:01.038082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.564943+0000","last_change":"2023-07-06T20:17:28.571639+0000","last_active":"2023-07-06T20:17:29.564943+0000","last_peered":"2023-07-06T20:17:29.564943+0000","last_clean":"2023-07-06T20:17:29.564943+0000","last_became_active":"2023-07-06T20:17:28.571356+0000","last_became_peered":"2023-07-06T20:17:28.571356+0000","last_unstale":"2023-07-06T20:17:29.564943+0000","last_undegraded":"2023-07-06T20:17:29.564943+0000","last_fullsized":"2023-07-06T20:17:29.564943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:20:33.572044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556007+0000","last_change":"2023-07-06T20:17:27.603771+0000","last_active":"2023-07-06T20:17:31.556007+0000","last_peered":"2023-07-06T20:17:31.556007+0000","last_clean":"2023-07-06T20:17:31.556007+0000","last_became_active":"2023-07-06T20:17:27.603404+0000","last_became_peered":"2023-07-06T20:17:27.603404+0000","last_unstale":"2023-07-06T20:17:31.556007+0000","last_undegraded":"2023-07-06T20:17:31.556007+0000","last_fullsized":"2023-07-06T20:17:31.556007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:14:29.214268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555982+0000","last_change":"2023-07-06T20:17:28.568371+0000","last_active":"2023-07-06T20:17:31.555982+0000","last_peered":"2023-07-06T20:17:31.555982+0000","last_clean":"2023-07-06T20:17:31.555982+0000","last_became_active":"2023-07-06T20:17:28.568034+0000","last_became_peered":"2023-07-06T20:17:28.568034+0000","last_unstale":"2023-07-06T20:17:31.555982+0000","last_undegraded":"2023-07-06T20:17:31.555982+0000","last_fullsized":"2023-07-06T20:17:31.555982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:16:18.752577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556266+0000","last_change":"2023-07-06T20:17:27.609833+0000","last_active":"2023-07-06T20:17:31.556266+0000","last_peered":"2023-07-06T20:17:31.556266+0000","last_clean":"2023-07-06T20:17:31.556266+0000","last_became_active":"2023-07-06T20:17:27.609520+0000","last_became_peered":"2023-07-06T20:17:27.609520+0000","last_unstale":"2023-07-06T20:17:31.556266+0000","last_undegraded":"2023-07-06T20:17:31.556266+0000","last_fullsized":"2023-07-06T20:17:31.556266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:40:41.429428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879431+0000","last_change":"2023-07-06T20:17:28.574520+0000","last_active":"2023-07-06T20:17:29.879431+0000","last_peered":"2023-07-06T20:17:29.879431+0000","last_clean":"2023-07-06T20:17:29.879431+0000","last_became_active":"2023-07-06T20:17:28.574158+0000","last_became_peered":"2023-07-06T20:17:28.574158+0000","last_unstale":"2023-07-06T20:17:29.879431+0000","last_undegraded":"2023-07-06T20:17:29.879431+0000","last_fullsized":"2023-07-06T20:17:29.879431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:42:31.557796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.562363+0000","last_change":"2023-07-06T20:17:27.601612+0000","last_active":"2023-07-06T20:17:32.562363+0000","last_peered":"2023-07-06T20:17:32.562363+0000","last_clean":"2023-07-06T20:17:32.562363+0000","last_became_active":"2023-07-06T20:17:27.601313+0000","last_became_peered":"2023-07-06T20:17:27.601313+0000","last_unstale":"2023-07-06T20:17:32.562363+0000","last_undegraded":"2023-07-06T20:17:32.562363+0000","last_fullsized":"2023-07-06T20:17:32.562363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:45:02.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:30.075593+0000","last_change":"2023-07-06T20:17:28.566389+0000","last_active":"2023-07-06T20:17:30.075593+0000","last_peered":"2023-07-06T20:17:30.075593+0000","last_clean":"2023-07-06T20:17:30.075593+0000","last_became_active":"2023-07-06T20:17:28.566094+0000","last_became_peered":"2023-07-06T20:17:28.566094+0000","last_unstale":"2023-07-06T20:17:30.075593+0000","last_undegraded":"2023-07-06T20:17:30.075593+0000","last_fullsized":"2023-07-06T20:17:30.075593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:47:12.929391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556276+0000","last_change":"2023-07-06T20:17:27.599893+0000","last_active":"2023-07-06T20:17:31.556276+0000","last_peered":"2023-07-06T20:17:31.556276+0000","last_clean":"2023-07-06T20:17:31.556276+0000","last_became_active":"2023-07-06T20:17:27.599544+0000","last_became_peered":"2023-07-06T20:17:27.599544+0000","last_unstale":"2023-07-06T20:17:31.556276+0000","last_undegraded":"2023-07-06T20:17:31.556276+0000","last_fullsized":"2023-07-06T20:17:31.556276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:03:13.210197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556719+0000","last_change":"2023-07-06T20:17:28.575794+0000","last_active":"2023-07-06T20:17:31.556719+0000","last_peered":"2023-07-06T20:17:31.556719+0000","last_clean":"2023-07-06T20:17:31.556719+0000","last_became_active":"2023-07-06T20:17:28.575456+0000","last_became_peered":"2023-07-06T20:17:28.575456+0000","last_unstale":"2023-07-06T20:17:31.556719+0000","last_undegraded":"2023-07-06T20:17:31.556719+0000","last_fullsized":"2023-07-06T20:17:31.556719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:57.974475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.620919+0000","last_change":"2023-07-06T20:17:27.596351+0000","last_active":"2023-07-06T20:17:31.620919+0000","last_peered":"2023-07-06T20:17:31.620919+0000","last_clean":"2023-07-06T20:17:31.620919+0000","last_became_active":"2023-07-06T20:17:27.595982+0000","last_became_peered":"2023-07-06T20:17:27.595982+0000","last_unstale":"2023-07-06T20:17:31.620919+0000","last_undegraded":"2023-07-06T20:17:31.620919+0000","last_fullsized":"2023-07-06T20:17:31.620919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:13:06.029344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555937+0000","last_change":"2023-07-06T20:17:28.570018+0000","last_active":"2023-07-06T20:17:31.555937+0000","last_peered":"2023-07-06T20:17:31.555937+0000","last_clean":"2023-07-06T20:17:31.555937+0000","last_became_active":"2023-07-06T20:17:28.569674+0000","last_became_peered":"2023-07-06T20:17:28.569674+0000","last_unstale":"2023-07-06T20:17:31.555937+0000","last_undegraded":"2023-07-06T20:17:31.555937+0000","last_fullsized":"2023-07-06T20:17:31.555937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:59:52.145494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.563000+0000","last_change":"2023-07-06T20:17:27.602792+0000","last_active":"2023-07-06T20:17:32.563000+0000","last_peered":"2023-07-06T20:17:32.563000+0000","last_clean":"2023-07-06T20:17:32.563000+0000","last_became_active":"2023-07-06T20:17:27.602373+0000","last_became_peered":"2023-07-06T20:17:27.602373+0000","last_unstale":"2023-07-06T20:17:32.563000+0000","last_undegraded":"2023-07-06T20:17:32.563000+0000","last_fullsized":"2023-07-06T20:17:32.563000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:55:12.560893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556254+0000","last_change":"2023-07-06T20:17:28.575249+0000","last_active":"2023-07-06T20:17:31.556254+0000","last_peered":"2023-07-06T20:17:31.556254+0000","last_clean":"2023-07-06T20:17:31.556254+0000","last_became_active":"2023-07-06T20:17:28.574954+0000","last_became_peered":"2023-07-06T20:17:28.574954+0000","last_unstale":"2023-07-06T20:17:31.556254+0000","last_undegraded":"2023-07-06T20:17:31.556254+0000","last_fullsized":"2023-07-06T20:17:31.556254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:36:50.592568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.563699+0000","last_change":"2023-07-06T20:17:27.600552+0000","last_active":"2023-07-06T20:17:32.563699+0000","last_peered":"2023-07-06T20:17:32.563699+0000","last_clean":"2023-07-06T20:17:32.563699+0000","last_became_active":"2023-07-06T20:17:27.600016+0000","last_became_peered":"2023-07-06T20:17:27.600016+0000","last_unstale":"2023-07-06T20:17:32.563699+0000","last_undegraded":"2023-07-06T20:17:32.563699+0000","last_fullsized":"2023-07-06T20:17:32.563699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:42:56.520957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555603+0000","last_change":"2023-07-06T20:17:28.566364+0000","last_active":"2023-07-06T20:17:31.555603+0000","last_peered":"2023-07-06T20:17:31.555603+0000","last_clean":"2023-07-06T20:17:31.555603+0000","last_became_active":"2023-07-06T20:17:28.565474+0000","last_became_peered":"2023-07-06T20:17:28.565474+0000","last_unstale":"2023-07-06T20:17:31.555603+0000","last_undegraded":"2023-07-06T20:17:31.555603+0000","last_fullsized":"2023-07-06T20:17:31.555603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:20:32.320775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555897+0000","last_change":"2023-07-06T20:17:27.598606+0000","last_active":"2023-07-06T20:17:31.555897+0000","last_peered":"2023-07-06T20:17:31.555897+0000","last_clean":"2023-07-06T20:17:31.555897+0000","last_became_active":"2023-07-06T20:17:27.598292+0000","last_became_peered":"2023-07-06T20:17:27.598292+0000","last_unstale":"2023-07-06T20:17:31.555897+0000","last_undegraded":"2023-07-06T20:17:31.555897+0000","last_fullsized":"2023-07-06T20:17:31.555897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:32:06.168848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555943+0000","last_change":"2023-07-06T20:17:28.571246+0000","last_active":"2023-07-06T20:17:31.555943+0000","last_peered":"2023-07-06T20:17:31.555943+0000","last_clean":"2023-07-06T20:17:31.555943+0000","last_became_active":"2023-07-06T20:17:28.570846+0000","last_became_peered":"2023-07-06T20:17:28.570846+0000","last_unstale":"2023-07-06T20:17:31.555943+0000","last_undegraded":"2023-07-06T20:17:31.555943+0000","last_fullsized":"2023-07-06T20:17:31.555943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:41:17.390535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:33.558723+0000","last_change":"2023-07-06T20:17:27.595879+0000","last_active":"2023-07-06T20:17:33.558723+0000","last_peered":"2023-07-06T20:17:33.558723+0000","last_clean":"2023-07-06T20:17:33.558723+0000","last_became_active":"2023-07-06T20:17:27.595553+0000","last_became_peered":"2023-07-06T20:17:27.595553+0000","last_unstale":"2023-07-06T20:17:33.558723+0000","last_undegraded":"2023-07-06T20:17:33.558723+0000","last_fullsized":"2023-07-06T20:17:33.558723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:29:30.880922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:30.075914+0000","last_change":"2023-07-06T20:17:28.566816+0000","last_active":"2023-07-06T20:17:30.075914+0000","last_peered":"2023-07-06T20:17:30.075914+0000","last_clean":"2023-07-06T20:17:30.075914+0000","last_became_active":"2023-07-06T20:17:28.566565+0000","last_became_peered":"2023-07-06T20:17:28.566565+0000","last_unstale":"2023-07-06T20:17:30.075914+0000","last_undegraded":"2023-07-06T20:17:30.075914+0000","last_fullsized":"2023-07-06T20:17:30.075914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:52:54.354015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.560374+0000","last_change":"2023-07-06T20:17:27.598572+0000","last_active":"2023-07-06T20:17:28.560374+0000","last_peered":"2023-07-06T20:17:28.560374+0000","last_clean":"2023-07-06T20:17:28.560374+0000","last_became_active":"2023-07-06T20:17:27.598243+0000","last_became_peered":"2023-07-06T20:17:27.598243+0000","last_unstale":"2023-07-06T20:17:28.560374+0000","last_undegraded":"2023-07-06T20:17:28.560374+0000","last_fullsized":"2023-07-06T20:17:28.560374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:29:09.506737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879451+0000","last_change":"2023-07-06T20:17:28.572400+0000","last_active":"2023-07-06T20:17:29.879451+0000","last_peered":"2023-07-06T20:17:29.879451+0000","last_clean":"2023-07-06T20:17:29.879451+0000","last_became_active":"2023-07-06T20:17:28.572047+0000","last_became_peered":"2023-07-06T20:17:28.572047+0000","last_unstale":"2023-07-06T20:17:29.879451+0000","last_undegraded":"2023-07-06T20:17:29.879451+0000","last_fullsized":"2023-07-06T20:17:29.879451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:22:53.203510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556727+0000","last_change":"2023-07-06T20:17:27.603224+0000","last_active":"2023-07-06T20:17:31.556727+0000","last_peered":"2023-07-06T20:17:31.556727+0000","last_clean":"2023-07-06T20:17:31.556727+0000","last_became_active":"2023-07-06T20:17:27.602864+0000","last_became_peered":"2023-07-06T20:17:27.602864+0000","last_unstale":"2023-07-06T20:17:31.556727+0000","last_undegraded":"2023-07-06T20:17:31.556727+0000","last_fullsized":"2023-07-06T20:17:31.556727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:59:59.044611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:30.075651+0000","last_change":"2023-07-06T20:17:28.572683+0000","last_active":"2023-07-06T20:17:30.075651+0000","last_peered":"2023-07-06T20:17:30.075651+0000","last_clean":"2023-07-06T20:17:30.075651+0000","last_became_active":"2023-07-06T20:17:28.572352+0000","last_became_peered":"2023-07-06T20:17:28.572352+0000","last_unstale":"2023-07-06T20:17:30.075651+0000","last_undegraded":"2023-07-06T20:17:30.075651+0000","last_fullsized":"2023-07-06T20:17:30.075651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:03:04.106335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'18","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:35.029036+0000","last_change":"2023-07-06T20:17:27.597285+0000","last_active":"2023-07-06T20:17:35.029036+0000","last_peered":"2023-07-06T20:17:35.029036+0000","last_clean":"2023-07-06T20:17:35.029036+0000","last_became_active":"2023-07-06T20:17:27.596850+0000","last_became_peered":"2023-07-06T20:17:27.596850+0000","last_unstale":"2023-07-06T20:17:35.029036+0000","last_undegraded":"2023-07-06T20:17:35.029036+0000","last_fullsized":"2023-07-06T20:17:35.029036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:51:25.444929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36237,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":22,"num_write":19,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556263+0000","last_change":"2023-07-06T20:17:28.569855+0000","last_active":"2023-07-06T20:17:31.556263+0000","last_peered":"2023-07-06T20:17:31.556263+0000","last_clean":"2023-07-06T20:17:31.556263+0000","last_became_active":"2023-07-06T20:17:28.569495+0000","last_became_peered":"2023-07-06T20:17:28.569495+0000","last_unstale":"2023-07-06T20:17:31.556263+0000","last_undegraded":"2023-07-06T20:17:31.556263+0000","last_fullsized":"2023-07-06T20:17:31.556263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:37:13.491582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.562082+0000","last_change":"2023-07-06T20:17:27.605898+0000","last_active":"2023-07-06T20:17:32.562082+0000","last_peered":"2023-07-06T20:17:32.562082+0000","last_clean":"2023-07-06T20:17:32.562082+0000","last_became_active":"2023-07-06T20:17:27.605605+0000","last_became_peered":"2023-07-06T20:17:27.605605+0000","last_unstale":"2023-07-06T20:17:32.562082+0000","last_undegraded":"2023-07-06T20:17:32.562082+0000","last_fullsized":"2023-07-06T20:17:32.562082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:19:04.240869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.566733+0000","last_change":"2023-07-06T20:17:28.579236+0000","last_active":"2023-07-06T20:17:29.566733+0000","last_peered":"2023-07-06T20:17:29.566733+0000","last_clean":"2023-07-06T20:17:29.566733+0000","last_became_active":"2023-07-06T20:17:28.578854+0000","last_became_peered":"2023-07-06T20:17:28.578854+0000","last_unstale":"2023-07-06T20:17:29.566733+0000","last_undegraded":"2023-07-06T20:17:29.566733+0000","last_fullsized":"2023-07-06T20:17:29.566733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:25:51.789427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.620762+0000","last_change":"2023-07-06T20:17:27.595724+0000","last_active":"2023-07-06T20:17:31.620762+0000","last_peered":"2023-07-06T20:17:31.620762+0000","last_clean":"2023-07-06T20:17:31.620762+0000","last_became_active":"2023-07-06T20:17:27.595360+0000","last_became_peered":"2023-07-06T20:17:27.595360+0000","last_unstale":"2023-07-06T20:17:31.620762+0000","last_undegraded":"2023-07-06T20:17:31.620762+0000","last_fullsized":"2023-07-06T20:17:31.620762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:25:06.238847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574678+0000","last_change":"2023-07-06T20:17:28.573447+0000","last_active":"2023-07-06T20:17:28.574678+0000","last_peered":"2023-07-06T20:17:28.574678+0000","last_clean":"2023-07-06T20:17:28.574678+0000","last_became_active":"2023-07-06T20:17:28.573161+0000","last_became_peered":"2023-07-06T20:17:28.573161+0000","last_unstale":"2023-07-06T20:17:28.574678+0000","last_undegraded":"2023-07-06T20:17:28.574678+0000","last_fullsized":"2023-07-06T20:17:28.574678+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:08:06.240576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.564909+0000","last_change":"2023-07-06T20:17:27.596254+0000","last_active":"2023-07-06T20:17:29.564909+0000","last_peered":"2023-07-06T20:17:29.564909+0000","last_clean":"2023-07-06T20:17:29.564909+0000","last_became_active":"2023-07-06T20:17:27.595178+0000","last_became_peered":"2023-07-06T20:17:27.595178+0000","last_unstale":"2023-07-06T20:17:29.564909+0000","last_undegraded":"2023-07-06T20:17:29.564909+0000","last_fullsized":"2023-07-06T20:17:29.564909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:49.788828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574738+0000","last_change":"2023-07-06T20:17:28.569745+0000","last_active":"2023-07-06T20:17:28.574738+0000","last_peered":"2023-07-06T20:17:28.574738+0000","last_clean":"2023-07-06T20:17:28.574738+0000","last_became_active":"2023-07-06T20:17:28.569468+0000","last_became_peered":"2023-07-06T20:17:28.569468+0000","last_unstale":"2023-07-06T20:17:28.574738+0000","last_undegraded":"2023-07-06T20:17:28.574738+0000","last_fullsized":"2023-07-06T20:17:28.574738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:47:12.397526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.564783+0000","last_change":"2023-07-06T20:17:27.608166+0000","last_active":"2023-07-06T20:17:32.564783+0000","last_peered":"2023-07-06T20:17:32.564783+0000","last_clean":"2023-07-06T20:17:32.564783+0000","last_became_active":"2023-07-06T20:17:27.607904+0000","last_became_peered":"2023-07-06T20:17:27.607904+0000","last_unstale":"2023-07-06T20:17:32.564783+0000","last_undegraded":"2023-07-06T20:17:32.564783+0000","last_fullsized":"2023-07-06T20:17:32.564783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:19:42.369761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556531+0000","last_change":"2023-07-06T20:17:28.575390+0000","last_active":"2023-07-06T20:17:31.556531+0000","last_peered":"2023-07-06T20:17:31.556531+0000","last_clean":"2023-07-06T20:17:31.556531+0000","last_became_active":"2023-07-06T20:17:28.575083+0000","last_became_peered":"2023-07-06T20:17:28.575083+0000","last_unstale":"2023-07-06T20:17:31.556531+0000","last_undegraded":"2023-07-06T20:17:31.556531+0000","last_fullsized":"2023-07-06T20:17:31.556531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:08:34.646494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556326+0000","last_change":"2023-07-06T20:17:27.600891+0000","last_active":"2023-07-06T20:17:31.556326+0000","last_peered":"2023-07-06T20:17:31.556326+0000","last_clean":"2023-07-06T20:17:31.556326+0000","last_became_active":"2023-07-06T20:17:27.600533+0000","last_became_peered":"2023-07-06T20:17:27.600533+0000","last_unstale":"2023-07-06T20:17:31.556326+0000","last_undegraded":"2023-07-06T20:17:31.556326+0000","last_fullsized":"2023-07-06T20:17:31.556326+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:22:04.466952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.617546+0000","last_change":"2023-07-06T20:17:28.574205+0000","last_active":"2023-07-06T20:17:31.617546+0000","last_peered":"2023-07-06T20:17:31.617546+0000","last_clean":"2023-07-06T20:17:31.617546+0000","last_became_active":"2023-07-06T20:17:28.573699+0000","last_became_peered":"2023-07-06T20:17:28.573699+0000","last_unstale":"2023-07-06T20:17:31.617546+0000","last_undegraded":"2023-07-06T20:17:31.617546+0000","last_fullsized":"2023-07-06T20:17:31.617546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:58:45.139117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555888+0000","last_change":"2023-07-06T20:17:27.604464+0000","last_active":"2023-07-06T20:17:31.555888+0000","last_peered":"2023-07-06T20:17:31.555888+0000","last_clean":"2023-07-06T20:17:31.555888+0000","last_became_active":"2023-07-06T20:17:27.604036+0000","last_became_peered":"2023-07-06T20:17:27.604036+0000","last_unstale":"2023-07-06T20:17:31.555888+0000","last_undegraded":"2023-07-06T20:17:31.555888+0000","last_fullsized":"2023-07-06T20:17:31.555888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:08:32.646765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556251+0000","last_change":"2023-07-06T20:17:28.577302+0000","last_active":"2023-07-06T20:17:31.556251+0000","last_peered":"2023-07-06T20:17:31.556251+0000","last_clean":"2023-07-06T20:17:31.556251+0000","last_became_active":"2023-07-06T20:17:28.576998+0000","last_became_peered":"2023-07-06T20:17:28.576998+0000","last_unstale":"2023-07-06T20:17:31.556251+0000","last_undegraded":"2023-07-06T20:17:31.556251+0000","last_fullsized":"2023-07-06T20:17:31.556251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:55:59.882268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.560306+0000","last_change":"2023-07-06T20:17:27.608391+0000","last_active":"2023-07-06T20:17:28.560306+0000","last_peered":"2023-07-06T20:17:28.560306+0000","last_clean":"2023-07-06T20:17:28.560306+0000","last_became_active":"2023-07-06T20:17:27.608073+0000","last_became_peered":"2023-07-06T20:17:27.608073+0000","last_unstale":"2023-07-06T20:17:28.560306+0000","last_undegraded":"2023-07-06T20:17:28.560306+0000","last_fullsized":"2023-07-06T20:17:28.560306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:54:19.558084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556268+0000","last_change":"2023-07-06T20:17:28.577554+0000","last_active":"2023-07-06T20:17:31.556268+0000","last_peered":"2023-07-06T20:17:31.556268+0000","last_clean":"2023-07-06T20:17:31.556268+0000","last_became_active":"2023-07-06T20:17:28.577208+0000","last_became_peered":"2023-07-06T20:17:28.577208+0000","last_unstale":"2023-07-06T20:17:31.556268+0000","last_undegraded":"2023-07-06T20:17:31.556268+0000","last_fullsized":"2023-07-06T20:17:31.556268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:29:19.435901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556713+0000","last_change":"2023-07-06T20:17:28.571678+0000","last_active":"2023-07-06T20:17:31.556713+0000","last_peered":"2023-07-06T20:17:31.556713+0000","last_clean":"2023-07-06T20:17:31.556713+0000","last_became_active":"2023-07-06T20:17:28.570974+0000","last_became_peered":"2023-07-06T20:17:28.570974+0000","last_unstale":"2023-07-06T20:17:31.556713+0000","last_undegraded":"2023-07-06T20:17:31.556713+0000","last_fullsized":"2023-07-06T20:17:31.556713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:14.827580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.564171+0000","last_change":"2023-07-06T20:17:27.607172+0000","last_active":"2023-07-06T20:17:32.564171+0000","last_peered":"2023-07-06T20:17:32.564171+0000","last_clean":"2023-07-06T20:17:32.564171+0000","last_became_active":"2023-07-06T20:17:27.606841+0000","last_became_peered":"2023-07-06T20:17:27.606841+0000","last_unstale":"2023-07-06T20:17:32.564171+0000","last_undegraded":"2023-07-06T20:17:32.564171+0000","last_fullsized":"2023-07-06T20:17:32.564171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:10:16.491664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556335+0000","last_change":"2023-07-06T20:17:28.567707+0000","last_active":"2023-07-06T20:17:31.556335+0000","last_peered":"2023-07-06T20:17:31.556335+0000","last_clean":"2023-07-06T20:17:31.556335+0000","last_became_active":"2023-07-06T20:17:28.567394+0000","last_became_peered":"2023-07-06T20:17:28.567394+0000","last_unstale":"2023-07-06T20:17:31.556335+0000","last_undegraded":"2023-07-06T20:17:31.556335+0000","last_fullsized":"2023-07-06T20:17:31.556335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:09:58.514942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.619257+0000","last_change":"2023-07-06T20:17:27.595746+0000","last_active":"2023-07-06T20:17:31.619257+0000","last_peered":"2023-07-06T20:17:31.619257+0000","last_clean":"2023-07-06T20:17:31.619257+0000","last_became_active":"2023-07-06T20:17:27.595039+0000","last_became_peered":"2023-07-06T20:17:27.595039+0000","last_unstale":"2023-07-06T20:17:31.619257+0000","last_undegraded":"2023-07-06T20:17:31.619257+0000","last_fullsized":"2023-07-06T20:17:31.619257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:00:16.991853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":331,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.560399+0000","last_change":"2023-07-06T20:16:23.542324+0000","last_active":"2023-07-06T20:17:28.560399+0000","last_peered":"2023-07-06T20:17:28.560399+0000","last_clean":"2023-07-06T20:17:28.560399+0000","last_became_active":"2023-07-06T20:16:23.541832+0000","last_became_peered":"2023-07-06T20:16:23.541832+0000","last_unstale":"2023-07-06T20:17:28.560399+0000","last_undegraded":"2023-07-06T20:17:28.560399+0000","last_fullsized":"2023-07-06T20:17:28.560399+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_clean_scrub_stamp":"2023-07-06T20:13:14.072924+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:39:31.829270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.566606+0000","last_change":"2023-07-06T20:17:28.577309+0000","last_active":"2023-07-06T20:17:29.566606+0000","last_peered":"2023-07-06T20:17:29.566606+0000","last_clean":"2023-07-06T20:17:29.566606+0000","last_became_active":"2023-07-06T20:17:28.576962+0000","last_became_peered":"2023-07-06T20:17:28.576962+0000","last_unstale":"2023-07-06T20:17:29.566606+0000","last_undegraded":"2023-07-06T20:17:29.566606+0000","last_fullsized":"2023-07-06T20:17:29.566606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:58:48.351245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.561010+0000","last_change":"2023-07-06T20:17:27.601487+0000","last_active":"2023-07-06T20:17:28.561010+0000","last_peered":"2023-07-06T20:17:28.561010+0000","last_clean":"2023-07-06T20:17:28.561010+0000","last_became_active":"2023-07-06T20:17:27.601195+0000","last_became_peered":"2023-07-06T20:17:27.601195+0000","last_unstale":"2023-07-06T20:17:28.561010+0000","last_undegraded":"2023-07-06T20:17:28.561010+0000","last_fullsized":"2023-07-06T20:17:28.561010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:56:39.336345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.619412+0000","last_change":"2023-07-06T20:17:27.610374+0000","last_active":"2023-07-06T20:17:31.619412+0000","last_peered":"2023-07-06T20:17:31.619412+0000","last_clean":"2023-07-06T20:17:31.619412+0000","last_became_active":"2023-07-06T20:17:27.610104+0000","last_became_peered":"2023-07-06T20:17:27.610104+0000","last_unstale":"2023-07-06T20:17:31.619412+0000","last_undegraded":"2023-07-06T20:17:31.619412+0000","last_fullsized":"2023-07-06T20:17:31.619412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:41:02.053169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.567031+0000","last_change":"2023-07-06T20:17:28.570146+0000","last_active":"2023-07-06T20:17:29.567031+0000","last_peered":"2023-07-06T20:17:29.567031+0000","last_clean":"2023-07-06T20:17:29.567031+0000","last_became_active":"2023-07-06T20:17:28.569341+0000","last_became_peered":"2023-07-06T20:17:28.569341+0000","last_unstale":"2023-07-06T20:17:29.567031+0000","last_undegraded":"2023-07-06T20:17:29.567031+0000","last_fullsized":"2023-07-06T20:17:29.567031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:08:42.339199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555949+0000","last_change":"2023-07-06T20:17:27.604630+0000","last_active":"2023-07-06T20:17:31.555949+0000","last_peered":"2023-07-06T20:17:31.555949+0000","last_clean":"2023-07-06T20:17:31.555949+0000","last_became_active":"2023-07-06T20:17:27.604118+0000","last_became_peered":"2023-07-06T20:17:27.604118+0000","last_unstale":"2023-07-06T20:17:31.555949+0000","last_undegraded":"2023-07-06T20:17:31.555949+0000","last_fullsized":"2023-07-06T20:17:31.555949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:59:34.337697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556287+0000","last_change":"2023-07-06T20:17:28.571753+0000","last_active":"2023-07-06T20:17:31.556287+0000","last_peered":"2023-07-06T20:17:31.556287+0000","last_clean":"2023-07-06T20:17:31.556287+0000","last_became_active":"2023-07-06T20:17:28.571073+0000","last_became_peered":"2023-07-06T20:17:28.571073+0000","last_unstale":"2023-07-06T20:17:31.556287+0000","last_undegraded":"2023-07-06T20:17:31.556287+0000","last_fullsized":"2023-07-06T20:17:31.556287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:39:21.241942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.617659+0000","last_change":"2023-07-06T20:17:27.594354+0000","last_active":"2023-07-06T20:17:31.617659+0000","last_peered":"2023-07-06T20:17:31.617659+0000","last_clean":"2023-07-06T20:17:31.617659+0000","last_became_active":"2023-07-06T20:17:27.594042+0000","last_became_peered":"2023-07-06T20:17:27.594042+0000","last_unstale":"2023-07-06T20:17:31.617659+0000","last_undegraded":"2023-07-06T20:17:31.617659+0000","last_fullsized":"2023-07-06T20:17:31.617659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:26:46.689491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879415+0000","last_change":"2023-07-06T20:17:28.576170+0000","last_active":"2023-07-06T20:17:29.879415+0000","last_peered":"2023-07-06T20:17:29.879415+0000","last_clean":"2023-07-06T20:17:29.879415+0000","last_became_active":"2023-07-06T20:17:28.575493+0000","last_became_peered":"2023-07-06T20:17:28.575493+0000","last_unstale":"2023-07-06T20:17:29.879415+0000","last_undegraded":"2023-07-06T20:17:29.879415+0000","last_fullsized":"2023-07-06T20:17:29.879415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:52:28.276995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.619859+0000","last_change":"2023-07-06T20:17:27.602984+0000","last_active":"2023-07-06T20:17:31.619859+0000","last_peered":"2023-07-06T20:17:31.619859+0000","last_clean":"2023-07-06T20:17:31.619859+0000","last_became_active":"2023-07-06T20:17:27.602720+0000","last_became_peered":"2023-07-06T20:17:27.602720+0000","last_unstale":"2023-07-06T20:17:31.619859+0000","last_undegraded":"2023-07-06T20:17:31.619859+0000","last_fullsized":"2023-07-06T20:17:31.619859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:51:09.808812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555911+0000","last_change":"2023-07-06T20:17:28.565954+0000","last_active":"2023-07-06T20:17:31.555911+0000","last_peered":"2023-07-06T20:17:31.555911+0000","last_clean":"2023-07-06T20:17:31.555911+0000","last_became_active":"2023-07-06T20:17:28.565300+0000","last_became_peered":"2023-07-06T20:17:28.565300+0000","last_unstale":"2023-07-06T20:17:31.555911+0000","last_undegraded":"2023-07-06T20:17:31.555911+0000","last_fullsized":"2023-07-06T20:17:31.555911+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:06:35.879942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.618718+0000","last_change":"2023-07-06T20:17:27.597859+0000","last_active":"2023-07-06T20:17:31.618718+0000","last_peered":"2023-07-06T20:17:31.618718+0000","last_clean":"2023-07-06T20:17:31.618718+0000","last_became_active":"2023-07-06T20:17:27.597479+0000","last_became_peered":"2023-07-06T20:17:27.597479+0000","last_unstale":"2023-07-06T20:17:31.618718+0000","last_undegraded":"2023-07-06T20:17:31.618718+0000","last_fullsized":"2023-07-06T20:17:31.618718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:18:38.533163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.617499+0000","last_change":"2023-07-06T20:17:28.573069+0000","last_active":"2023-07-06T20:17:31.617499+0000","last_peered":"2023-07-06T20:17:31.617499+0000","last_clean":"2023-07-06T20:17:31.617499+0000","last_became_active":"2023-07-06T20:17:28.572742+0000","last_became_peered":"2023-07-06T20:17:28.572742+0000","last_unstale":"2023-07-06T20:17:31.617499+0000","last_undegraded":"2023-07-06T20:17:31.617499+0000","last_fullsized":"2023-07-06T20:17:31.617499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:29:10.236041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.564683+0000","last_change":"2023-07-06T20:17:27.599259+0000","last_active":"2023-07-06T20:17:29.564683+0000","last_peered":"2023-07-06T20:17:29.564683+0000","last_clean":"2023-07-06T20:17:29.564683+0000","last_became_active":"2023-07-06T20:17:27.598883+0000","last_became_peered":"2023-07-06T20:17:27.598883+0000","last_unstale":"2023-07-06T20:17:29.564683+0000","last_undegraded":"2023-07-06T20:17:29.564683+0000","last_fullsized":"2023-07-06T20:17:29.564683+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:09:03.059986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555763+0000","last_change":"2023-07-06T20:17:28.573874+0000","last_active":"2023-07-06T20:17:31.555763+0000","last_peered":"2023-07-06T20:17:31.555763+0000","last_clean":"2023-07-06T20:17:31.555763+0000","last_became_active":"2023-07-06T20:17:28.573379+0000","last_became_peered":"2023-07-06T20:17:28.573379+0000","last_unstale":"2023-07-06T20:17:31.555763+0000","last_undegraded":"2023-07-06T20:17:31.555763+0000","last_fullsized":"2023-07-06T20:17:31.555763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:53:53.176147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.620120+0000","last_change":"2023-07-06T20:17:27.601474+0000","last_active":"2023-07-06T20:17:31.620120+0000","last_peered":"2023-07-06T20:17:31.620120+0000","last_clean":"2023-07-06T20:17:31.620120+0000","last_became_active":"2023-07-06T20:17:27.601099+0000","last_became_peered":"2023-07-06T20:17:27.601099+0000","last_unstale":"2023-07-06T20:17:31.620120+0000","last_undegraded":"2023-07-06T20:17:31.620120+0000","last_fullsized":"2023-07-06T20:17:31.620120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:54:48.745926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574411+0000","last_change":"2023-07-06T20:17:28.571170+0000","last_active":"2023-07-06T20:17:28.574411+0000","last_peered":"2023-07-06T20:17:28.574411+0000","last_clean":"2023-07-06T20:17:28.574411+0000","last_became_active":"2023-07-06T20:17:28.570874+0000","last_became_peered":"2023-07-06T20:17:28.570874+0000","last_unstale":"2023-07-06T20:17:28.574411+0000","last_undegraded":"2023-07-06T20:17:28.574411+0000","last_fullsized":"2023-07-06T20:17:28.574411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:47:44.919469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:32.564205+0000","last_change":"2023-07-06T20:17:27.603604+0000","last_active":"2023-07-06T20:17:32.564205+0000","last_peered":"2023-07-06T20:17:32.564205+0000","last_clean":"2023-07-06T20:17:32.564205+0000","last_became_active":"2023-07-06T20:17:27.603015+0000","last_became_peered":"2023-07-06T20:17:27.603015+0000","last_unstale":"2023-07-06T20:17:32.564205+0000","last_undegraded":"2023-07-06T20:17:32.564205+0000","last_fullsized":"2023-07-06T20:17:32.564205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:36:14.981024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574692+0000","last_change":"2023-07-06T20:17:28.567906+0000","last_active":"2023-07-06T20:17:28.574692+0000","last_peered":"2023-07-06T20:17:28.574692+0000","last_clean":"2023-07-06T20:17:28.574692+0000","last_became_active":"2023-07-06T20:17:28.567509+0000","last_became_peered":"2023-07-06T20:17:28.567509+0000","last_unstale":"2023-07-06T20:17:28.574692+0000","last_undegraded":"2023-07-06T20:17:28.574692+0000","last_fullsized":"2023-07-06T20:17:28.574692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:30:16.287812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556464+0000","last_change":"2023-07-06T20:17:27.598323+0000","last_active":"2023-07-06T20:17:31.556464+0000","last_peered":"2023-07-06T20:17:31.556464+0000","last_clean":"2023-07-06T20:17:31.556464+0000","last_became_active":"2023-07-06T20:17:27.598032+0000","last_became_peered":"2023-07-06T20:17:27.598032+0000","last_unstale":"2023-07-06T20:17:31.556464+0000","last_undegraded":"2023-07-06T20:17:31.556464+0000","last_fullsized":"2023-07-06T20:17:31.556464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:47:59.236059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574391+0000","last_change":"2023-07-06T20:17:28.571756+0000","last_active":"2023-07-06T20:17:28.574391+0000","last_peered":"2023-07-06T20:17:28.574391+0000","last_clean":"2023-07-06T20:17:28.574391+0000","last_became_active":"2023-07-06T20:17:28.571440+0000","last_became_peered":"2023-07-06T20:17:28.571440+0000","last_unstale":"2023-07-06T20:17:28.574391+0000","last_undegraded":"2023-07-06T20:17:28.574391+0000","last_fullsized":"2023-07-06T20:17:28.574391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:06.531688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.567014+0000","last_change":"2023-07-06T20:17:27.602620+0000","last_active":"2023-07-06T20:17:29.567014+0000","last_peered":"2023-07-06T20:17:29.567014+0000","last_clean":"2023-07-06T20:17:29.567014+0000","last_became_active":"2023-07-06T20:17:27.602224+0000","last_became_peered":"2023-07-06T20:17:27.602224+0000","last_unstale":"2023-07-06T20:17:29.567014+0000","last_undegraded":"2023-07-06T20:17:29.567014+0000","last_fullsized":"2023-07-06T20:17:29.567014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:16:04.139943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-06T20:17:28.574255+0000","last_change":"2023-07-06T20:17:28.569203+0000","last_active":"2023-07-06T20:17:28.574255+0000","last_peered":"2023-07-06T20:17:28.574255+0000","last_clean":"2023-07-06T20:17:28.574255+0000","last_became_active":"2023-07-06T20:17:28.568948+0000","last_became_peered":"2023-07-06T20:17:28.568948+0000","last_unstale":"2023-07-06T20:17:28.574255+0000","last_undegraded":"2023-07-06T20:17:28.574255+0000","last_fullsized":"2023-07-06T20:17:28.574255+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:30:21.073327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.617865+0000","last_change":"2023-07-06T20:17:28.568829+0000","last_active":"2023-07-06T20:17:31.617865+0000","last_peered":"2023-07-06T20:17:31.617865+0000","last_clean":"2023-07-06T20:17:31.617865+0000","last_became_active":"2023-07-06T20:17:28.568511+0000","last_became_peered":"2023-07-06T20:17:28.568511+0000","last_unstale":"2023-07-06T20:17:31.617865+0000","last_undegraded":"2023-07-06T20:17:31.617865+0000","last_fullsized":"2023-07-06T20:17:31.617865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:02:33.443179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.564861+0000","last_change":"2023-07-06T20:17:28.574919+0000","last_active":"2023-07-06T20:17:29.564861+0000","last_peered":"2023-07-06T20:17:29.564861+0000","last_clean":"2023-07-06T20:17:29.564861+0000","last_became_active":"2023-07-06T20:17:28.574545+0000","last_became_peered":"2023-07-06T20:17:28.574545+0000","last_unstale":"2023-07-06T20:17:29.564861+0000","last_undegraded":"2023-07-06T20:17:29.564861+0000","last_fullsized":"2023-07-06T20:17:29.564861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:45:50.951066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.564636+0000","last_change":"2023-07-06T20:17:28.572264+0000","last_active":"2023-07-06T20:17:29.564636+0000","last_peered":"2023-07-06T20:17:29.564636+0000","last_clean":"2023-07-06T20:17:29.564636+0000","last_became_active":"2023-07-06T20:17:28.571914+0000","last_became_peered":"2023-07-06T20:17:28.571914+0000","last_unstale":"2023-07-06T20:17:29.564636+0000","last_undegraded":"2023-07-06T20:17:29.564636+0000","last_fullsized":"2023-07-06T20:17:29.564636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:23:40.059497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.879427+0000","last_change":"2023-07-06T20:17:28.581374+0000","last_active":"2023-07-06T20:17:29.879427+0000","last_peered":"2023-07-06T20:17:29.879427+0000","last_clean":"2023-07-06T20:17:29.879427+0000","last_became_active":"2023-07-06T20:17:28.580995+0000","last_became_peered":"2023-07-06T20:17:28.580995+0000","last_unstale":"2023-07-06T20:17:29.879427+0000","last_undegraded":"2023-07-06T20:17:29.879427+0000","last_fullsized":"2023-07-06T20:17:29.879427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:00:43.859988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.567118+0000","last_change":"2023-07-06T20:17:28.578926+0000","last_active":"2023-07-06T20:17:29.567118+0000","last_peered":"2023-07-06T20:17:29.567118+0000","last_clean":"2023-07-06T20:17:29.567118+0000","last_became_active":"2023-07-06T20:17:28.578620+0000","last_became_peered":"2023-07-06T20:17:28.578620+0000","last_unstale":"2023-07-06T20:17:29.567118+0000","last_undegraded":"2023-07-06T20:17:29.567118+0000","last_fullsized":"2023-07-06T20:17:29.567118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:28:51.567310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.617809+0000","last_change":"2023-07-06T20:17:28.566612+0000","last_active":"2023-07-06T20:17:31.617809+0000","last_peered":"2023-07-06T20:17:31.617809+0000","last_clean":"2023-07-06T20:17:31.617809+0000","last_became_active":"2023-07-06T20:17:28.566288+0000","last_became_peered":"2023-07-06T20:17:28.566288+0000","last_unstale":"2023-07-06T20:17:31.617809+0000","last_undegraded":"2023-07-06T20:17:31.617809+0000","last_fullsized":"2023-07-06T20:17:31.617809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:58:57.543076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.565042+0000","last_change":"2023-07-06T20:17:28.567965+0000","last_active":"2023-07-06T20:17:29.565042+0000","last_peered":"2023-07-06T20:17:29.565042+0000","last_clean":"2023-07-06T20:17:29.565042+0000","last_became_active":"2023-07-06T20:17:28.567618+0000","last_became_peered":"2023-07-06T20:17:28.567618+0000","last_unstale":"2023-07-06T20:17:29.565042+0000","last_undegraded":"2023-07-06T20:17:29.565042+0000","last_fullsized":"2023-07-06T20:17:29.565042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:25:08.361862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556395+0000","last_change":"2023-07-06T20:17:28.571934+0000","last_active":"2023-07-06T20:17:31.556395+0000","last_peered":"2023-07-06T20:17:31.556395+0000","last_clean":"2023-07-06T20:17:31.556395+0000","last_became_active":"2023-07-06T20:17:28.571472+0000","last_became_peered":"2023-07-06T20:17:28.571472+0000","last_unstale":"2023-07-06T20:17:31.556395+0000","last_undegraded":"2023-07-06T20:17:31.556395+0000","last_fullsized":"2023-07-06T20:17:31.556395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:21:18.002079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.556069+0000","last_change":"2023-07-06T20:17:28.571904+0000","last_active":"2023-07-06T20:17:31.556069+0000","last_peered":"2023-07-06T20:17:31.556069+0000","last_clean":"2023-07-06T20:17:31.556069+0000","last_became_active":"2023-07-06T20:17:28.571375+0000","last_became_peered":"2023-07-06T20:17:28.571375+0000","last_unstale":"2023-07-06T20:17:31.556069+0000","last_undegraded":"2023-07-06T20:17:31.556069+0000","last_fullsized":"2023-07-06T20:17:31.556069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:21:22.902205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.555529+0000","last_change":"2023-07-06T20:17:28.569139+0000","last_active":"2023-07-06T20:17:31.555529+0000","last_peered":"2023-07-06T20:17:31.555529+0000","last_clean":"2023-07-06T20:17:31.555529+0000","last_became_active":"2023-07-06T20:17:28.568828+0000","last_became_peered":"2023-07-06T20:17:28.568828+0000","last_unstale":"2023-07-06T20:17:31.555529+0000","last_undegraded":"2023-07-06T20:17:31.555529+0000","last_fullsized":"2023-07-06T20:17:31.555529+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:47:40.913978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:30.075587+0000","last_change":"2023-07-06T20:17:28.562352+0000","last_active":"2023-07-06T20:17:30.075587+0000","last_peered":"2023-07-06T20:17:30.075587+0000","last_clean":"2023-07-06T20:17:30.075587+0000","last_became_active":"2023-07-06T20:17:28.562128+0000","last_became_peered":"2023-07-06T20:17:28.562128+0000","last_unstale":"2023-07-06T20:17:30.075587+0000","last_undegraded":"2023-07-06T20:17:30.075587+0000","last_fullsized":"2023-07-06T20:17:30.075587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:48:50.486178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:29.566941+0000","last_change":"2023-07-06T20:17:28.578940+0000","last_active":"2023-07-06T20:17:29.566941+0000","last_peered":"2023-07-06T20:17:29.566941+0000","last_clean":"2023-07-06T20:17:29.566941+0000","last_became_active":"2023-07-06T20:17:28.578660+0000","last_became_peered":"2023-07-06T20:17:28.578660+0000","last_unstale":"2023-07-06T20:17:29.566941+0000","last_undegraded":"2023-07-06T20:17:29.566941+0000","last_fullsized":"2023-07-06T20:17:29.566941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:51:32.527059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-06T20:17:31.617356+0000","last_change":"2023-07-06T20:17:28.573964+0000","last_active":"2023-07-06T20:17:31.617356+0000","last_peered":"2023-07-06T20:17:31.617356+0000","last_clean":"2023-07-06T20:17:31.617356+0000","last_became_active":"2023-07-06T20:17:28.573618+0000","last_became_peered":"2023-07-06T20:17:28.573618+0000","last_unstale":"2023-07-06T20:17:31.617356+0000","last_undegraded":"2023-07-06T20:17:31.617356+0000","last_fullsized":"2023-07-06T20:17:31.617356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:42:04.680868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":39177,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":19,"num_read_kb":32,"num_write":115,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":46605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449101,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":417248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]}]},{"osd":10,"up_from":71,"seq":304942678033,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1548288,"data_stored":1037158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":1,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":1.131},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.02}]},{"osd":7,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.71999999999999997}]},{"osd":9,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":5.4219999999999997,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.85499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":9,"up_from":66,"seq":283467841556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":426602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60199999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.66400000000000003}]},{"osd":8,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.90600000000000003}]},{"osd":10,"last update":"Thu Jul 6 20:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.93700000000000006}]}]},{"osd":8,"up_from":58,"seq":249108103193,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":426163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.441}]},{"osd":1,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.498},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.622},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.9139999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.64800000000000002}]},{"osd":9,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.75},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.73499999999999999}]},{"osd":10,"last update":"Thu Jul 6 20:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.70399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332126,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28392,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716760,"statfs":{"total":95995035648,"available":95965962240,"internally_reserved":0,"allocated":1474560,"data_stored":1008198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51300000000000001,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85299999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.38100000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.41799999999999998,"15min":0.40899999999999997},"min":{"1min":0.29499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.437},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77900000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83799999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.57799999999999996,"15min":0.56399999999999995},"min":{"1min":0.54000000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68700000000000006,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85099999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.378}]},{"osd":5,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84699999999999998,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.498,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.46700000000000003}]},{"osd":6,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.53400000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.54900000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88300000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.55900000000000005}]},{"osd":8,"last update":"Thu Jul 6 20:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.5}]},{"osd":9,"last update":"Thu Jul 6 20:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58199999999999996}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54200000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495651,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.48599999999999999,"15min":0.47199999999999998},"min":{"1min":0.40799999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.66700000000000004,"15min":0.65200000000000002},"min":{"1min":0.57299999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.504,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.85899999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45200000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.497},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0509999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.74299999999999999,"15min":0.72699999999999998},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jul 6 20:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.181,"5min":1.393,"15min":1.393},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.77100000000000002,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jul 6 20:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.78500000000000003,"15min":0.77500000000000002},"min":{"1min":0.60699999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.78800000000000003},"min":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.55600000000000005}]},{"osd":8,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.93799999999999994}]},{"osd":9,"last update":"Thu Jul 6 20:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.66900000000000004}]},{"osd":10,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.752},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.63900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":427382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.504,"15min":0.47599999999999998},"min":{"1min":0.34799999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.78600000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.53900000000000003,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.248,"15min":0.248},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.52900000000000003,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.95699999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.53700000000000003,"15min":0.501},"min":{"1min":0.43099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.871,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.875}]},{"osd":3,"last update":"Thu Jul 6 20:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.35499999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.85299999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57899999999999996,"15min":0.55100000000000005},"min":{"1min":0.44900000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.876,"15min":0.876},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jul 6 20:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.40400000000000003,"15min":0.38400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.433,"15min":0.41299999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41299999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.52700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0169999999999999,"5min":1.042,"15min":1.042},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jul 6 20:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.438,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.43099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90500000000000003,"5min":1.026,"15min":1.026},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jul 6 20:17:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.32600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.39900000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.59699999999999998}]},{"osd":8,"last update":"Thu Jul 6 20:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.90500000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66600000000000004}]},{"osd":10,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.67300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738431,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.0229999999999999,"5min":1.778,"15min":1.778},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60699999999999998,"15min":0.57199999999999995},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85399999999999998,"5min":4.1379999999999999,"15min":4.1379999999999999},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61399999999999999,"15min":0.58199999999999996},"min":{"1min":0.45100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85399999999999998,"5min":1.085,"15min":1.085},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.622,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":1.258,"15min":1.258},"last":0.84399999999999997}]},{"osd":3,"last update":"Thu Jul 6 20:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72199999999999998,"15min":0.74399999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.046,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":0.755}]},{"osd":5,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65400000000000003,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0389999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0069999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.79400000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.103,"5min":1.319,"15min":1.319},"last":0.77100000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.03,"5min":1.131,"15min":1.131},"last":0.505},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79700000000000004,"15min":0.79600000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.179,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.84399999999999997}]},{"osd":8,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70399999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.83099999999999996}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.93799999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":428684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55800000000000005,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.88700000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.56000000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69399999999999995,"5min":0.876,"15min":0.876},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.32100000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:16:45 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.995,"15min":0.995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.626,"15min":0.626},"min":{"1min":0.55900000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84499999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47199999999999998,"15min":0.45200000000000001},"min":{"1min":0.33500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.47299999999999998,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.34499999999999997}]},{"osd":6,"last update":"Thu Jul 6 20:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.64900000000000002,"15min":0.63800000000000001},"min":{"1min":0.41899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.65900000000000003,"15min":0.64500000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.621}]},{"osd":7,"last update":"Thu Jul 6 20:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.624},"min":{"1min":0.435,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81899999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.40100000000000002}]},{"osd":9,"last update":"Thu Jul 6 20:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.58099999999999996}]},{"osd":10,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.75},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.77000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":426348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.313,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.41999999999999998,"15min":0.39800000000000002},"min":{"1min":0.33500000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.60899999999999999,"15min":0.56899999999999995},"min":{"1min":0.45600000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.6360000000000001,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.752},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.83699999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.65000000000000002,"15min":0.61599999999999999},"min":{"1min":0.48299999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.6720000000000002,"5min":3.6720000000000002,"15min":3.6720000000000002},"last":0.79000000000000004}]},{"osd":4,"last update":"Thu Jul 6 20:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.64600000000000002,"15min":0.58899999999999997},"min":{"1min":0.442,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99199999999999999,"5min":3.6179999999999999,"15min":3.6179999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.60899999999999999,"15min":0.57799999999999996},"min":{"1min":0.52300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.97799999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.65000000000000002,"15min":0.624},"min":{"1min":0.56899999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.501,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.69199999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.876}]},{"osd":8,"last update":"Thu Jul 6 20:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.58299999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:17:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.69999999999999996}]},{"osd":10,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.96199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920812,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.55200000000000005,"15min":0.501},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":6.1539999999999999,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.51900000000000002,"15min":0.49299999999999999},"min":{"1min":0.45700000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.35099999999999998},"min":{"1min":0.255,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63500000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.501},{"interface":"front","average":{"1min":0.434,"5min":0.376,"15min":0.35099999999999998},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.71199999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.38500000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85899999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.377,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88300000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.58699999999999997,"15min":0.52500000000000002},"min":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":5.593,"15min":5.593},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59599999999999997,"15min":0.54500000000000004},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":5.5529999999999999,"15min":5.5529999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jul 6 20:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.55600000000000005},"min":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89800000000000002,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.90900000000000003}]},{"osd":6,"last update":"Thu Jul 6 20:16:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64100000000000001,"15min":0.63700000000000001},"min":{"1min":0.51200000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0149999999999999,"5min":1.109,"15min":1.109},"last":0.628},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.109,"5min":1.163,"15min":1.163},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.55100000000000005,"15min":0.53100000000000003},"min":{"1min":0.35799999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.371,"5min":0.314,"15min":0.314},"max":{"1min":0.89100000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.84199999999999997}]},{"osd":8,"last update":"Thu Jul 6 20:16:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.93700000000000006}]},{"osd":9,"last update":"Thu Jul 6 20:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.87},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.88100000000000001}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.89700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757288,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1507328,"data_stored":1017283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80600000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69399999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.309}]},{"osd":1,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78100000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40300000000000002,"15min":0.39800000000000002},"min":{"1min":0.29299999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.436,"15min":0.42999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.26000000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76200000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.434},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83299999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.32900000000000001}]},{"osd":6,"last update":"Thu Jul 6 20:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.622,"15min":0.622},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64300000000000002,"15min":0.63500000000000001},"min":{"1min":0.47399999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.45600000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70099999999999996,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.498}]},{"osd":8,"last update":"Thu Jul 6 20:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.47199999999999998}]},{"osd":9,"last update":"Thu Jul 6 20:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.54100000000000004}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.58399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":38157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:17:37.356 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T20:17:37.357 INFO:tasks.cephadm:Setting up client nodes... 2023-07-06T20:17:37.358 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-06T20:17:37.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:37 smithi098 ceph-mon[128968]: from='client.24710 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:17:38.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:37 smithi069 ceph-mon[126399]: from='client.24710 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:17:38.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:37 smithi156 ceph-mon[129280]: from='client.24710 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:17:38.107 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:38.938 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:38 smithi069 ceph-mon[126399]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:17:38.939 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:38 smithi069 ceph-mon[126399]: pgmap v273: 97 pgs: 97 active+clean; 615 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8.4 KiB/s wr, 18 op/s 2023-07-06T20:17:39.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:38 smithi156 ceph-mon[129280]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:17:39.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:38 smithi156 ceph-mon[129280]: pgmap v273: 97 pgs: 97 active+clean; 615 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8.4 KiB/s wr, 18 op/s 2023-07-06T20:17:39.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:38 smithi098 ceph-mon[128968]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:17:39.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:38 smithi098 ceph-mon[128968]: pgmap v273: 97 pgs: 97 active+clean; 615 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8.4 KiB/s wr, 18 op/s 2023-07-06T20:17:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.526+0000 7f3a4b620700 1 -- 172.21.15.69:0/1226220163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a44107ba0 msgr2=0x7f3a4410a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.526+0000 7f3a4b620700 1 --2- 172.21.15.69:0/1226220163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a44107ba0 0x7f3a4410a020 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00b0a0 tx=0x7f3a3c00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:17:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.526+0000 7f3a4b620700 1 -- 172.21.15.69:0/1226220163 shutdown_connections 2023-07-06T20:17:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.526+0000 7f3a4b620700 1 --2- 172.21.15.69:0/1226220163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a44107ba0 0x7f3a4410a020 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.526+0000 7f3a4b620700 1 --2- 172.21.15.69:0/1226220163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a440745f0 0x7f3a44072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:39.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.526+0000 7f3a4b620700 1 --2- 172.21.15.69:0/1226220163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a44073020 0x7f3a44073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:39.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.526+0000 7f3a4b620700 1 -- 172.21.15.69:0/1226220163 >> 172.21.15.69:0/1226220163 conn(0x7f3a440fb330 msgr2=0x7f3a440fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:39.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.526+0000 7f3a4b620700 1 -- 172.21.15.69:0/1226220163 shutdown_connections 2023-07-06T20:17:39.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.526+0000 7f3a4b620700 1 -- 172.21.15.69:0/1226220163 wait complete. 2023-07-06T20:17:39.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.527+0000 7f3a4b620700 1 Processor -- start 2023-07-06T20:17:39.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.527+0000 7f3a4b620700 1 -- start start 2023-07-06T20:17:39.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a4b620700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a44073020 0x7f3a44197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:39.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a4b620700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a440745f0 0x7f3a44198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:39.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a4b620700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a44107ba0 0x7f3a4419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:39.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a4b620700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4419f840 con 0x7f3a44073020 2023-07-06T20:17:39.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a4b620700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4419f980 con 0x7f3a440745f0 2023-07-06T20:17:39.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a4b620700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4419fb40 con 0x7f3a44107ba0 2023-07-06T20:17:39.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a493bc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a44073020 0x7f3a44197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:39.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a493bc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a44073020 0x7f3a44197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51990/0 (socket says 172.21.15.69:51990) 2023-07-06T20:17:39.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a493bc700 1 -- 172.21.15.69:0/1743819504 learned_addr learned my addr 172.21.15.69:0/1743819504 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a48bbb700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a440745f0 0x7f3a44198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a49bbd700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a44107ba0 0x7f3a4419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a493bc700 1 -- 172.21.15.69:0/1743819504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a44107ba0 msgr2=0x7f3a4419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a493bc700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a44107ba0 0x7f3a4419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a493bc700 1 -- 172.21.15.69:0/1743819504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a440745f0 msgr2=0x7f3a44198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a493bc700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a440745f0 0x7f3a44198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:39.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.528+0000 7f3a493bc700 1 -- 172.21.15.69:0/1743819504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c009d00 con 0x7f3a44073020 2023-07-06T20:17:39.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.529+0000 7f3a493bc700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a44073020 0x7f3a44197cf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000d980 tx=0x7f3a4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:39.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.529+0000 7f3a3a7fc700 1 -- 172.21.15.69:0/1743819504 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40004a30 con 0x7f3a44073020 2023-07-06T20:17:39.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.529+0000 7f3a3a7fc700 1 -- 172.21.15.69:0/1743819504 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a400054e0 con 0x7f3a44073020 2023-07-06T20:17:39.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.529+0000 7f3a4b620700 1 -- 172.21.15.69:0/1743819504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4419fe20 con 0x7f3a44073020 2023-07-06T20:17:39.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.529+0000 7f3a3a7fc700 1 -- 172.21.15.69:0/1743819504 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000fbc0 con 0x7f3a44073020 2023-07-06T20:17:39.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.529+0000 7f3a4b620700 1 -- 172.21.15.69:0/1743819504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a441a01e0 con 0x7f3a44073020 2023-07-06T20:17:39.535 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.531+0000 7f3a4b620700 1 -- 172.21.15.69:0/1743819504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a441062e0 con 0x7f3a44073020 2023-07-06T20:17:39.535 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.532+0000 7f3a3a7fc700 1 -- 172.21.15.69:0/1743819504 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3a40004c20 con 0x7f3a44073020 2023-07-06T20:17:39.535 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.534+0000 7f3a3a7fc700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3a300a11e0 0x7f3a300a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:39.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.534+0000 7f3a3a7fc700 1 -- 172.21.15.69:0/1743819504 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3a400cc7d0 con 0x7f3a44073020 2023-07-06T20:17:39.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.534+0000 7f3a48bbb700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3a300a11e0 0x7f3a300a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:39.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.534+0000 7f3a48bbb700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3a300a11e0 0x7f3a300a3690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3a34007c60 tx=0x7f3a340073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:39.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.538+0000 7f3a3a7fc700 1 -- 172.21.15.69:0/1743819504 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3a40095e70 con 0x7f3a44073020 2023-07-06T20:17:39.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.813+0000 7f3a4b620700 1 -- 172.21.15.69:0/1743819504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f3a4410d9f0 con 0x7f3a44073020 2023-07-06T20:17:39.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.818+0000 7f3a3a7fc700 1 -- 172.21.15.69:0/1743819504 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f3a40099490 con 0x7f3a44073020 2023-07-06T20:17:39.819 INFO:teuthology.orchestra.run.smithi069.stdout:[client.0] 2023-07-06T20:17:39.819 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBjIadkldWQMBAARvdyesIX0T3x3QasRFC3ww== 2023-07-06T20:17:39.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.819+0000 7f3a4b620700 1 -- 172.21.15.69:0/1743819504 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3a300a11e0 msgr2=0x7f3a300a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:39.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.819+0000 7f3a4b620700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3a300a11e0 0x7f3a300a3690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3a34007c60 tx=0x7f3a340073d0 comp rx=0 tx=0).stop 2023-07-06T20:17:39.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.819+0000 7f3a4b620700 1 -- 172.21.15.69:0/1743819504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a44073020 msgr2=0x7f3a44197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:39.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.819+0000 7f3a4b620700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a44073020 0x7f3a44197cf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000d980 tx=0x7f3a4000dc90 comp rx=0 tx=0).stop 2023-07-06T20:17:39.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.820+0000 7f3a4b620700 1 -- 172.21.15.69:0/1743819504 shutdown_connections 2023-07-06T20:17:39.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.820+0000 7f3a4b620700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3a300a11e0 0x7f3a300a3690 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:39.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.820+0000 7f3a4b620700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a44107ba0 0x7f3a4419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:39.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.820+0000 7f3a4b620700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a44073020 0x7f3a44197cf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:39.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.820+0000 7f3a4b620700 1 --2- 172.21.15.69:0/1743819504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a440745f0 0x7f3a44198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:39.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.820+0000 7f3a4b620700 1 -- 172.21.15.69:0/1743819504 >> 172.21.15.69:0/1743819504 conn(0x7f3a440fb330 msgr2=0x7f3a440fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:39.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.820+0000 7f3a4b620700 1 -- 172.21.15.69:0/1743819504 shutdown_connections 2023-07-06T20:17:39.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:39.820+0000 7f3a4b620700 1 -- 172.21.15.69:0/1743819504 wait complete. 2023-07-06T20:17:40.287 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:17:40.287 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-06T20:17:40.288 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-06T20:17:40.387 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-06T20:17:40.388 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-06T20:17:40.389 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph mgr dump --format=json 2023-07-06T20:17:41.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:40 smithi069 ceph-mon[126399]: pgmap v274: 97 pgs: 97 active+clean; 616 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 7.1 KiB/s wr, 18 op/s 2023-07-06T20:17:41.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:40 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1743819504' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-06T20:17:41.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:40 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1743819504' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-06T20:17:41.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:40 smithi156 ceph-mon[129280]: pgmap v274: 97 pgs: 97 active+clean; 616 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 7.1 KiB/s wr, 18 op/s 2023-07-06T20:17:41.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:40 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1743819504' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-06T20:17:41.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:40 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1743819504' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-06T20:17:41.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:40 smithi098 ceph-mon[128968]: pgmap v274: 97 pgs: 97 active+clean; 616 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 7.1 KiB/s wr, 18 op/s 2023-07-06T20:17:41.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:40 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1743819504' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-06T20:17:41.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:40 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1743819504' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-06T20:17:41.145 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:42.638 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.637+0000 7fb1df459700 1 -- 172.21.15.69:0/1522473855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1d8100d90 msgr2=0x7fb1d8101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:42.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.637+0000 7fb1df459700 1 --2- 172.21.15.69:0/1522473855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1d8100d90 0x7fb1d8101160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009990 tx=0x7fb1d4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:42.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.637+0000 7fb1df459700 1 -- 172.21.15.69:0/1522473855 shutdown_connections 2023-07-06T20:17:42.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.637+0000 7fb1df459700 1 --2- 172.21.15.69:0/1522473855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d810a2b0 0x7fb1d810c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:42.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.637+0000 7fb1df459700 1 --2- 172.21.15.69:0/1522473855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1d8100d90 0x7fb1d8101160 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:42.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.637+0000 7fb1df459700 1 --2- 172.21.15.69:0/1522473855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1d81016a0 0x7fb1d8109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:42.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.637+0000 7fb1df459700 1 -- 172.21.15.69:0/1522473855 >> 172.21.15.69:0/1522473855 conn(0x7fb1d8075ca0 msgr2=0x7fb1d80760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:42.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.637+0000 7fb1df459700 1 -- 172.21.15.69:0/1522473855 shutdown_connections 2023-07-06T20:17:42.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.637+0000 7fb1df459700 1 -- 172.21.15.69:0/1522473855 wait complete. 2023-07-06T20:17:42.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.638+0000 7fb1df459700 1 Processor -- start 2023-07-06T20:17:42.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.638+0000 7fb1df459700 1 -- start start 2023-07-06T20:17:42.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1df459700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d8100d90 0x7fb1d8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:42.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1df459700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1d81016a0 0x7fb1d8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:42.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1df459700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1d810a2b0 0x7fb1d819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:42.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1df459700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d819cfb0 con 0x7fb1d81016a0 2023-07-06T20:17:42.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1df459700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d819d120 con 0x7fb1d8100d90 2023-07-06T20:17:42.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1df459700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d819d2e0 con 0x7fb1d810a2b0 2023-07-06T20:17:42.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1dd1f5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d8100d90 0x7fb1d8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:42.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1dc9f4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1d81016a0 0x7fb1d8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:42.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1dc9f4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1d81016a0 0x7fb1d8198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52010/0 (socket says 172.21.15.69:52010) 2023-07-06T20:17:42.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1dc9f4700 1 -- 172.21.15.69:0/444867066 learned_addr learned my addr 172.21.15.69:0/444867066 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:42.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1dd1f5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d8100d90 0x7fb1d8197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:56894/0 (socket says 172.21.15.69:56894) 2023-07-06T20:17:42.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1dd9f6700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1d810a2b0 0x7fb1d819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:42.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1dc9f4700 1 -- 172.21.15.69:0/444867066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1d810a2b0 msgr2=0x7fb1d819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:42.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1dc9f4700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1d810a2b0 0x7fb1d819c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:42.646 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1dc9f4700 1 -- 172.21.15.69:0/444867066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d8100d90 msgr2=0x7fb1d8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:42.646 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1dc9f4700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d8100d90 0x7fb1d8197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:42.646 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.639+0000 7fb1dc9f4700 1 -- 172.21.15.69:0/444867066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d40095f0 con 0x7fb1d81016a0 2023-07-06T20:17:42.647 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.640+0000 7fb1dc9f4700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1d81016a0 0x7fb1d8198530 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800e9a0 tx=0x7fb1c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:42.647 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.640+0000 7fb1ce7fc700 1 -- 172.21.15.69:0/444867066 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8009790 con 0x7fb1d81016a0 2023-07-06T20:17:42.647 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.640+0000 7fb1df459700 1 -- 172.21.15.69:0/444867066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d819d5c0 con 0x7fb1d81016a0 2023-07-06T20:17:42.648 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.640+0000 7fb1ce7fc700 1 -- 172.21.15.69:0/444867066 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1c800cbd0 con 0x7fb1d81016a0 2023-07-06T20:17:42.648 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.640+0000 7fb1ce7fc700 1 -- 172.21.15.69:0/444867066 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8018680 con 0x7fb1d81016a0 2023-07-06T20:17:42.648 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.640+0000 7fb1df459700 1 -- 172.21.15.69:0/444867066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d81a4e20 con 0x7fb1d81016a0 2023-07-06T20:17:42.649 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.641+0000 7fb1df459700 1 -- 172.21.15.69:0/444867066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d81078c0 con 0x7fb1d81016a0 2023-07-06T20:17:42.649 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.643+0000 7fb1ce7fc700 1 -- 172.21.15.69:0/444867066 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb1c800cd40 con 0x7fb1d81016a0 2023-07-06T20:17:42.649 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.644+0000 7fb1ce7fc700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb1c40a1260 0x7fb1c40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:42.649 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.645+0000 7fb1ce7fc700 1 -- 172.21.15.69:0/444867066 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb1c8014070 con 0x7fb1d81016a0 2023-07-06T20:17:42.649 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.645+0000 7fb1dd1f5700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb1c40a1260 0x7fb1c40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:42.650 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.645+0000 7fb1dd1f5700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb1c40a1260 0x7fb1c40a3710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400b2e0 tx=0x7fb1d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:42.650 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.649+0000 7fb1ce7fc700 1 -- 172.21.15.69:0/444867066 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb1c8051770 con 0x7fb1d81016a0 2023-07-06T20:17:42.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:42 smithi069 ceph-mon[126399]: pgmap v275: 97 pgs: 97 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 16 op/s 2023-07-06T20:17:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.898+0000 7fb1df459700 1 -- 172.21.15.69:0/444867066 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb1d802c8e0 con 0x7fb1d81016a0 2023-07-06T20:17:42.904 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.903+0000 7fb1ce7fc700 1 -- 172.21.15.69:0/444867066 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260126 (secure 0 0 0) 0x7fb1c80962e0 con 0x7fb1d81016a0 2023-07-06T20:17:42.907 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.908+0000 7fb1df459700 1 -- 172.21.15.69:0/444867066 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb1c40a1260 msgr2=0x7fb1c40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.908+0000 7fb1df459700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb1c40a1260 0x7fb1c40a3710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400b2e0 tx=0x7fb1d4005fd0 comp rx=0 tx=0).stop 2023-07-06T20:17:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.908+0000 7fb1df459700 1 -- 172.21.15.69:0/444867066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1d81016a0 msgr2=0x7fb1d8198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.908+0000 7fb1df459700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1d81016a0 0x7fb1d8198530 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800e9a0 tx=0x7fb1c800ecb0 comp rx=0 tx=0).stop 2023-07-06T20:17:42.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.908+0000 7fb1df459700 1 -- 172.21.15.69:0/444867066 shutdown_connections 2023-07-06T20:17:42.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.908+0000 7fb1df459700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb1c40a1260 0x7fb1c40a3710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:42.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.908+0000 7fb1df459700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1d810a2b0 0x7fb1d819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:42.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.908+0000 7fb1df459700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d8100d90 0x7fb1d8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:42.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.908+0000 7fb1df459700 1 --2- 172.21.15.69:0/444867066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1d81016a0 0x7fb1d8198530 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:42.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.908+0000 7fb1df459700 1 -- 172.21.15.69:0/444867066 >> 172.21.15.69:0/444867066 conn(0x7fb1d8075ca0 msgr2=0x7fb1d80ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:42.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.908+0000 7fb1df459700 1 -- 172.21.15.69:0/444867066 shutdown_connections 2023-07-06T20:17:42.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:42.908+0000 7fb1df459700 1 -- 172.21.15.69:0/444867066 wait complete. 2023-07-06T20:17:43.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:42 smithi156 ceph-mon[129280]: pgmap v275: 97 pgs: 97 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 16 op/s 2023-07-06T20:17:43.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:42 smithi098 ceph-mon[128968]: pgmap v275: 97 pgs: 97 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 16 op/s 2023-07-06T20:17:43.517 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":4215258217},{"type":"v1","addr":"172.21.15.69:6801","nonce":4215258217}]},"active_addr":"172.21.15.69:6801/4215258217","active_change":"2023-07-06T20:10:57.432872+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.69:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2392941374}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":4003303292}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":1213432936}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2449578480}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2180910776}]}]} 2023-07-06T20:17:43.524 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-06T20:17:43.525 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-06T20:17:43.525 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd dump --format=json 2023-07-06T20:17:43.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:43 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/444867066' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-06T20:17:44.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:43 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/444867066' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-06T20:17:44.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:43 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/444867066' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-06T20:17:44.280 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:44.825 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:44 smithi069 ceph-mon[126399]: pgmap v276: 97 pgs: 97 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.1 KiB/s wr, 17 op/s 2023-07-06T20:17:45.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:44 smithi156 ceph-mon[129280]: pgmap v276: 97 pgs: 97 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.1 KiB/s wr, 17 op/s 2023-07-06T20:17:45.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:44 smithi098 ceph-mon[128968]: pgmap v276: 97 pgs: 97 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.1 KiB/s wr, 17 op/s 2023-07-06T20:17:45.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.776+0000 7f28f3849700 1 -- 172.21.15.69:0/3211336431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28ec104fd0 msgr2=0x7f28ec10cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:45.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.776+0000 7f28f3849700 1 --2- 172.21.15.69:0/3211336431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28ec104fd0 0x7f28ec10cb50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f28e400b0d0 tx=0x7f28e400b3e0 comp rx=0 tx=0).stop 2023-07-06T20:17:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.777+0000 7f28f3849700 1 -- 172.21.15.69:0/3211336431 shutdown_connections 2023-07-06T20:17:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.777+0000 7f28f3849700 1 --2- 172.21.15.69:0/3211336431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28ec104fd0 0x7f28ec10cb50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.777+0000 7f28f3849700 1 --2- 172.21.15.69:0/3211336431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ec0fff30 0x7f28ec100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.777+0000 7f28f3849700 1 --2- 172.21.15.69:0/3211336431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ec1008d0 0x7f28ec1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.777+0000 7f28f3849700 1 -- 172.21.15.69:0/3211336431 >> 172.21.15.69:0/3211336431 conn(0x7f28ec0fb790 msgr2=0x7f28ec0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.777+0000 7f28f3849700 1 -- 172.21.15.69:0/3211336431 shutdown_connections 2023-07-06T20:17:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.777+0000 7f28f3849700 1 -- 172.21.15.69:0/3211336431 wait complete. 2023-07-06T20:17:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f3849700 1 Processor -- start 2023-07-06T20:17:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f3849700 1 -- start start 2023-07-06T20:17:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f3849700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ec0fff30 0x7f28ec195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f3849700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ec1008d0 0x7f28ec1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f3849700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28ec104fd0 0x7f28ec19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f3849700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec19ae50 con 0x7f28ec104fd0 2023-07-06T20:17:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f3849700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec19afc0 con 0x7f28ec1008d0 2023-07-06T20:17:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f3849700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec19b180 con 0x7f28ec0fff30 2023-07-06T20:17:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f15e5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ec0fff30 0x7f28ec195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f15e5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ec0fff30 0x7f28ec195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:60194/0 (socket says 172.21.15.69:60194) 2023-07-06T20:17:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f0de4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ec1008d0 0x7f28ec1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f15e5700 1 -- 172.21.15.69:0/3291486715 learned_addr learned my addr 172.21.15.69:0/3291486715 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.778+0000 7f28f1de6700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28ec104fd0 0x7f28ec19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28f15e5700 1 -- 172.21.15.69:0/3291486715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ec1008d0 msgr2=0x7f28ec1963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28f15e5700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ec1008d0 0x7f28ec1963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28f15e5700 1 -- 172.21.15.69:0/3291486715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28ec104fd0 msgr2=0x7f28ec19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28f15e5700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28ec104fd0 0x7f28ec19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28f15e5700 1 -- 172.21.15.69:0/3291486715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e4009d00 con 0x7f28ec0fff30 2023-07-06T20:17:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28f1de6700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28ec104fd0 0x7f28ec19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28f0de4700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ec1008d0 0x7f28ec1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:17:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28f15e5700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ec0fff30 0x7f28ec195e90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f28e800d980 tx=0x7f28e800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28e27fc700 1 -- 172.21.15.69:0/3291486715 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8004220 con 0x7f28ec0fff30 2023-07-06T20:17:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28e27fc700 1 -- 172.21.15.69:0/3291486715 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28e8007640 con 0x7f28ec0fff30 2023-07-06T20:17:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28f3849700 1 -- 172.21.15.69:0/3291486715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ec19b460 con 0x7f28ec0fff30 2023-07-06T20:17:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28e27fc700 1 -- 172.21.15.69:0/3291486715 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8005690 con 0x7f28ec0fff30 2023-07-06T20:17:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.779+0000 7f28f3849700 1 -- 172.21.15.69:0/3291486715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ec1a2cc0 con 0x7f28ec0fff30 2023-07-06T20:17:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.780+0000 7f28f3849700 1 -- 172.21.15.69:0/3291486715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ec04e340 con 0x7f28ec0fff30 2023-07-06T20:17:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.782+0000 7f28e27fc700 1 -- 172.21.15.69:0/3291486715 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f28e8004750 con 0x7f28ec0fff30 2023-07-06T20:17:45.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.784+0000 7f28e27fc700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f28d80a11e0 0x7f28d80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:45.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.784+0000 7f28e27fc700 1 -- 172.21.15.69:0/3291486715 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f28e80ccaa0 con 0x7f28ec0fff30 2023-07-06T20:17:45.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.784+0000 7f28f0de4700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f28d80a11e0 0x7f28d80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:45.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.784+0000 7f28f0de4700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f28d80a11e0 0x7f28d80a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f28ec1974b0 tx=0x7f28dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:45.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.786+0000 7f28e27fc700 1 -- 172.21.15.69:0/3291486715 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f28e8096140 con 0x7f28ec0fff30 2023-07-06T20:17:45.986 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.984+0000 7f28f3849700 1 -- 172.21.15.69:0/3291486715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28ec1a2f20 con 0x7f28ec0fff30 2023-07-06T20:17:45.987 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.985+0000 7f28e27fc700 1 -- 172.21.15.69:0/3291486715 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22065 (secure 0 0 0) 0x7f28e804d650 con 0x7f28ec0fff30 2023-07-06T20:17:45.988 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:45.988 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":83,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:17:29.552171+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:17:45.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.988+0000 7f28f3849700 1 -- 172.21.15.69:0/3291486715 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f28d80a11e0 msgr2=0x7f28d80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:45.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.988+0000 7f28f3849700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f28d80a11e0 0x7f28d80a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f28ec1974b0 tx=0x7f28dc00b410 comp rx=0 tx=0).stop 2023-07-06T20:17:45.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.988+0000 7f28f3849700 1 -- 172.21.15.69:0/3291486715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ec0fff30 msgr2=0x7f28ec195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:45.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.988+0000 7f28f3849700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ec0fff30 0x7f28ec195e90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f28e800d980 tx=0x7f28e800dc90 comp rx=0 tx=0).stop 2023-07-06T20:17:45.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.988+0000 7f28f3849700 1 -- 172.21.15.69:0/3291486715 shutdown_connections 2023-07-06T20:17:45.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.988+0000 7f28f3849700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f28d80a11e0 0x7f28d80a3690 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:45.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.988+0000 7f28f3849700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28ec104fd0 0x7f28ec19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:45.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.988+0000 7f28f3849700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28ec0fff30 0x7f28ec195e90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:45.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.988+0000 7f28f3849700 1 --2- 172.21.15.69:0/3291486715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ec1008d0 0x7f28ec1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:45.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.988+0000 7f28f3849700 1 -- 172.21.15.69:0/3291486715 >> 172.21.15.69:0/3291486715 conn(0x7f28ec0fb790 msgr2=0x7f28ec0fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:45.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.988+0000 7f28f3849700 1 -- 172.21.15.69:0/3291486715 shutdown_connections 2023-07-06T20:17:45.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:45.988+0000 7f28f3849700 1 -- 172.21.15.69:0/3291486715 wait complete. 2023-07-06T20:17:46.452 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-06T20:17:46.452 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd dump --format=json 2023-07-06T20:17:46.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:46 smithi069 ceph-mon[126399]: pgmap v277: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2023-07-06T20:17:46.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:46 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3291486715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:47.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:46 smithi156 ceph-mon[129280]: pgmap v277: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2023-07-06T20:17:47.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:46 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3291486715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:47.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:46 smithi098 ceph-mon[128968]: pgmap v277: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2023-07-06T20:17:47.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:46 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3291486715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:47.173 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:48.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.676+0000 7f4bceb38700 1 -- 172.21.15.69:0/4088390982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bc80fff30 msgr2=0x7f4bc8100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:48.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.676+0000 7f4bceb38700 1 --2- 172.21.15.69:0/4088390982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bc80fff30 0x7f4bc8100300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0099c0 tx=0x7f4bbc009cd0 comp rx=0 tx=0).stop 2023-07-06T20:17:48.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.676+0000 7f4bceb38700 1 -- 172.21.15.69:0/4088390982 shutdown_connections 2023-07-06T20:17:48.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.676+0000 7f4bceb38700 1 --2- 172.21.15.69:0/4088390982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4bc8104fd0 0x7f4bc810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:48.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.676+0000 7f4bceb38700 1 --2- 172.21.15.69:0/4088390982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bc80fff30 0x7f4bc8100300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:48.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.676+0000 7f4bceb38700 1 --2- 172.21.15.69:0/4088390982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bc81008d0 0x7f4bc81048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:48.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.676+0000 7f4bceb38700 1 -- 172.21.15.69:0/4088390982 >> 172.21.15.69:0/4088390982 conn(0x7f4bc80fb790 msgr2=0x7f4bc80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:48.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.676+0000 7f4bceb38700 1 -- 172.21.15.69:0/4088390982 shutdown_connections 2023-07-06T20:17:48.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.676+0000 7f4bceb38700 1 -- 172.21.15.69:0/4088390982 wait complete. 2023-07-06T20:17:48.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.677+0000 7f4bceb38700 1 Processor -- start 2023-07-06T20:17:48.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.677+0000 7f4bceb38700 1 -- start start 2023-07-06T20:17:48.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bceb38700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bc80fff30 0x7f4bc8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:48.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bceb38700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4bc81008d0 0x7f4bc81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:48.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bceb38700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bc8104fd0 0x7f4bc819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:48.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bceb38700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc819cf60 con 0x7f4bc8104fd0 2023-07-06T20:17:48.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bceb38700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc819d0d0 con 0x7f4bc81008d0 2023-07-06T20:17:48.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bceb38700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc819d290 con 0x7f4bc80fff30 2023-07-06T20:17:48.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bcd0d5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bc8104fd0 0x7f4bc819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:48.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bcc8d4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bc80fff30 0x7f4bc8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:48.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bcd0d5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bc8104fd0 0x7f4bc819c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55966/0 (socket says 172.21.15.69:55966) 2023-07-06T20:17:48.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bcd0d5700 1 -- 172.21.15.69:0/265628989 learned_addr learned my addr 172.21.15.69:0/265628989 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:48.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bc7fff700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4bc81008d0 0x7f4bc81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:48.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bcd0d5700 1 -- 172.21.15.69:0/265628989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bc80fff30 msgr2=0x7f4bc8197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:48.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bcd0d5700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bc80fff30 0x7f4bc8197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:48.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bcd0d5700 1 -- 172.21.15.69:0/265628989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4bc81008d0 msgr2=0x7f4bc81984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:48.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bcd0d5700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4bc81008d0 0x7f4bc81984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:48.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.678+0000 7f4bcd0d5700 1 -- 172.21.15.69:0/265628989 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc0095f0 con 0x7f4bc8104fd0 2023-07-06T20:17:48.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.679+0000 7f4bcd0d5700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bc8104fd0 0x7f4bc819c950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0007ee0 tx=0x7f4bc000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:48.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.679+0000 7f4bc5ffb700 1 -- 172.21.15.69:0/265628989 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc000dd20 con 0x7f4bc8104fd0 2023-07-06T20:17:48.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.679+0000 7f4bc5ffb700 1 -- 172.21.15.69:0/265628989 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bc000f040 con 0x7f4bc8104fd0 2023-07-06T20:17:48.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.679+0000 7f4bceb38700 1 -- 172.21.15.69:0/265628989 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc819d570 con 0x7f4bc8104fd0 2023-07-06T20:17:48.685 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.679+0000 7f4bc5ffb700 1 -- 172.21.15.69:0/265628989 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00048a0 con 0x7f4bc8104fd0 2023-07-06T20:17:48.685 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.679+0000 7f4bceb38700 1 -- 172.21.15.69:0/265628989 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc81a4dd0 con 0x7f4bc8104fd0 2023-07-06T20:17:48.685 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.680+0000 7f4bceb38700 1 -- 172.21.15.69:0/265628989 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc804e340 con 0x7f4bc8104fd0 2023-07-06T20:17:48.685 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.682+0000 7f4bc5ffb700 1 -- 172.21.15.69:0/265628989 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4bc0003a00 con 0x7f4bc8104fd0 2023-07-06T20:17:48.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.684+0000 7f4bc5ffb700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4bb80a11e0 0x7f4bb80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:48.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.685+0000 7f4bc5ffb700 1 -- 172.21.15.69:0/265628989 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4bc00cc200 con 0x7f4bc8104fd0 2023-07-06T20:17:48.687 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.685+0000 7f4bcc8d4700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4bb80a11e0 0x7f4bb80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:48.687 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.685+0000 7f4bcc8d4700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4bb80a11e0 0x7f4bb80a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc000c00 tx=0x7f4bbc003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:48.690 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.688+0000 7f4bc5ffb700 1 -- 172.21.15.69:0/265628989 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4bc004e770 con 0x7f4bc8104fd0 2023-07-06T20:17:48.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:48 smithi069 ceph-mon[126399]: pgmap v278: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2023-07-06T20:17:48.886 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.884+0000 7f4bceb38700 1 -- 172.21.15.69:0/265628989 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bc8199300 con 0x7f4bc8104fd0 2023-07-06T20:17:48.887 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.885+0000 7f4bc5ffb700 1 -- 172.21.15.69:0/265628989 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22065 (secure 0 0 0) 0x7f4bc0095070 con 0x7f4bc8104fd0 2023-07-06T20:17:48.888 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:17:48.888 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":83,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:17:29.552171+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:17:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.888+0000 7f4bceb38700 1 -- 172.21.15.69:0/265628989 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4bb80a11e0 msgr2=0x7f4bb80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.888+0000 7f4bceb38700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4bb80a11e0 0x7f4bb80a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc000c00 tx=0x7f4bbc003680 comp rx=0 tx=0).stop 2023-07-06T20:17:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.888+0000 7f4bceb38700 1 -- 172.21.15.69:0/265628989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bc8104fd0 msgr2=0x7f4bc819c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.888+0000 7f4bceb38700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bc8104fd0 0x7f4bc819c950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0007ee0 tx=0x7f4bc000d3b0 comp rx=0 tx=0).stop 2023-07-06T20:17:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.888+0000 7f4bceb38700 1 -- 172.21.15.69:0/265628989 shutdown_connections 2023-07-06T20:17:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.888+0000 7f4bceb38700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4bb80a11e0 0x7f4bb80a3690 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.888+0000 7f4bceb38700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bc8104fd0 0x7f4bc819c950 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.888+0000 7f4bceb38700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bc80fff30 0x7f4bc8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.888+0000 7f4bceb38700 1 --2- 172.21.15.69:0/265628989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4bc81008d0 0x7f4bc81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.888+0000 7f4bceb38700 1 -- 172.21.15.69:0/265628989 >> 172.21.15.69:0/265628989 conn(0x7f4bc80fb790 msgr2=0x7f4bc80fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.888+0000 7f4bceb38700 1 -- 172.21.15.69:0/265628989 shutdown_connections 2023-07-06T20:17:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:48.888+0000 7f4bceb38700 1 -- 172.21.15.69:0/265628989 wait complete. 2023-07-06T20:17:49.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:48 smithi156 ceph-mon[129280]: pgmap v278: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2023-07-06T20:17:49.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:48 smithi098 ceph-mon[128968]: pgmap v278: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2023-07-06T20:17:49.344 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-06T20:17:49.345 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-06T20:17:49.346 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-06T20:17:49.346 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-06T20:17:49.347 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-06T20:17:49.347 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-06T20:17:49.348 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-06T20:17:49.348 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-06T20:17:49.349 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-06T20:17:49.349 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-06T20:17:49.350 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-06T20:17:49.350 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-06T20:17:50.019 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:49 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/265628989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:50.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:49 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/265628989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:50.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:49 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/265628989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:17:50.799 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:50 smithi069 ceph-mon[126399]: pgmap v279: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2023-07-06T20:17:51.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:50 smithi156 ceph-mon[129280]: pgmap v279: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2023-07-06T20:17:51.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:50 smithi098 ceph-mon[128968]: pgmap v279: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2023-07-06T20:17:51.257 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:51.257 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:51.450 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:51.540 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:51.546 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:51.554 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:52.426 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:52.958 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:52.960 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:52.962 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:17:52.991 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:52 smithi069 ceph-mon[126399]: pgmap v280: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-07-06T20:17:53.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:52 smithi156 ceph-mon[129280]: pgmap v280: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-07-06T20:17:53.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:52 smithi098 ceph-mon[128968]: pgmap v280: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-07-06T20:17:54.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.385+0000 7f043826a700 1 -- 172.21.15.69:0/7214275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0430072980 msgr2=0x7f0430072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:54.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.385+0000 7f043826a700 1 --2- 172.21.15.69:0/7214275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0430072980 0x7f0430072d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f042c009990 tx=0x7f042c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:54.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.385+0000 7f043826a700 1 -- 172.21.15.69:0/7214275 shutdown_connections 2023-07-06T20:17:54.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.385+0000 7f043826a700 1 --2- 172.21.15.69:0/7214275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f043010c570 0x7f043010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:54.389 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.385+0000 7f043826a700 1 --2- 172.21.15.69:0/7214275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0430072980 0x7f0430072d50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:54.389 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.385+0000 7f043826a700 1 --2- 172.21.15.69:0/7214275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0430073290 0x7f043010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:54.389 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.385+0000 7f043826a700 1 -- 172.21.15.69:0/7214275 >> 172.21.15.69:0/7214275 conn(0x7f04300fb790 msgr2=0x7f04300fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:54.389 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.386+0000 7f043826a700 1 -- 172.21.15.69:0/7214275 shutdown_connections 2023-07-06T20:17:54.389 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.386+0000 7f043826a700 1 -- 172.21.15.69:0/7214275 wait complete. 2023-07-06T20:17:54.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.387+0000 7f043826a700 1 Processor -- start 2023-07-06T20:17:54.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.387+0000 7f043826a700 1 -- start start 2023-07-06T20:17:54.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.387+0000 7f043826a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0430072980 0x7f0430197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:54.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.387+0000 7f043826a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0430073290 0x7f0430198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:54.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.387+0000 7f043826a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f043010c570 0x7f043019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:54.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.387+0000 7f043826a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043019cfa0 con 0x7f0430073290 2023-07-06T20:17:54.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.387+0000 7f043826a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043019d110 con 0x7f0430072980 2023-07-06T20:17:54.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.387+0000 7f043826a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043019d2d0 con 0x7f043010c570 2023-07-06T20:17:54.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.388+0000 7f0435805700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0430073290 0x7f0430198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:54.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.388+0000 7f0436807700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f043010c570 0x7f043019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:54.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.388+0000 7f0435805700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0430073290 0x7f0430198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42092/0 (socket says 172.21.15.69:42092) 2023-07-06T20:17:54.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.388+0000 7f0436006700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0430072980 0x7f0430197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:54.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.388+0000 7f0435805700 1 -- 172.21.15.69:0/1638335437 learned_addr learned my addr 172.21.15.69:0/1638335437 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:54.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.388+0000 7f0435805700 1 -- 172.21.15.69:0/1638335437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f043010c570 msgr2=0x7f043019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:54.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.388+0000 7f0435805700 1 --2- 172.21.15.69:0/1638335437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f043010c570 0x7f043019c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:54.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.388+0000 7f0435805700 1 -- 172.21.15.69:0/1638335437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0430072980 msgr2=0x7f0430197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:54.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.388+0000 7f0435805700 1 --2- 172.21.15.69:0/1638335437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0430072980 0x7f0430197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:54.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.388+0000 7f0435805700 1 -- 172.21.15.69:0/1638335437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c0095f0 con 0x7f0430073290 2023-07-06T20:17:54.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.388+0000 7f0435805700 1 --2- 172.21.15.69:0/1638335437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0430073290 0x7f0430198520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f042000e9a0 tx=0x7f042000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:54.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.388+0000 7f04277fe700 1 -- 172.21.15.69:0/1638335437 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420009790 con 0x7f0430073290 2023-07-06T20:17:54.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.389+0000 7f043826a700 1 -- 172.21.15.69:0/1638335437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043019d5b0 con 0x7f0430073290 2023-07-06T20:17:54.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.389+0000 7f04277fe700 1 -- 172.21.15.69:0/1638335437 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0420010430 con 0x7f0430073290 2023-07-06T20:17:54.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.389+0000 7f04277fe700 1 -- 172.21.15.69:0/1638335437 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420018680 con 0x7f0430073290 2023-07-06T20:17:54.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.389+0000 7f043826a700 1 -- 172.21.15.69:0/1638335437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04301a4e10 con 0x7f0430073290 2023-07-06T20:17:54.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.390+0000 7f043826a700 1 -- 172.21.15.69:0/1638335437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0430112090 con 0x7f0430073290 2023-07-06T20:17:54.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.392+0000 7f04277fe700 1 -- 172.21.15.69:0/1638335437 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f04200105a0 con 0x7f0430073290 2023-07-06T20:17:54.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.393+0000 7f04277fe700 1 --2- 172.21.15.69:0/1638335437 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f041c0a1260 0x7f041c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:54.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.394+0000 7f04277fe700 1 -- 172.21.15.69:0/1638335437 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0420014070 con 0x7f0430073290 2023-07-06T20:17:54.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.394+0000 7f0436006700 1 --2- 172.21.15.69:0/1638335437 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f041c0a1260 0x7f041c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:54.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.394+0000 7f04277fe700 1 --2- 172.21.15.69:0/1638335437 >> [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946] conn(0x7f041c0aaa30 0x7f041c0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:54.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.394+0000 7f04277fe700 1 -- 172.21.15.69:0/1638335437 --> [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f041c0ad4f0 con 0x7f041c0aaa30 2023-07-06T20:17:54.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.394+0000 7f04277fe700 1 -- 172.21.15.69:0/1638335437 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04200cd180 con 0x7f0430073290 2023-07-06T20:17:54.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.394+0000 7f0436006700 1 --2- 172.21.15.69:0/1638335437 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f041c0a1260 0x7f041c0a3710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f042c00b2e0 tx=0x7f042c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.394+0000 7f0436807700 1 --2- 172.21.15.69:0/1638335437 >> [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946] conn(0x7f041c0aaa30 0x7f041c0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.395+0000 7f0436807700 1 --2- 172.21.15.69:0/1638335437 >> [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946] conn(0x7f041c0aaa30 0x7f041c0ace40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.396+0000 7f04277fe700 1 -- 172.21.15.69:0/1638335437 <== osd.2 v2:172.21.15.156:6800/1470225946 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f041c0ad4f0 con 0x7f041c0aaa30 2023-07-06T20:17:54.414 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.412+0000 7f043826a700 1 -- 172.21.15.69:0/1638335437 --> [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f043004e340 con 0x7f041c0aaa30 2023-07-06T20:17:54.414 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.412+0000 7f04277fe700 1 -- 172.21.15.69:0/1638335437 <== osd.2 v2:172.21.15.156:6800/1470225946 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f043004e340 con 0x7f041c0aaa30 2023-07-06T20:17:54.416 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.413+0000 7f043826a700 1 -- 172.21.15.69:0/1638335437 >> [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946] conn(0x7f041c0aaa30 msgr2=0x7f041c0ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:54.416 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.413+0000 7f043826a700 1 --2- 172.21.15.69:0/1638335437 >> [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946] conn(0x7f041c0aaa30 0x7f041c0ace40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:54.416 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.413+0000 7f043826a700 1 -- 172.21.15.69:0/1638335437 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f041c0a1260 msgr2=0x7f041c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:54.416 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.413+0000 7f043826a700 1 --2- 172.21.15.69:0/1638335437 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f041c0a1260 0x7f041c0a3710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f042c00b2e0 tx=0x7f042c005fb0 comp rx=0 tx=0).stop 2023-07-06T20:17:54.417 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.413+0000 7f043826a700 1 -- 172.21.15.69:0/1638335437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0430073290 msgr2=0x7f0430198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:54.417 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.413+0000 7f043826a700 1 --2- 172.21.15.69:0/1638335437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0430073290 0x7f0430198520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f042000e9a0 tx=0x7f042000ecb0 comp rx=0 tx=0).stop 2023-07-06T20:17:54.417 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.413+0000 7f0436807700 1 -- 172.21.15.69:0/1638335437 reap_dead start 2023-07-06T20:17:54.417 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.413+0000 7f043826a700 1 -- 172.21.15.69:0/1638335437 shutdown_connections 2023-07-06T20:17:54.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.413+0000 7f043826a700 1 -- 172.21.15.69:0/1638335437 >> 172.21.15.69:0/1638335437 conn(0x7f04300fb790 msgr2=0x7f043010e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:54.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.413+0000 7f043826a700 1 -- 172.21.15.69:0/1638335437 shutdown_connections 2023-07-06T20:17:54.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:54.413+0000 7f043826a700 1 -- 172.21.15.69:0/1638335437 wait complete. 2023-07-06T20:17:54.928 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:54 smithi069 ceph-mon[126399]: pgmap v281: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-07-06T20:17:55.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:54 smithi156 ceph-mon[129280]: pgmap v281: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-07-06T20:17:55.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:54 smithi098 ceph-mon[128968]: pgmap v281: 97 pgs: 97 active+clean; 633 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-07-06T20:17:57.000 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:56 smithi069 ceph-mon[126399]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-07-06T20:17:57.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:56 smithi156 ceph-mon[129280]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-07-06T20:17:57.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:56 smithi098 ceph-mon[128968]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-07-06T20:17:57.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.067+0000 7f803c750700 1 -- 172.21.15.69:0/1328711488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8034110ce0 msgr2=0x7f80341130c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:57.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.067+0000 7f803c750700 1 --2- 172.21.15.69:0/1328711488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8034110ce0 0x7f80341130c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f802c00b0a0 tx=0x7f802c00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:17:57.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.068+0000 7f803c750700 1 -- 172.21.15.69:0/1328711488 shutdown_connections 2023-07-06T20:17:57.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.068+0000 7f803c750700 1 --2- 172.21.15.69:0/1328711488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8034110ce0 0x7f80341130c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:57.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.068+0000 7f803c750700 1 --2- 172.21.15.69:0/1328711488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80340729e0 0x7f8034072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:57.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.068+0000 7f803c750700 1 --2- 172.21.15.69:0/1328711488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80340732f0 0x7f80341107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:57.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.068+0000 7f803c750700 1 -- 172.21.15.69:0/1328711488 >> 172.21.15.69:0/1328711488 conn(0x7f80340fb790 msgr2=0x7f80340fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:57.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.068+0000 7f803c750700 1 -- 172.21.15.69:0/1328711488 shutdown_connections 2023-07-06T20:17:57.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.068+0000 7f803c750700 1 -- 172.21.15.69:0/1328711488 wait complete. 2023-07-06T20:17:57.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.069+0000 7f803c750700 1 Processor -- start 2023-07-06T20:17:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.069+0000 7f803c750700 1 -- start start 2023-07-06T20:17:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.069+0000 7f803c750700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80340729e0 0x7f803419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.069+0000 7f803c750700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80340732f0 0x7f803419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.069+0000 7f803c750700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8034110ce0 0x7f80341a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803c750700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80341a1480 con 0x7f80340729e0 2023-07-06T20:17:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803c750700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80341a15f0 con 0x7f8034110ce0 2023-07-06T20:17:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803c750700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80341a17b0 con 0x7f80340732f0 2023-07-06T20:17:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803a4ec700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80340729e0 0x7f803419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:57.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803a4ec700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80340729e0 0x7f803419c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42112/0 (socket says 172.21.15.69:42112) 2023-07-06T20:17:57.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803a4ec700 1 -- 172.21.15.69:0/2660467468 learned_addr learned my addr 172.21.15.69:0/2660467468 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:57.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f8039ceb700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80340732f0 0x7f803419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:57.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803aced700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8034110ce0 0x7f80341a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:57.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803a4ec700 1 -- 172.21.15.69:0/2660467468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80340732f0 msgr2=0x7f803419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:57.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803a4ec700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80340732f0 0x7f803419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:57.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803a4ec700 1 -- 172.21.15.69:0/2660467468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8034110ce0 msgr2=0x7f80341a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803a4ec700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8034110ce0 0x7f80341a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803a4ec700 1 -- 172.21.15.69:0/2660467468 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c009d00 con 0x7f80340729e0 2023-07-06T20:17:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.070+0000 7f803a4ec700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80340729e0 0x7f803419c4c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f803000d980 tx=0x7f803000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.071+0000 7f802b7fe700 1 -- 172.21.15.69:0/2660467468 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030004220 con 0x7f80340729e0 2023-07-06T20:17:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.071+0000 7f803c750700 1 -- 172.21.15.69:0/2660467468 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80341a1a90 con 0x7f80340729e0 2023-07-06T20:17:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.071+0000 7f802b7fe700 1 -- 172.21.15.69:0/2660467468 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8030007640 con 0x7f80340729e0 2023-07-06T20:17:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.071+0000 7f802b7fe700 1 -- 172.21.15.69:0/2660467468 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030005690 con 0x7f80340729e0 2023-07-06T20:17:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.071+0000 7f803c750700 1 -- 172.21.15.69:0/2660467468 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80341a92f0 con 0x7f80340729e0 2023-07-06T20:17:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.072+0000 7f803c750700 1 -- 172.21.15.69:0/2660467468 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8034116460 con 0x7f80340729e0 2023-07-06T20:17:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.075+0000 7f802b7fe700 1 -- 172.21.15.69:0/2660467468 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8030004750 con 0x7f80340729e0 2023-07-06T20:17:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.077+0000 7f802b7fe700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80200a11e0 0x7f80200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.077+0000 7f802b7fe700 1 -- 172.21.15.69:0/2660467468 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f80300cc7c0 con 0x7f80340729e0 2023-07-06T20:17:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.077+0000 7f8039ceb700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80200a11e0 0x7f80200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.077+0000 7f802b7fe700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097] conn(0x7f80200aa9b0 0x7f80200acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.077+0000 7f802b7fe700 1 -- 172.21.15.69:0/2660467468 --> [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80200ad470 con 0x7f80200aa9b0 2023-07-06T20:17:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.077+0000 7f802b7fe700 1 -- 172.21.15.69:0/2660467468 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80300ceda0 con 0x7f80340729e0 2023-07-06T20:17:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.078+0000 7f803aced700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097] conn(0x7f80200aa9b0 0x7f80200acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.078+0000 7f8039ceb700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80200a11e0 0x7f80200a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f803419dae0 tx=0x7f8024009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.078+0000 7f803aced700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097] conn(0x7f80200aa9b0 0x7f80200acdc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.079+0000 7f802b7fe700 1 -- 172.21.15.69:0/2660467468 <== osd.9 v2:172.21.15.69:6826/2424610097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f80200ad470 con 0x7f80200aa9b0 2023-07-06T20:17:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.095+0000 7f803c750700 1 -- 172.21.15.69:0/2660467468 --> [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f803404e340 con 0x7f80200aa9b0 2023-07-06T20:17:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.096+0000 7f802b7fe700 1 -- 172.21.15.69:0/2660467468 <== osd.9 v2:172.21.15.69:6826/2424610097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f803404e340 con 0x7f80200aa9b0 2023-07-06T20:17:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.096+0000 7f803c750700 1 -- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097] conn(0x7f80200aa9b0 msgr2=0x7f80200acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:57.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.096+0000 7f803c750700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097] conn(0x7f80200aa9b0 0x7f80200acdc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:57.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.096+0000 7f803c750700 1 -- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80200a11e0 msgr2=0x7f80200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:57.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.096+0000 7f803c750700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80200a11e0 0x7f80200a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f803419dae0 tx=0x7f8024009260 comp rx=0 tx=0).stop 2023-07-06T20:17:57.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.096+0000 7f803c750700 1 -- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80340729e0 msgr2=0x7f803419c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:57.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.096+0000 7f803c750700 1 --2- 172.21.15.69:0/2660467468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80340729e0 0x7f803419c4c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f803000d980 tx=0x7f803000dc90 comp rx=0 tx=0).stop 2023-07-06T20:17:57.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.096+0000 7f803aced700 1 -- 172.21.15.69:0/2660467468 reap_dead start 2023-07-06T20:17:57.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.096+0000 7f803c750700 1 -- 172.21.15.69:0/2660467468 shutdown_connections 2023-07-06T20:17:57.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.096+0000 7f803c750700 1 -- 172.21.15.69:0/2660467468 >> 172.21.15.69:0/2660467468 conn(0x7f80340fb790 msgr2=0x7f80340fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:57.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.097+0000 7f803c750700 1 -- 172.21.15.69:0/2660467468 shutdown_connections 2023-07-06T20:17:57.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:57.097+0000 7f803c750700 1 -- 172.21.15.69:0/2660467468 wait complete. 2023-07-06T20:17:57.479 INFO:teuthology.orchestra.run.smithi069.stdout:77309411388 2023-07-06T20:17:57.479 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-06T20:17:58.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11, 2, 1]}]: dispatch 2023-07-06T20:17:58.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-07-06T20:17:58.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2023-07-06T20:17:58.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2023-07-06T20:17:58.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-07-06T20:17:58.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2023-07-06T20:17:58.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2023-07-06T20:17:58.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2023-07-06T20:17:58.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-07-06T20:17:58.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11, 2, 1]}]: dispatch 2023-07-06T20:17:58.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-07-06T20:17:58.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2023-07-06T20:17:58.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2023-07-06T20:17:58.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-07-06T20:17:58.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2023-07-06T20:17:58.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2023-07-06T20:17:58.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2023-07-06T20:17:58.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-07-06T20:17:58.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11, 2, 1]}]: dispatch 2023-07-06T20:17:58.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-07-06T20:17:58.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2023-07-06T20:17:58.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2023-07-06T20:17:58.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-07-06T20:17:58.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2023-07-06T20:17:58.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2023-07-06T20:17:58.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2023-07-06T20:17:58.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-07-06T20:17:58.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.765+0000 7f3794cf9700 1 -- 172.21.15.69:0/3245255117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37900729e0 msgr2=0x7f3790072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:58.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.765+0000 7f3794cf9700 1 --2- 172.21.15.69:0/3245255117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37900729e0 0x7f3790072db0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f37800098f0 tx=0x7f3780009c00 comp rx=0 tx=0).stop 2023-07-06T20:17:58.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.765+0000 7f3794cf9700 1 -- 172.21.15.69:0/3245255117 shutdown_connections 2023-07-06T20:17:58.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.765+0000 7f3794cf9700 1 --2- 172.21.15.69:0/3245255117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3790110ce0 0x7f37901130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.765+0000 7f3794cf9700 1 --2- 172.21.15.69:0/3245255117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37900729e0 0x7f3790072db0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.765+0000 7f3794cf9700 1 --2- 172.21.15.69:0/3245255117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37900732f0 0x7f37901107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.765+0000 7f3794cf9700 1 -- 172.21.15.69:0/3245255117 >> 172.21.15.69:0/3245255117 conn(0x7f37900fb790 msgr2=0x7f37900fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:58.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.765+0000 7f3794cf9700 1 -- 172.21.15.69:0/3245255117 shutdown_connections 2023-07-06T20:17:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.765+0000 7f3794cf9700 1 -- 172.21.15.69:0/3245255117 wait complete. 2023-07-06T20:17:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.766+0000 7f3794cf9700 1 Processor -- start 2023-07-06T20:17:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.766+0000 7f3794cf9700 1 -- start start 2023-07-06T20:17:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.766+0000 7f3794cf9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37900729e0 0x7f379019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.766+0000 7f3794cf9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37900732f0 0x7f379019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f3794cf9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3790110ce0 0x7f37901a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f3794cf9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37901a1420 con 0x7f37900729e0 2023-07-06T20:17:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f3794cf9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37901a1590 con 0x7f37900732f0 2023-07-06T20:17:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f3794cf9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37901a1750 con 0x7f3790110ce0 2023-07-06T20:17:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f378e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37900729e0 0x7f379019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:58.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f378dd9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37900732f0 0x7f379019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:58.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f378dd9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37900732f0 0x7f379019c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:58152/0 (socket says 172.21.15.69:58152) 2023-07-06T20:17:58.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f378e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37900729e0 0x7f379019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42138/0 (socket says 172.21.15.69:42138) 2023-07-06T20:17:58.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f378dd9b700 1 -- 172.21.15.69:0/4239576629 learned_addr learned my addr 172.21.15.69:0/4239576629 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:58.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f378ed9d700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3790110ce0 0x7f37901a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:58.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f378dd9b700 1 -- 172.21.15.69:0/4239576629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3790110ce0 msgr2=0x7f37901a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:58.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f378dd9b700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3790110ce0 0x7f37901a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f378dd9b700 1 -- 172.21.15.69:0/4239576629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37900729e0 msgr2=0x7f379019c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:58.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f378dd9b700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37900729e0 0x7f379019c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f378dd9b700 1 -- 172.21.15.69:0/4239576629 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37840095f0 con 0x7f37900732f0 2023-07-06T20:17:58.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.767+0000 7f378dd9b700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37900732f0 0x7f379019c9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f378400ec10 tx=0x7f378400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:58.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.768+0000 7f377f7fe700 1 -- 172.21.15.69:0/4239576629 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400cc90 con 0x7f37900732f0 2023-07-06T20:17:58.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.768+0000 7f377f7fe700 1 -- 172.21.15.69:0/4239576629 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37840044e0 con 0x7f37900732f0 2023-07-06T20:17:58.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.768+0000 7f377f7fe700 1 -- 172.21.15.69:0/4239576629 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784010430 con 0x7f37900732f0 2023-07-06T20:17:58.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.768+0000 7f3794cf9700 1 -- 172.21.15.69:0/4239576629 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3780009520 con 0x7f37900732f0 2023-07-06T20:17:58.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.768+0000 7f3794cf9700 1 -- 172.21.15.69:0/4239576629 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37901a90c0 con 0x7f37900732f0 2023-07-06T20:17:58.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.769+0000 7f3794cf9700 1 -- 172.21.15.69:0/4239576629 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f37901164a0 con 0x7f37900732f0 2023-07-06T20:17:58.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.771+0000 7f377f7fe700 1 -- 172.21.15.69:0/4239576629 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3784003fb0 con 0x7f37900732f0 2023-07-06T20:17:58.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.772+0000 7f377f7fe700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f37780a11e0 0x7f37780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:58.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.773+0000 7f377f7fe700 1 -- 172.21.15.69:0/4239576629 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f3784014070 con 0x7f37900732f0 2023-07-06T20:17:58.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.773+0000 7f378e59c700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f37780a11e0 0x7f37780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:58.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.773+0000 7f377f7fe700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049] conn(0x7f37780aad90 0x7f37780ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:58.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.773+0000 7f377f7fe700 1 -- 172.21.15.69:0/4239576629 --> [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37780ad8f0 con 0x7f37780aad90 2023-07-06T20:17:58.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.773+0000 7f377f7fe700 1 -- 172.21.15.69:0/4239576629 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37840cdcd0 con 0x7f37900732f0 2023-07-06T20:17:58.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.773+0000 7f378e59c700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f37780a11e0 0x7f37780a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3780005e90 tx=0x7f3780019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:58.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.773+0000 7f378ed9d700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049] conn(0x7f37780aad90 0x7f37780ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:58.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.774+0000 7f378ed9d700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049] conn(0x7f37780aad90 0x7f37780ad220 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:58.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.775+0000 7f377f7fe700 1 -- 172.21.15.69:0/4239576629 <== osd.10 v2:172.21.15.98:6824/2499526049 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f37780ad8f0 con 0x7f37780aad90 2023-07-06T20:17:58.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.791+0000 7f3794cf9700 1 -- 172.21.15.69:0/4239576629 --> [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f379004e340 con 0x7f37780aad90 2023-07-06T20:17:58.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.792+0000 7f377f7fe700 1 -- 172.21.15.69:0/4239576629 <== osd.10 v2:172.21.15.98:6824/2499526049 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f379004e340 con 0x7f37780aad90 2023-07-06T20:17:58.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.792+0000 7f3794cf9700 1 -- 172.21.15.69:0/4239576629 >> [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049] conn(0x7f37780aad90 msgr2=0x7f37780ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:58.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.792+0000 7f3794cf9700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049] conn(0x7f37780aad90 0x7f37780ad220 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.792+0000 7f3794cf9700 1 -- 172.21.15.69:0/4239576629 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f37780a11e0 msgr2=0x7f37780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:58.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.792+0000 7f3794cf9700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f37780a11e0 0x7f37780a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3780005e90 tx=0x7f3780019040 comp rx=0 tx=0).stop 2023-07-06T20:17:58.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.792+0000 7f3794cf9700 1 -- 172.21.15.69:0/4239576629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37900732f0 msgr2=0x7f379019c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:58.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.792+0000 7f3794cf9700 1 --2- 172.21.15.69:0/4239576629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37900732f0 0x7f379019c9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f378400ec10 tx=0x7f378400ef20 comp rx=0 tx=0).stop 2023-07-06T20:17:58.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.792+0000 7f378ed9d700 1 -- 172.21.15.69:0/4239576629 reap_dead start 2023-07-06T20:17:58.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.792+0000 7f3794cf9700 1 -- 172.21.15.69:0/4239576629 shutdown_connections 2023-07-06T20:17:58.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.792+0000 7f3794cf9700 1 -- 172.21.15.69:0/4239576629 >> 172.21.15.69:0/4239576629 conn(0x7f37900fb790 msgr2=0x7f37900fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:58.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.792+0000 7f3794cf9700 1 -- 172.21.15.69:0/4239576629 shutdown_connections 2023-07-06T20:17:58.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.793+0000 7f3794cf9700 1 -- 172.21.15.69:0/4239576629 wait complete. 2023-07-06T20:17:58.874 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.872+0000 7f9647d6b700 1 -- 172.21.15.69:0/3855752847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9640072980 msgr2=0x7f9640072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:58.874 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.872+0000 7f9647d6b700 1 --2- 172.21.15.69:0/3855752847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9640072980 0x7f9640072d50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f963c009990 tx=0x7f963c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.872+0000 7f9647d6b700 1 -- 172.21.15.69:0/3855752847 shutdown_connections 2023-07-06T20:17:58.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.872+0000 7f9647d6b700 1 --2- 172.21.15.69:0/3855752847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f964010c570 0x7f964010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.872+0000 7f9647d6b700 1 --2- 172.21.15.69:0/3855752847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9640072980 0x7f9640072d50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.872+0000 7f9647d6b700 1 --2- 172.21.15.69:0/3855752847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9640073290 0x7f964010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.872+0000 7f9647d6b700 1 -- 172.21.15.69:0/3855752847 >> 172.21.15.69:0/3855752847 conn(0x7f96400fb790 msgr2=0x7f96400fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:58.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.872+0000 7f9647d6b700 1 -- 172.21.15.69:0/3855752847 shutdown_connections 2023-07-06T20:17:58.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.872+0000 7f9647d6b700 1 -- 172.21.15.69:0/3855752847 wait complete. 2023-07-06T20:17:58.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.873+0000 7f9647d6b700 1 Processor -- start 2023-07-06T20:17:58.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.873+0000 7f9647d6b700 1 -- start start 2023-07-06T20:17:58.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.874+0000 7f9647d6b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9640072980 0x7f9640197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:58.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.874+0000 7f9647d6b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9640073290 0x7f96401984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:58.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.874+0000 7f9647d6b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f964010c570 0x7f964019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:58.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.874+0000 7f9647d6b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964019cf40 con 0x7f9640073290 2023-07-06T20:17:58.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.874+0000 7f9647d6b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964019d0b0 con 0x7f9640072980 2023-07-06T20:17:58.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.874+0000 7f9647d6b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964019d270 con 0x7f964010c570 2023-07-06T20:17:58.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.874+0000 7f9645306700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9640073290 0x7f96401984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:58.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.874+0000 7f9645306700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9640073290 0x7f96401984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42148/0 (socket says 172.21.15.69:42148) 2023-07-06T20:17:58.878 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.874+0000 7f9645306700 1 -- 172.21.15.69:0/4236157173 learned_addr learned my addr 172.21.15.69:0/4236157173 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:58.878 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.875+0000 7f9645306700 1 -- 172.21.15.69:0/4236157173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f964010c570 msgr2=0x7f964019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:58.878 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.875+0000 7f9645b07700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9640072980 0x7f9640197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:58.878 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.875+0000 7f9645306700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f964010c570 0x7f964019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.875+0000 7f9645306700 1 -- 172.21.15.69:0/4236157173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9640072980 msgr2=0x7f9640197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:58.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.875+0000 7f9645306700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9640072980 0x7f9640197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.875+0000 7f9645306700 1 -- 172.21.15.69:0/4236157173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c0095f0 con 0x7f9640073290 2023-07-06T20:17:58.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.875+0000 7f9645b07700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9640072980 0x7f9640197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:17:58.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.875+0000 7f9645306700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9640073290 0x7f96401984c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f963000e920 tx=0x7f963000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:58.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.875+0000 7f9636ffd700 1 -- 172.21.15.69:0/4236157173 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630009790 con 0x7f9640073290 2023-07-06T20:17:58.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.875+0000 7f9647d6b700 1 -- 172.21.15.69:0/4236157173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964019d550 con 0x7f9640073290 2023-07-06T20:17:58.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.876+0000 7f9636ffd700 1 -- 172.21.15.69:0/4236157173 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f963000cbd0 con 0x7f9640073290 2023-07-06T20:17:58.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.876+0000 7f9636ffd700 1 -- 172.21.15.69:0/4236157173 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630018680 con 0x7f9640073290 2023-07-06T20:17:58.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.876+0000 7f9647d6b700 1 -- 172.21.15.69:0/4236157173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96401a4db0 con 0x7f9640073290 2023-07-06T20:17:58.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.877+0000 7f9647d6b700 1 -- 172.21.15.69:0/4236157173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96401120d0 con 0x7f9640073290 2023-07-06T20:17:58.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.879+0000 7f9636ffd700 1 -- 172.21.15.69:0/4236157173 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f963000cd40 con 0x7f9640073290 2023-07-06T20:17:58.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.881+0000 7f9636ffd700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f962c0a1260 0x7f962c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:58.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.881+0000 7f9636ffd700 1 -- 172.21.15.69:0/4236157173 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9630014070 con 0x7f9640073290 2023-07-06T20:17:58.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.881+0000 7f9645b07700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f962c0a1260 0x7f962c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:58.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.881+0000 7f9636ffd700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140] conn(0x7f962c0aae10 0x7f962c0ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:58.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.882+0000 7f9636ffd700 1 -- 172.21.15.69:0/4236157173 --> [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f962c0ad970 con 0x7f962c0aae10 2023-07-06T20:17:58.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.882+0000 7f9636ffd700 1 -- 172.21.15.69:0/4236157173 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96300cd470 con 0x7f9640073290 2023-07-06T20:17:58.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.882+0000 7f9646308700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140] conn(0x7f962c0aae10 0x7f962c0ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:58.885 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.882+0000 7f9645b07700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f962c0a1260 0x7f962c0a3710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f963c005cd0 tx=0x7f963c005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:58.885 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.883+0000 7f9646308700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140] conn(0x7f962c0aae10 0x7f962c0ad2a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:58.885 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.883+0000 7f9636ffd700 1 -- 172.21.15.69:0/4236157173 <== osd.7 v2:172.21.15.98:6816/238502140 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f962c0ad970 con 0x7f962c0aae10 2023-07-06T20:17:58.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.904+0000 7f9647d6b700 1 -- 172.21.15.69:0/4236157173 --> [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f964004e340 con 0x7f962c0aae10 2023-07-06T20:17:58.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.905+0000 7f9636ffd700 1 -- 172.21.15.69:0/4236157173 <== osd.7 v2:172.21.15.98:6816/238502140 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f964004e340 con 0x7f962c0aae10 2023-07-06T20:17:58.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.906+0000 7f9647d6b700 1 -- 172.21.15.69:0/4236157173 >> [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140] conn(0x7f962c0aae10 msgr2=0x7f962c0ad2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:58.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.906+0000 7f9647d6b700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140] conn(0x7f962c0aae10 0x7f962c0ad2a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.906+0000 7f9647d6b700 1 -- 172.21.15.69:0/4236157173 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f962c0a1260 msgr2=0x7f962c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:58.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.906+0000 7f9647d6b700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f962c0a1260 0x7f962c0a3710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f963c005cd0 tx=0x7f963c005c60 comp rx=0 tx=0).stop 2023-07-06T20:17:58.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.906+0000 7f9647d6b700 1 -- 172.21.15.69:0/4236157173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9640073290 msgr2=0x7f96401984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:58.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.906+0000 7f9647d6b700 1 --2- 172.21.15.69:0/4236157173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9640073290 0x7f96401984c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f963000e920 tx=0x7f963000ece0 comp rx=0 tx=0).stop 2023-07-06T20:17:58.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.906+0000 7f9646308700 1 -- 172.21.15.69:0/4236157173 reap_dead start 2023-07-06T20:17:58.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.906+0000 7f9647d6b700 1 -- 172.21.15.69:0/4236157173 shutdown_connections 2023-07-06T20:17:58.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.906+0000 7f9647d6b700 1 -- 172.21.15.69:0/4236157173 >> 172.21.15.69:0/4236157173 conn(0x7f96400fb790 msgr2=0x7f9640103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:58.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.906+0000 7f9647d6b700 1 -- 172.21.15.69:0/4236157173 shutdown_connections 2023-07-06T20:17:58.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:58.906+0000 7f9647d6b700 1 -- 172.21.15.69:0/4236157173 wait complete. 2023-07-06T20:17:59.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:58 smithi069 ceph-mon[126399]: pgmap v283: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-06T20:17:59.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11, 2, 1]}]': finished 2023-07-06T20:17:59.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-07-06T20:17:59.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2023-07-06T20:17:59.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2023-07-06T20:17:59.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-07-06T20:17:59.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2023-07-06T20:17:59.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2023-07-06T20:17:59.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2023-07-06T20:17:59.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-07-06T20:17:59.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:58 smithi069 ceph-mon[126399]: osdmap e84: 12 total, 12 up, 12 in 2023-07-06T20:17:59.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:58 smithi156 ceph-mon[129280]: pgmap v283: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-06T20:17:59.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11, 2, 1]}]': finished 2023-07-06T20:17:59.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-07-06T20:17:59.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2023-07-06T20:17:59.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2023-07-06T20:17:59.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-07-06T20:17:59.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2023-07-06T20:17:59.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2023-07-06T20:17:59.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2023-07-06T20:17:59.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-07-06T20:17:59.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:58 smithi156 ceph-mon[129280]: osdmap e84: 12 total, 12 up, 12 in 2023-07-06T20:17:59.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:58 smithi098 ceph-mon[128968]: pgmap v283: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-06T20:17:59.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11, 2, 1]}]': finished 2023-07-06T20:17:59.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-07-06T20:17:59.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2023-07-06T20:17:59.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2023-07-06T20:17:59.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-07-06T20:17:59.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2023-07-06T20:17:59.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2023-07-06T20:17:59.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2023-07-06T20:17:59.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-07-06T20:17:59.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:58 smithi098 ceph-mon[128968]: osdmap e84: 12 total, 12 up, 12 in 2023-07-06T20:17:59.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.255+0000 7f36c8f70700 1 -- 172.21.15.69:0/2860451570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36c40fff30 msgr2=0x7f36c4100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:59.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.255+0000 7f36c8f70700 1 --2- 172.21.15.69:0/2860451570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36c40fff30 0x7f36c4100300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36b40099c0 tx=0x7f36b4009cd0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.256+0000 7f36c8f70700 1 -- 172.21.15.69:0/2860451570 shutdown_connections 2023-07-06T20:17:59.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.256+0000 7f36c8f70700 1 --2- 172.21.15.69:0/2860451570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36c4104fd0 0x7f36c410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.256+0000 7f36c8f70700 1 --2- 172.21.15.69:0/2860451570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36c40fff30 0x7f36c4100300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.256+0000 7f36c8f70700 1 --2- 172.21.15.69:0/2860451570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c41008d0 0x7f36c41048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.256+0000 7f36c8f70700 1 -- 172.21.15.69:0/2860451570 >> 172.21.15.69:0/2860451570 conn(0x7f36c40fb790 msgr2=0x7f36c40fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:59.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.256+0000 7f36c8f70700 1 -- 172.21.15.69:0/2860451570 shutdown_connections 2023-07-06T20:17:59.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.256+0000 7f36c8f70700 1 -- 172.21.15.69:0/2860451570 wait complete. 2023-07-06T20:17:59.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.257+0000 7f36c8f70700 1 Processor -- start 2023-07-06T20:17:59.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.257+0000 7f36c8f70700 1 -- start start 2023-07-06T20:17:59.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.257+0000 7f36c8f70700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36c40fff30 0x7f36c4197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:59.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.257+0000 7f36c8f70700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c41008d0 0x7f36c4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:59.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.257+0000 7f36c8f70700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36c4104fd0 0x7f36c419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:59.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.257+0000 7f36c8f70700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c419cf80 con 0x7f36c41008d0 2023-07-06T20:17:59.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.257+0000 7f36c8f70700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c419d0f0 con 0x7f36c40fff30 2023-07-06T20:17:59.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.257+0000 7f36c8f70700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c419d2b0 con 0x7f36c4104fd0 2023-07-06T20:17:59.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.257+0000 7f36c1d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c41008d0 0x7f36c4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:59.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36c1d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c41008d0 0x7f36c4198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42174/0 (socket says 172.21.15.69:42174) 2023-07-06T20:17:59.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36c259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36c40fff30 0x7f36c4197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:59.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36c1d9b700 1 -- 172.21.15.69:0/1946313129 learned_addr learned my addr 172.21.15.69:0/1946313129 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:59.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36c2d9d700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36c4104fd0 0x7f36c419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:59.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36c1d9b700 1 -- 172.21.15.69:0/1946313129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36c4104fd0 msgr2=0x7f36c419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:59.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36c1d9b700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36c4104fd0 0x7f36c419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36c1d9b700 1 -- 172.21.15.69:0/1946313129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36c40fff30 msgr2=0x7f36c4197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:59.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36c1d9b700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36c40fff30 0x7f36c4197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36c1d9b700 1 -- 172.21.15.69:0/1946313129 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b8009520 con 0x7f36c41008d0 2023-07-06T20:17:59.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36c1d9b700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c41008d0 0x7f36c4198500 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f36b800ea30 tx=0x7f36b800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:59.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36b37fe700 1 -- 172.21.15.69:0/1946313129 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b800cbd0 con 0x7f36c41008d0 2023-07-06T20:17:59.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36c8f70700 1 -- 172.21.15.69:0/1946313129 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b40095f0 con 0x7f36c41008d0 2023-07-06T20:17:59.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.258+0000 7f36b37fe700 1 -- 172.21.15.69:0/1946313129 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36b800eed0 con 0x7f36c41008d0 2023-07-06T20:17:59.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.259+0000 7f36b37fe700 1 -- 172.21.15.69:0/1946313129 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8003d20 con 0x7f36c41008d0 2023-07-06T20:17:59.265 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.259+0000 7f36c8f70700 1 -- 172.21.15.69:0/1946313129 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c41a4c20 con 0x7f36c41008d0 2023-07-06T20:17:59.265 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.260+0000 7f36c8f70700 1 -- 172.21.15.69:0/1946313129 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f36c410fe90 con 0x7f36c41008d0 2023-07-06T20:17:59.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.262+0000 7f36b37fe700 1 -- 172.21.15.69:0/1946313129 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f36b8003e80 con 0x7f36c41008d0 2023-07-06T20:17:59.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.263+0000 7f36b37fe700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f36ac0a1260 0x7f36ac0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:59.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.263+0000 7f36b37fe700 1 -- 172.21.15.69:0/1946313129 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f36b8014070 con 0x7f36c41008d0 2023-07-06T20:17:59.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.263+0000 7f36c259c700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f36ac0a1260 0x7f36ac0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:59.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.263+0000 7f36b37fe700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813] conn(0x7f36ac0aae10 0x7f36ac0ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:59.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.263+0000 7f36b37fe700 1 -- 172.21.15.69:0/1946313129 --> [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36ac0ad970 con 0x7f36ac0aae10 2023-07-06T20:17:59.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.263+0000 7f36b37fe700 1 -- 172.21.15.69:0/1946313129 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36b80cd200 con 0x7f36c41008d0 2023-07-06T20:17:59.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.264+0000 7f36c259c700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f36ac0a1260 0x7f36ac0a3710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009fd0 tx=0x7f36b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:59.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.264+0000 7f36c2d9d700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813] conn(0x7f36ac0aae10 0x7f36ac0ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:59.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.264+0000 7f36c2d9d700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813] conn(0x7f36ac0aae10 0x7f36ac0ad2a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:59.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.265+0000 7f36b37fe700 1 -- 172.21.15.69:0/1946313129 <== osd.8 v2:172.21.15.156:6816/3522956813 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f36ac0ad970 con 0x7f36ac0aae10 2023-07-06T20:17:59.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.281+0000 7f36c8f70700 1 -- 172.21.15.69:0/1946313129 --> [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f36c404e340 con 0x7f36ac0aae10 2023-07-06T20:17:59.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.282+0000 7f36b37fe700 1 -- 172.21.15.69:0/1946313129 <== osd.8 v2:172.21.15.156:6816/3522956813 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f36c404e340 con 0x7f36ac0aae10 2023-07-06T20:17:59.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.283+0000 7f36c8f70700 1 -- 172.21.15.69:0/1946313129 >> [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813] conn(0x7f36ac0aae10 msgr2=0x7f36ac0ad2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:59.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.283+0000 7f36c8f70700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813] conn(0x7f36ac0aae10 0x7f36ac0ad2a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.283+0000 7f36c8f70700 1 -- 172.21.15.69:0/1946313129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f36ac0a1260 msgr2=0x7f36ac0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:59.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.283+0000 7f36c8f70700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f36ac0a1260 0x7f36ac0a3710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009fd0 tx=0x7f36b4005fb0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.283+0000 7f36c8f70700 1 -- 172.21.15.69:0/1946313129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c41008d0 msgr2=0x7f36c4198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:59.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.283+0000 7f36c8f70700 1 --2- 172.21.15.69:0/1946313129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36c41008d0 0x7f36c4198500 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f36b800ea30 tx=0x7f36b800ed40 comp rx=0 tx=0).stop 2023-07-06T20:17:59.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.283+0000 7f36c2d9d700 1 -- 172.21.15.69:0/1946313129 reap_dead start 2023-07-06T20:17:59.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.283+0000 7f36c8f70700 1 -- 172.21.15.69:0/1946313129 shutdown_connections 2023-07-06T20:17:59.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.283+0000 7f36c8f70700 1 -- 172.21.15.69:0/1946313129 >> 172.21.15.69:0/1946313129 conn(0x7f36c40fb790 msgr2=0x7f36c410be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:59.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.283+0000 7f36c8f70700 1 -- 172.21.15.69:0/1946313129 shutdown_connections 2023-07-06T20:17:59.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.283+0000 7f36c8f70700 1 -- 172.21.15.69:0/1946313129 wait complete. 2023-07-06T20:17:59.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.521+0000 7f25e24d7700 1 -- 172.21.15.69:0/2929381677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25dc1008d0 msgr2=0x7f25dc1048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:59.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.521+0000 7f25e24d7700 1 --2- 172.21.15.69:0/2929381677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25dc1008d0 0x7f25dc1048a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f25d00099c0 tx=0x7f25d0009cd0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.521+0000 7f25e24d7700 1 -- 172.21.15.69:0/2929381677 shutdown_connections 2023-07-06T20:17:59.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.521+0000 7f25e24d7700 1 --2- 172.21.15.69:0/2929381677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc104fd0 0x7f25dc10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.521+0000 7f25e24d7700 1 --2- 172.21.15.69:0/2929381677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25dc0fff30 0x7f25dc100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.521+0000 7f25e24d7700 1 --2- 172.21.15.69:0/2929381677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25dc1008d0 0x7f25dc1048a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.521+0000 7f25e24d7700 1 -- 172.21.15.69:0/2929381677 >> 172.21.15.69:0/2929381677 conn(0x7f25dc0fb790 msgr2=0x7f25dc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:59.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.522+0000 7f25e24d7700 1 -- 172.21.15.69:0/2929381677 shutdown_connections 2023-07-06T20:17:59.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.522+0000 7f25e24d7700 1 -- 172.21.15.69:0/2929381677 wait complete. 2023-07-06T20:17:59.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.522+0000 7f25e24d7700 1 Processor -- start 2023-07-06T20:17:59.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.523+0000 7f25e24d7700 1 -- start start 2023-07-06T20:17:59.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.523+0000 7f25e24d7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc0fff30 0x7f25dc198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:59.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.523+0000 7f25e24d7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25dc1008d0 0x7f25dc198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:59.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.523+0000 7f25e24d7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25dc104fd0 0x7f25dc19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:59.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.523+0000 7f25e24d7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc19cfe0 con 0x7f25dc1008d0 2023-07-06T20:17:59.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.523+0000 7f25e24d7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc19d150 con 0x7f25dc0fff30 2023-07-06T20:17:59.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.523+0000 7f25e24d7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc19d310 con 0x7f25dc104fd0 2023-07-06T20:17:59.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.523+0000 7f25db7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25dc1008d0 0x7f25dc198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:59.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.523+0000 7f25dbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc0fff30 0x7f25dc198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:59.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.523+0000 7f25db7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25dc1008d0 0x7f25dc198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42190/0 (socket says 172.21.15.69:42190) 2023-07-06T20:17:59.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.523+0000 7f25db7fe700 1 -- 172.21.15.69:0/3896711932 learned_addr learned my addr 172.21.15.69:0/3896711932 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:17:59.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.523+0000 7f25e0a74700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25dc104fd0 0x7f25dc19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:59.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.524+0000 7f25db7fe700 1 -- 172.21.15.69:0/3896711932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25dc104fd0 msgr2=0x7f25dc19c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:59.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.524+0000 7f25db7fe700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25dc104fd0 0x7f25dc19c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.524+0000 7f25db7fe700 1 -- 172.21.15.69:0/3896711932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc0fff30 msgr2=0x7f25dc198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:59.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.524+0000 7f25db7fe700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25dc0fff30 0x7f25dc198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.524+0000 7f25db7fe700 1 -- 172.21.15.69:0/3896711932 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d00095f0 con 0x7f25dc1008d0 2023-07-06T20:17:59.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.524+0000 7f25db7fe700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25dc1008d0 0x7f25dc198560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009fd0 tx=0x7f25d000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:59.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.524+0000 7f25d97fa700 1 -- 172.21.15.69:0/3896711932 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d001d070 con 0x7f25dc1008d0 2023-07-06T20:17:59.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.524+0000 7f25e24d7700 1 -- 172.21.15.69:0/3896711932 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25dc19d590 con 0x7f25dc1008d0 2023-07-06T20:17:59.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.524+0000 7f25d97fa700 1 -- 172.21.15.69:0/3896711932 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25d0005050 con 0x7f25dc1008d0 2023-07-06T20:17:59.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.524+0000 7f25d97fa700 1 -- 172.21.15.69:0/3896711932 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0004a90 con 0x7f25dc1008d0 2023-07-06T20:17:59.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.524+0000 7f25e24d7700 1 -- 172.21.15.69:0/3896711932 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25dc1a4e50 con 0x7f25dc1008d0 2023-07-06T20:17:59.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.525+0000 7f25e24d7700 1 -- 172.21.15.69:0/3896711932 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25dc19d860 con 0x7f25dc1008d0 2023-07-06T20:17:59.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.527+0000 7f25d97fa700 1 -- 172.21.15.69:0/3896711932 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f25d0005470 con 0x7f25dc1008d0 2023-07-06T20:17:59.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.529+0000 7f25d97fa700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f25c40a11e0 0x7f25c40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:59.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.529+0000 7f25d97fa700 1 -- 172.21.15.69:0/3896711932 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f25d00cdf80 con 0x7f25dc1008d0 2023-07-06T20:17:59.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.529+0000 7f25dbfff700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f25c40a11e0 0x7f25c40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:59.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.529+0000 7f25d97fa700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539] conn(0x7f25c40aad90 0x7f25c40ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:17:59.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.529+0000 7f25d97fa700 1 -- 172.21.15.69:0/3896711932 --> [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25c40ad8f0 con 0x7f25c40aad90 2023-07-06T20:17:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.529+0000 7f25d97fa700 1 -- 172.21.15.69:0/3896711932 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25d00ce2f0 con 0x7f25dc1008d0 2023-07-06T20:17:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.529+0000 7f25e0a74700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539] conn(0x7f25c40aad90 0x7f25c40ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:17:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.529+0000 7f25dbfff700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f25c40a11e0 0x7f25c40a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f25cc007be0 tx=0x7f25cc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.530+0000 7f25e0a74700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539] conn(0x7f25c40aad90 0x7f25c40ad220 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:17:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.531+0000 7f25d97fa700 1 -- 172.21.15.69:0/3896711932 <== osd.11 v2:172.21.15.156:6824/266940539 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f25c40ad8f0 con 0x7f25c40aad90 2023-07-06T20:17:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.547+0000 7f25e24d7700 1 -- 172.21.15.69:0/3896711932 --> [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f25dc10fed0 con 0x7f25c40aad90 2023-07-06T20:17:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.548+0000 7f25d97fa700 1 -- 172.21.15.69:0/3896711932 <== osd.11 v2:172.21.15.156:6824/266940539 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f25d400f040 con 0x7f25c40aad90 2023-07-06T20:17:59.718 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.548+0000 7f25e24d7700 1 -- 172.21.15.69:0/3896711932 >> [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539] conn(0x7f25c40aad90 msgr2=0x7f25c40ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:59.718 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.548+0000 7f25e24d7700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539] conn(0x7f25c40aad90 0x7f25c40ad220 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.548+0000 7f25e24d7700 1 -- 172.21.15.69:0/3896711932 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f25c40a11e0 msgr2=0x7f25c40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:59.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.548+0000 7f25e24d7700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f25c40a11e0 0x7f25c40a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f25cc007be0 tx=0x7f25cc0073d0 comp rx=0 tx=0).stop 2023-07-06T20:17:59.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.548+0000 7f25e24d7700 1 -- 172.21.15.69:0/3896711932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25dc1008d0 msgr2=0x7f25dc198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:17:59.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.548+0000 7f25e24d7700 1 --2- 172.21.15.69:0/3896711932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25dc1008d0 0x7f25dc198560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009fd0 tx=0x7f25d000f690 comp rx=0 tx=0).stop 2023-07-06T20:17:59.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.548+0000 7f25e0a74700 1 -- 172.21.15.69:0/3896711932 reap_dead start 2023-07-06T20:17:59.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.548+0000 7f25e24d7700 1 -- 172.21.15.69:0/3896711932 shutdown_connections 2023-07-06T20:17:59.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.548+0000 7f25e24d7700 1 -- 172.21.15.69:0/3896711932 >> 172.21.15.69:0/3896711932 conn(0x7f25dc0fb790 msgr2=0x7f25dc0fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:17:59.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.549+0000 7f25e24d7700 1 -- 172.21.15.69:0/3896711932 shutdown_connections 2023-07-06T20:17:59.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:17:59.549+0000 7f25e24d7700 1 -- 172.21.15.69:0/3896711932 wait complete. 2023-07-06T20:17:59.819 INFO:teuthology.orchestra.run.smithi069.stdout:249108103199 2023-07-06T20:17:59.819 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-06T20:18:00.007 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:17:59 smithi069 ceph-mon[126399]: osdmap e85: 12 total, 12 up, 12 in 2023-07-06T20:18:00.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:17:59 smithi156 ceph-mon[129280]: osdmap e85: 12 total, 12 up, 12 in 2023-07-06T20:18:00.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:17:59 smithi098 ceph-mon[128968]: osdmap e85: 12 total, 12 up, 12 in 2023-07-06T20:18:00.171 INFO:teuthology.orchestra.run.smithi069.stdout:219043332132 2023-07-06T20:18:00.171 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-06T20:18:00.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.525+0000 7f80bd55d700 1 -- 172.21.15.69:0/3612985757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b80fff30 msgr2=0x7f80b8100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.525+0000 7f80bd55d700 1 --2- 172.21.15.69:0/3612985757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b80fff30 0x7f80b8100300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f80a80099c0 tx=0x7f80a8009cd0 comp rx=0 tx=0).stop 2023-07-06T20:18:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.525+0000 7f80bd55d700 1 -- 172.21.15.69:0/3612985757 shutdown_connections 2023-07-06T20:18:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.525+0000 7f80bd55d700 1 --2- 172.21.15.69:0/3612985757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80b8104fd0 0x7f80b810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.525+0000 7f80bd55d700 1 --2- 172.21.15.69:0/3612985757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b80fff30 0x7f80b8100300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.525+0000 7f80bd55d700 1 --2- 172.21.15.69:0/3612985757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b81008d0 0x7f80b81048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.525+0000 7f80bd55d700 1 -- 172.21.15.69:0/3612985757 >> 172.21.15.69:0/3612985757 conn(0x7f80b80fb790 msgr2=0x7f80b80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.525+0000 7f80bd55d700 1 -- 172.21.15.69:0/3612985757 shutdown_connections 2023-07-06T20:18:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.525+0000 7f80bd55d700 1 -- 172.21.15.69:0/3612985757 wait complete. 2023-07-06T20:18:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.526+0000 7f80bd55d700 1 Processor -- start 2023-07-06T20:18:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.527+0000 7f80bd55d700 1 -- start start 2023-07-06T20:18:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.527+0000 7f80bd55d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b80fff30 0x7f80b81980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.527+0000 7f80bd55d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b81008d0 0x7f80b81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.527+0000 7f80bd55d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80b8104fd0 0x7f80b819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.527+0000 7f80bd55d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819cfd0 con 0x7f80b80fff30 2023-07-06T20:18:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.527+0000 7f80bd55d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819d140 con 0x7f80b8104fd0 2023-07-06T20:18:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.527+0000 7f80bd55d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819d300 con 0x7f80b81008d0 2023-07-06T20:18:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.527+0000 7f80b67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b81008d0 0x7f80b81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.527+0000 7f80b67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b81008d0 0x7f80b81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:34058/0 (socket says 172.21.15.69:34058) 2023-07-06T20:18:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.527+0000 7f80b67fc700 1 -- 172.21.15.69:0/3382064206 learned_addr learned my addr 172.21.15.69:0/3382064206 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.527+0000 7f80b6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b80fff30 0x7f80b81980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.527+0000 7f80b77fe700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80b8104fd0 0x7f80b819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:00.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.528+0000 7f80b67fc700 1 -- 172.21.15.69:0/3382064206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80b8104fd0 msgr2=0x7f80b819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:00.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.528+0000 7f80b67fc700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80b8104fd0 0x7f80b819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:00.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.528+0000 7f80b67fc700 1 -- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b80fff30 msgr2=0x7f80b81980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:00.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.528+0000 7f80b67fc700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b80fff30 0x7f80b81980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:00.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.528+0000 7f80b67fc700 1 -- 172.21.15.69:0/3382064206 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a80095f0 con 0x7f80b81008d0 2023-07-06T20:18:00.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.528+0000 7f80b6ffd700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b80fff30 0x7f80b81980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:00.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.528+0000 7f80b67fc700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b81008d0 0x7f80b81985f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00e970 tx=0x7f80ac00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:00.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.528+0000 7f809ffff700 1 -- 172.21.15.69:0/3382064206 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac009790 con 0x7f80b81008d0 2023-07-06T20:18:00.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.528+0000 7f80bd55d700 1 -- 172.21.15.69:0/3382064206 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b819d5e0 con 0x7f80b81008d0 2023-07-06T20:18:00.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.528+0000 7f809ffff700 1 -- 172.21.15.69:0/3382064206 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80ac00c7e0 con 0x7f80b81008d0 2023-07-06T20:18:00.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.528+0000 7f809ffff700 1 -- 172.21.15.69:0/3382064206 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0054e0 con 0x7f80b81008d0 2023-07-06T20:18:00.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.529+0000 7f80bd55d700 1 -- 172.21.15.69:0/3382064206 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b81a4e40 con 0x7f80b81008d0 2023-07-06T20:18:00.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.529+0000 7f80bd55d700 1 -- 172.21.15.69:0/3382064206 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f80b810fef0 con 0x7f80b81008d0 2023-07-06T20:18:00.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.533+0000 7f809ffff700 1 -- 172.21.15.69:0/3382064206 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f80ac00c950 con 0x7f80b81008d0 2023-07-06T20:18:00.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.535+0000 7f809ffff700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80a00a11e0 0x7f80a00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:00.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.535+0000 7f809ffff700 1 -- 172.21.15.69:0/3382064206 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f80ac014070 con 0x7f80b81008d0 2023-07-06T20:18:00.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.535+0000 7f80b6ffd700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80a00a11e0 0x7f80a00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:00.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.535+0000 7f809ffff700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544] conn(0x7f80a00aad90 0x7f80a00ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:00.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.535+0000 7f809ffff700 1 -- 172.21.15.69:0/3382064206 --> [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80a00ad8f0 con 0x7f80a00aad90 2023-07-06T20:18:00.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.535+0000 7f809ffff700 1 -- 172.21.15.69:0/3382064206 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80ac0cd330 con 0x7f80b81008d0 2023-07-06T20:18:00.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.535+0000 7f80b77fe700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544] conn(0x7f80a00aad90 0x7f80a00ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:00.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.536+0000 7f80b6ffd700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80a00a11e0 0x7f80a00a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009990 tx=0x7f80a801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:00.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.536+0000 7f80b77fe700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544] conn(0x7f80a00aad90 0x7f80a00ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:00.598 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.537+0000 7f809ffff700 1 -- 172.21.15.69:0/3382064206 <== osd.6 v2:172.21.15.69:6818/2001926544 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f80a00ad8f0 con 0x7f80a00aad90 2023-07-06T20:18:00.598 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.553+0000 7f80bd55d700 1 -- 172.21.15.69:0/3382064206 --> [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f80b804e340 con 0x7f80a00aad90 2023-07-06T20:18:00.598 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.553+0000 7f809ffff700 1 -- 172.21.15.69:0/3382064206 <== osd.6 v2:172.21.15.69:6818/2001926544 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f80b804e340 con 0x7f80a00aad90 2023-07-06T20:18:00.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.554+0000 7f80bd55d700 1 -- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544] conn(0x7f80a00aad90 msgr2=0x7f80a00ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:00.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.554+0000 7f80bd55d700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544] conn(0x7f80a00aad90 0x7f80a00ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:00.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.554+0000 7f80bd55d700 1 -- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80a00a11e0 msgr2=0x7f80a00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:00.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.554+0000 7f80bd55d700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f80a00a11e0 0x7f80a00a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009990 tx=0x7f80a801a040 comp rx=0 tx=0).stop 2023-07-06T20:18:00.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.554+0000 7f80bd55d700 1 -- 172.21.15.69:0/3382064206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b81008d0 msgr2=0x7f80b81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:00.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.554+0000 7f80bd55d700 1 --2- 172.21.15.69:0/3382064206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b81008d0 0x7f80b81985f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00e970 tx=0x7f80ac00ec80 comp rx=0 tx=0).stop 2023-07-06T20:18:00.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.554+0000 7f80b77fe700 1 -- 172.21.15.69:0/3382064206 reap_dead start 2023-07-06T20:18:00.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.554+0000 7f80bd55d700 1 -- 172.21.15.69:0/3382064206 shutdown_connections 2023-07-06T20:18:00.600 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.554+0000 7f80bd55d700 1 -- 172.21.15.69:0/3382064206 >> 172.21.15.69:0/3382064206 conn(0x7f80b80fb790 msgr2=0x7f80b80fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:00.600 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.554+0000 7f80bd55d700 1 -- 172.21.15.69:0/3382064206 shutdown_connections 2023-07-06T20:18:00.600 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:00.554+0000 7f80bd55d700 1 -- 172.21.15.69:0/3382064206 wait complete. 2023-07-06T20:18:00.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:00 smithi069 ceph-mon[126399]: pgmap v286: 97 pgs: 1 activating+degraded, 96 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 1/192 objects degraded (0.521%); 0 B/s, 0 objects/s recovering 2023-07-06T20:18:00.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:00 smithi069 ceph-mon[126399]: Health check failed: Degraded data redundancy: 1/192 objects degraded (0.521%), 1 pg degraded (PG_DEGRADED) 2023-07-06T20:18:01.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:00 smithi156 ceph-mon[129280]: pgmap v286: 97 pgs: 1 activating+degraded, 96 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 1/192 objects degraded (0.521%); 0 B/s, 0 objects/s recovering 2023-07-06T20:18:01.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:00 smithi156 ceph-mon[129280]: Health check failed: Degraded data redundancy: 1/192 objects degraded (0.521%), 1 pg degraded (PG_DEGRADED) 2023-07-06T20:18:01.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:00 smithi098 ceph-mon[128968]: pgmap v286: 97 pgs: 1 activating+degraded, 96 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 1/192 objects degraded (0.521%); 0 B/s, 0 objects/s recovering 2023-07-06T20:18:01.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:00 smithi098 ceph-mon[128968]: Health check failed: Degraded data redundancy: 1/192 objects degraded (0.521%), 1 pg degraded (PG_DEGRADED) 2023-07-06T20:18:01.321 INFO:teuthology.orchestra.run.smithi069.stdout:335007449107 2023-07-06T20:18:01.322 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-06T20:18:01.587 INFO:teuthology.orchestra.run.smithi069.stdout:283467841561 2023-07-06T20:18:01.588 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-06T20:18:01.884 INFO:teuthology.orchestra.run.smithi069.stdout:197568495657 2023-07-06T20:18:01.885 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-06T20:18:02.041 INFO:teuthology.orchestra.run.smithi069.stdout:304942678038 2023-07-06T20:18:02.042 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-06T20:18:02.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:01 smithi069 ceph-mon[126399]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/192 objects degraded (0.521%), 1 pg degraded) 2023-07-06T20:18:02.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:01 smithi069 ceph-mon[126399]: Cluster is now healthy 2023-07-06T20:18:02.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:01 smithi156 ceph-mon[129280]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/192 objects degraded (0.521%), 1 pg degraded) 2023-07-06T20:18:02.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:01 smithi156 ceph-mon[129280]: Cluster is now healthy 2023-07-06T20:18:02.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:01 smithi098 ceph-mon[128968]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/192 objects degraded (0.521%), 1 pg degraded) 2023-07-06T20:18:02.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:01 smithi098 ceph-mon[128968]: Cluster is now healthy 2023-07-06T20:18:02.390 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.387+0000 7f0c04d3b700 1 -- 172.21.15.69:0/3431722370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c000fff30 msgr2=0x7f0c00100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.390 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.387+0000 7f0c04d3b700 1 --2- 172.21.15.69:0/3431722370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c000fff30 0x7f0c00100300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0bf00099c0 tx=0x7f0bf0009cd0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.390 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.388+0000 7f0c04d3b700 1 -- 172.21.15.69:0/3431722370 shutdown_connections 2023-07-06T20:18:02.390 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.388+0000 7f0c04d3b700 1 --2- 172.21.15.69:0/3431722370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c00104fd0 0x7f0c0010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.388+0000 7f0c04d3b700 1 --2- 172.21.15.69:0/3431722370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c000fff30 0x7f0c00100300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.388+0000 7f0c04d3b700 1 --2- 172.21.15.69:0/3431722370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c001008d0 0x7f0c001048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.388+0000 7f0c04d3b700 1 -- 172.21.15.69:0/3431722370 >> 172.21.15.69:0/3431722370 conn(0x7f0c000fb790 msgr2=0x7f0c000fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:02.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.388+0000 7f0c04d3b700 1 -- 172.21.15.69:0/3431722370 shutdown_connections 2023-07-06T20:18:02.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.388+0000 7f0c04d3b700 1 -- 172.21.15.69:0/3431722370 wait complete. 2023-07-06T20:18:02.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.389+0000 7f0c04d3b700 1 Processor -- start 2023-07-06T20:18:02.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.389+0000 7f0c04d3b700 1 -- start start 2023-07-06T20:18:02.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.390+0000 7f0c04d3b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c000fff30 0x7f0c001980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.390+0000 7f0c04d3b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c001008d0 0x7f0c001985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.390+0000 7f0c04d3b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c00104fd0 0x7f0c0019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.390+0000 7f0c04d3b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0019cfc0 con 0x7f0c00104fd0 2023-07-06T20:18:02.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.390+0000 7f0c04d3b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0019d130 con 0x7f0c000fff30 2023-07-06T20:18:02.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.390+0000 7f0c04d3b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0019d2f0 con 0x7f0c001008d0 2023-07-06T20:18:02.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.390+0000 7f0bfe59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c000fff30 0x7f0c001980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.390+0000 7f0bfe59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c000fff30 0x7f0c001980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:58248/0 (socket says 172.21.15.69:58248) 2023-07-06T20:18:02.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.390+0000 7f0bfe59c700 1 -- 172.21.15.69:0/49199820 learned_addr learned my addr 172.21.15.69:0/49199820 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:02.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.390+0000 7f0bfed9d700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c00104fd0 0x7f0c0019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.390+0000 7f0bfdd9b700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c001008d0 0x7f0c001985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.391+0000 7f0bfe59c700 1 -- 172.21.15.69:0/49199820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c001008d0 msgr2=0x7f0c001985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.391+0000 7f0bfe59c700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c001008d0 0x7f0c001985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.391+0000 7f0bfe59c700 1 -- 172.21.15.69:0/49199820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c00104fd0 msgr2=0x7f0c0019c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.391+0000 7f0bfe59c700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c00104fd0 0x7f0c0019c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.391+0000 7f0bfe59c700 1 -- 172.21.15.69:0/49199820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf00095f0 con 0x7f0c000fff30 2023-07-06T20:18:02.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.391+0000 7f0bfdd9b700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c001008d0 0x7f0c001985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:02.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.391+0000 7f0bfe59c700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c000fff30 0x7f0c001980a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b580 tx=0x7f0bf000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:02.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.391+0000 7f0bef7fe700 1 -- 172.21.15.69:0/49199820 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf001d070 con 0x7f0c000fff30 2023-07-06T20:18:02.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.391+0000 7f0c04d3b700 1 -- 172.21.15.69:0/49199820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0019d570 con 0x7f0c000fff30 2023-07-06T20:18:02.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.391+0000 7f0bef7fe700 1 -- 172.21.15.69:0/49199820 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bf0005050 con 0x7f0c000fff30 2023-07-06T20:18:02.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.391+0000 7f0bef7fe700 1 -- 172.21.15.69:0/49199820 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0004a40 con 0x7f0c000fff30 2023-07-06T20:18:02.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.391+0000 7f0c04d3b700 1 -- 172.21.15.69:0/49199820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c001a4e30 con 0x7f0c000fff30 2023-07-06T20:18:02.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.392+0000 7f0c04d3b700 1 -- 172.21.15.69:0/49199820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c0004e340 con 0x7f0c000fff30 2023-07-06T20:18:02.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.394+0000 7f0bef7fe700 1 -- 172.21.15.69:0/49199820 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0bf00051c0 con 0x7f0c000fff30 2023-07-06T20:18:02.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.396+0000 7f0bef7fe700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0be80a11e0 0x7f0be80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.396+0000 7f0bef7fe700 1 -- 172.21.15.69:0/49199820 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f0bf00ce060 con 0x7f0c000fff30 2023-07-06T20:18:02.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.396+0000 7f0bfdd9b700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0be80a11e0 0x7f0be80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.396+0000 7f0bef7fe700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634] conn(0x7f0be80aad90 0x7f0be80ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.396+0000 7f0bef7fe700 1 -- 172.21.15.69:0/49199820 --> [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0be80ad8f0 con 0x7f0be80aad90 2023-07-06T20:18:02.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.396+0000 7f0bef7fe700 1 -- 172.21.15.69:0/49199820 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf00ce470 con 0x7f0c000fff30 2023-07-06T20:18:02.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.396+0000 7f0bfdd9b700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0be80a11e0 0x7f0be80a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0c001996c0 tx=0x7f0bf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:02.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.397+0000 7f0bfed9d700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634] conn(0x7f0be80aad90 0x7f0be80ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.397+0000 7f0bfed9d700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634] conn(0x7f0be80aad90 0x7f0be80ad220 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:02.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.398+0000 7f0bef7fe700 1 -- 172.21.15.69:0/49199820 <== osd.5 v2:172.21.15.156:6808/428236634 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0be80ad8f0 con 0x7f0be80aad90 2023-07-06T20:18:02.416 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.414+0000 7f0c04d3b700 1 -- 172.21.15.69:0/49199820 --> [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c0010fee0 con 0x7f0be80aad90 2023-07-06T20:18:02.417 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.415+0000 7f0bef7fe700 1 -- 172.21.15.69:0/49199820 <== osd.5 v2:172.21.15.156:6808/428236634 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0bf800f040 con 0x7f0be80aad90 2023-07-06T20:18:02.417 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.415+0000 7f0c04d3b700 1 -- 172.21.15.69:0/49199820 >> [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634] conn(0x7f0be80aad90 msgr2=0x7f0be80ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.417 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.415+0000 7f0c04d3b700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634] conn(0x7f0be80aad90 0x7f0be80ad220 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.415+0000 7f0c04d3b700 1 -- 172.21.15.69:0/49199820 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0be80a11e0 msgr2=0x7f0be80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.415+0000 7f0c04d3b700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0be80a11e0 0x7f0be80a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0c001996c0 tx=0x7f0bf4009260 comp rx=0 tx=0).stop 2023-07-06T20:18:02.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.415+0000 7f0c04d3b700 1 -- 172.21.15.69:0/49199820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c000fff30 msgr2=0x7f0c001980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.415+0000 7f0c04d3b700 1 --2- 172.21.15.69:0/49199820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c000fff30 0x7f0c001980a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b580 tx=0x7f0bf000bfd0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.415+0000 7f0bfed9d700 1 -- 172.21.15.69:0/49199820 reap_dead start 2023-07-06T20:18:02.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.415+0000 7f0c04d3b700 1 -- 172.21.15.69:0/49199820 shutdown_connections 2023-07-06T20:18:02.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.415+0000 7f0c04d3b700 1 -- 172.21.15.69:0/49199820 >> 172.21.15.69:0/49199820 conn(0x7f0c000fb790 msgr2=0x7f0c000fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:02.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.415+0000 7f0c04d3b700 1 -- 172.21.15.69:0/49199820 shutdown_connections 2023-07-06T20:18:02.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.415+0000 7f0c04d3b700 1 -- 172.21.15.69:0/49199820 wait complete. 2023-07-06T20:18:02.421 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.295+0000 7f123b59e700 1 -- 172.21.15.69:0/3403540248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0729e0 msgr2=0x7f123c072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.421 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.295+0000 7f123b59e700 1 --2- 172.21.15.69:0/3403540248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0729e0 0x7f123c072db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f122c0098f0 tx=0x7f122c009c00 comp rx=0 tx=0).stop 2023-07-06T20:18:02.421 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.295+0000 7f123b59e700 1 -- 172.21.15.69:0/3403540248 shutdown_connections 2023-07-06T20:18:02.421 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.295+0000 7f123b59e700 1 --2- 172.21.15.69:0/3403540248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f123c10c350 0x7f123c10e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.421 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.295+0000 7f123b59e700 1 --2- 172.21.15.69:0/3403540248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0729e0 0x7f123c072db0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.421 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.295+0000 7f123b59e700 1 --2- 172.21.15.69:0/3403540248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123c0732f0 0x7f123c10be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.295+0000 7f123b59e700 1 -- 172.21.15.69:0/3403540248 >> 172.21.15.69:0/3403540248 conn(0x7f123c0fb690 msgr2=0x7f123c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:02.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.296+0000 7f123b59e700 1 -- 172.21.15.69:0/3403540248 shutdown_connections 2023-07-06T20:18:02.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.296+0000 7f123b59e700 1 -- 172.21.15.69:0/3403540248 wait complete. 2023-07-06T20:18:02.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.297+0000 7f123b59e700 1 Processor -- start 2023-07-06T20:18:02.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.297+0000 7f123b59e700 1 -- start start 2023-07-06T20:18:02.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.297+0000 7f123b59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f123c0729e0 0x7f123c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.297+0000 7f123b59e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0732f0 0x7f123c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.297+0000 7f123b59e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123c10c350 0x7f123c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.297+0000 7f123b59e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c19ce20 con 0x7f123c10c350 2023-07-06T20:18:02.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.297+0000 7f123b59e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c19cf90 con 0x7f123c0732f0 2023-07-06T20:18:02.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.297+0000 7f123b59e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c19d150 con 0x7f123c0729e0 2023-07-06T20:18:02.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.298+0000 7f123ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123c10c350 0x7f123c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.298+0000 7f123ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123c10c350 0x7f123c19c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42212/0 (socket says 172.21.15.69:42212) 2023-07-06T20:18:02.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.298+0000 7f123ad9d700 1 -- 172.21.15.69:0/295816043 learned_addr learned my addr 172.21.15.69:0/295816043 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:02.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.298+0000 7f123a59c700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f123c0729e0 0x7f123c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.298+0000 7f1239d9b700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0732f0 0x7f123c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.298+0000 7f123ad9d700 1 -- 172.21.15.69:0/295816043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f123c0729e0 msgr2=0x7f123c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.298+0000 7f123ad9d700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f123c0729e0 0x7f123c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.298+0000 7f123ad9d700 1 -- 172.21.15.69:0/295816043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0732f0 msgr2=0x7f123c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.298+0000 7f123ad9d700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0732f0 0x7f123c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.298+0000 7f123ad9d700 1 -- 172.21.15.69:0/295816043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12300095f0 con 0x7f123c10c350 2023-07-06T20:18:02.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.298+0000 7f1239d9b700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f123c0732f0 0x7f123c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:02.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.298+0000 7f123ad9d700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123c10c350 0x7f123c19c780 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f123400c370 tx=0x7f123400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:02.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.299+0000 7f122b7fe700 1 -- 172.21.15.69:0/295816043 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234017070 con 0x7f123c10c350 2023-07-06T20:18:02.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.299+0000 7f122b7fe700 1 -- 172.21.15.69:0/295816043 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f123400f040 con 0x7f123c10c350 2023-07-06T20:18:02.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.299+0000 7f122b7fe700 1 -- 172.21.15.69:0/295816043 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123400e050 con 0x7f123c10c350 2023-07-06T20:18:02.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.299+0000 7f123b59e700 1 -- 172.21.15.69:0/295816043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1234009c40 con 0x7f123c10c350 2023-07-06T20:18:02.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.299+0000 7f123b59e700 1 -- 172.21.15.69:0/295816043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c009520 con 0x7f123c10c350 2023-07-06T20:18:02.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.300+0000 7f123b59e700 1 -- 172.21.15.69:0/295816043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f123c04eb90 con 0x7f123c10c350 2023-07-06T20:18:02.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.302+0000 7f122b7fe700 1 -- 172.21.15.69:0/295816043 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f123400cb60 con 0x7f123c10c350 2023-07-06T20:18:02.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.305+0000 7f122b7fe700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f12240a11e0 0x7f12240a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.305+0000 7f122b7fe700 1 -- 172.21.15.69:0/295816043 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f12340ccf00 con 0x7f123c10c350 2023-07-06T20:18:02.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.305+0000 7f123a59c700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f12240a11e0 0x7f12240a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.306+0000 7f122b7fe700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200] conn(0x7f12240aad90 0x7f12240ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.306+0000 7f122b7fe700 1 -- 172.21.15.69:0/295816043 --> [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12240ad8f0 con 0x7f12240aad90 2023-07-06T20:18:02.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.306+0000 7f122b7fe700 1 -- 172.21.15.69:0/295816043 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12340cd270 con 0x7f123c10c350 2023-07-06T20:18:02.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.306+0000 7f1239d9b700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200] conn(0x7f12240aad90 0x7f12240ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.306+0000 7f123a59c700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f12240a11e0 0x7f12240a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f122c009fd0 tx=0x7f122c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:02.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.307+0000 7f1239d9b700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200] conn(0x7f12240aad90 0x7f12240ad220 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:02.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.308+0000 7f122b7fe700 1 -- 172.21.15.69:0/295816043 <== osd.3 v2:172.21.15.69:6810/3049408200 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f12240ad8f0 con 0x7f12240aad90 2023-07-06T20:18:02.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.325+0000 7f123b59e700 1 -- 172.21.15.69:0/295816043 --> [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f123c111ea0 con 0x7f12240aad90 2023-07-06T20:18:02.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.325+0000 7f122b7fe700 1 -- 172.21.15.69:0/295816043 <== osd.3 v2:172.21.15.69:6810/3049408200 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1230009e80 con 0x7f12240aad90 2023-07-06T20:18:02.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.326+0000 7f123b59e700 1 -- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200] conn(0x7f12240aad90 msgr2=0x7f12240ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.326+0000 7f123b59e700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200] conn(0x7f12240aad90 0x7f12240ad220 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.326+0000 7f123b59e700 1 -- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f12240a11e0 msgr2=0x7f12240a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.326+0000 7f123b59e700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f12240a11e0 0x7f12240a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f122c009fd0 tx=0x7f122c019040 comp rx=0 tx=0).stop 2023-07-06T20:18:02.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.326+0000 7f123b59e700 1 -- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123c10c350 msgr2=0x7f123c19c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.326+0000 7f123b59e700 1 --2- 172.21.15.69:0/295816043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123c10c350 0x7f123c19c780 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f123400c370 tx=0x7f123400c680 comp rx=0 tx=0).stop 2023-07-06T20:18:02.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.326+0000 7f123ad9d700 1 -- 172.21.15.69:0/295816043 reap_dead start 2023-07-06T20:18:02.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.326+0000 7f123b59e700 1 -- 172.21.15.69:0/295816043 shutdown_connections 2023-07-06T20:18:02.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.326+0000 7f123b59e700 1 -- 172.21.15.69:0/295816043 >> 172.21.15.69:0/295816043 conn(0x7f123c0fb690 msgr2=0x7f123c10e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:02.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.326+0000 7f123b59e700 1 -- 172.21.15.69:0/295816043 shutdown_connections 2023-07-06T20:18:02.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.326+0000 7f123b59e700 1 -- 172.21.15.69:0/295816043 wait complete. 2023-07-06T20:18:02.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.678+0000 7f92f8aa5700 1 -- 172.21.15.69:0/2716467208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92f40fec80 msgr2=0x7f92f40ff050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.678+0000 7f92f8aa5700 1 --2- 172.21.15.69:0/2716467208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92f40fec80 0x7f92f40ff050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009990 tx=0x7f92e4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.678+0000 7f92f8aa5700 1 -- 172.21.15.69:0/2716467208 shutdown_connections 2023-07-06T20:18:02.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.678+0000 7f92f8aa5700 1 --2- 172.21.15.69:0/2716467208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92f4104090 0x7f92f4104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.678+0000 7f92f8aa5700 1 --2- 172.21.15.69:0/2716467208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92f40fec80 0x7f92f40ff050 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.692 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.678+0000 7f92f8aa5700 1 --2- 172.21.15.69:0/2716467208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92f40ff590 0x7f92f4103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.692 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.678+0000 7f92f8aa5700 1 -- 172.21.15.69:0/2716467208 >> 172.21.15.69:0/2716467208 conn(0x7f92f40fa6d0 msgr2=0x7f92f40fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:02.692 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.678+0000 7f92f8aa5700 1 -- 172.21.15.69:0/2716467208 shutdown_connections 2023-07-06T20:18:02.692 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.679+0000 7f92f8aa5700 1 -- 172.21.15.69:0/2716467208 wait complete. 2023-07-06T20:18:02.692 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.679+0000 7f92f8aa5700 1 Processor -- start 2023-07-06T20:18:02.692 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.680+0000 7f92f8aa5700 1 -- start start 2023-07-06T20:18:02.692 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.680+0000 7f92f8aa5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92f40fec80 0x7f92f419c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.680+0000 7f92f8aa5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92f40ff590 0x7f92f419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.680+0000 7f92f8aa5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92f4104090 0x7f92f41a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.680+0000 7f92f8aa5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f41a1360 con 0x7f92f4104090 2023-07-06T20:18:02.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.680+0000 7f92f8aa5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f41a14d0 con 0x7f92f40fec80 2023-07-06T20:18:02.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.680+0000 7f92f8aa5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f41a1690 con 0x7f92f40ff590 2023-07-06T20:18:02.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.680+0000 7f92f259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92f40fec80 0x7f92f419c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.680+0000 7f92f259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92f40fec80 0x7f92f419c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:58268/0 (socket says 172.21.15.69:58268) 2023-07-06T20:18:02.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.680+0000 7f92f1d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92f40ff590 0x7f92f419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.680+0000 7f92f259c700 1 -- 172.21.15.69:0/1382646804 learned_addr learned my addr 172.21.15.69:0/1382646804 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:02.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.680+0000 7f92f2d9d700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92f4104090 0x7f92f41a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.681+0000 7f92f259c700 1 -- 172.21.15.69:0/1382646804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92f40ff590 msgr2=0x7f92f419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.681+0000 7f92f259c700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92f40ff590 0x7f92f419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.681+0000 7f92f259c700 1 -- 172.21.15.69:0/1382646804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92f4104090 msgr2=0x7f92f41a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.681+0000 7f92f259c700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92f4104090 0x7f92f41a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.681+0000 7f92f259c700 1 -- 172.21.15.69:0/1382646804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e40095f0 con 0x7f92f40fec80 2023-07-06T20:18:02.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.681+0000 7f92f1d9b700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92f40ff590 0x7f92f419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:02.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.681+0000 7f92f259c700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92f40fec80 0x7f92f419c3a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92e4005910 tx=0x7f92e4005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:02.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.681+0000 7f92e37fe700 1 -- 172.21.15.69:0/1382646804 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e401d070 con 0x7f92f40fec80 2023-07-06T20:18:02.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.681+0000 7f92e37fe700 1 -- 172.21.15.69:0/1382646804 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92e4022470 con 0x7f92f40fec80 2023-07-06T20:18:02.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.681+0000 7f92f8aa5700 1 -- 172.21.15.69:0/1382646804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f41a1910 con 0x7f92f40fec80 2023-07-06T20:18:02.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.681+0000 7f92e37fe700 1 -- 172.21.15.69:0/1382646804 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4017650 con 0x7f92f40fec80 2023-07-06T20:18:02.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.682+0000 7f92f8aa5700 1 -- 172.21.15.69:0/1382646804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f41a91d0 con 0x7f92f40fec80 2023-07-06T20:18:02.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.682+0000 7f92f8aa5700 1 -- 172.21.15.69:0/1382646804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f92f40682d0 con 0x7f92f40fec80 2023-07-06T20:18:02.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.685+0000 7f92e37fe700 1 -- 172.21.15.69:0/1382646804 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f92e40225e0 con 0x7f92f40fec80 2023-07-06T20:18:02.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.687+0000 7f92e37fe700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f92dc0a11e0 0x7f92dc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.687+0000 7f92e37fe700 1 -- 172.21.15.69:0/1382646804 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f92e40ce8b0 con 0x7f92f40fec80 2023-07-06T20:18:02.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.687+0000 7f92f1d9b700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f92dc0a11e0 0x7f92dc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.687+0000 7f92e37fe700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125] conn(0x7f92dc0aad90 0x7f92dc0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:02.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.687+0000 7f92e37fe700 1 -- 172.21.15.69:0/1382646804 --> [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92dc0ad8f0 con 0x7f92dc0aad90 2023-07-06T20:18:02.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.687+0000 7f92e37fe700 1 -- 172.21.15.69:0/1382646804 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92e40cec20 con 0x7f92f40fec80 2023-07-06T20:18:02.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.687+0000 7f92f2d9d700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125] conn(0x7f92dc0aad90 0x7f92dc0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:02.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.687+0000 7f92f1d9b700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f92dc0a11e0 0x7f92dc0a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f92f419d9c0 tx=0x7f92e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:02.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.688+0000 7f92f2d9d700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125] conn(0x7f92dc0aad90 0x7f92dc0ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:02.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.689+0000 7f92e37fe700 1 -- 172.21.15.69:0/1382646804 <== osd.1 v2:172.21.15.98:6800/3741678125 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f92dc0ad8f0 con 0x7f92dc0aad90 2023-07-06T20:18:02.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.712+0000 7f92f8aa5700 1 -- 172.21.15.69:0/1382646804 --> [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f92f4116490 con 0x7f92dc0aad90 2023-07-06T20:18:02.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.712+0000 7f92e37fe700 1 -- 172.21.15.69:0/1382646804 <== osd.1 v2:172.21.15.98:6800/3741678125 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f92ec00f040 con 0x7f92dc0aad90 2023-07-06T20:18:02.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.713+0000 7f92f8aa5700 1 -- 172.21.15.69:0/1382646804 >> [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125] conn(0x7f92dc0aad90 msgr2=0x7f92dc0ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.713+0000 7f92f8aa5700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125] conn(0x7f92dc0aad90 0x7f92dc0ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:02.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.713+0000 7f92f8aa5700 1 -- 172.21.15.69:0/1382646804 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f92dc0a11e0 msgr2=0x7f92dc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.713+0000 7f92f8aa5700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f92dc0a11e0 0x7f92dc0a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f92f419d9c0 tx=0x7f92e8009260 comp rx=0 tx=0).stop 2023-07-06T20:18:02.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.713+0000 7f92f8aa5700 1 -- 172.21.15.69:0/1382646804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92f40fec80 msgr2=0x7f92f419c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:02.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.713+0000 7f92f8aa5700 1 --2- 172.21.15.69:0/1382646804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92f40fec80 0x7f92f419c3a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92e4005910 tx=0x7f92e4005720 comp rx=0 tx=0).stop 2023-07-06T20:18:02.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.713+0000 7f92f2d9d700 1 -- 172.21.15.69:0/1382646804 reap_dead start 2023-07-06T20:18:02.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.713+0000 7f92f8aa5700 1 -- 172.21.15.69:0/1382646804 shutdown_connections 2023-07-06T20:18:02.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.713+0000 7f92f8aa5700 1 -- 172.21.15.69:0/1382646804 >> 172.21.15.69:0/1382646804 conn(0x7f92f40fa6d0 msgr2=0x7f92f4101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:02.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.713+0000 7f92f8aa5700 1 -- 172.21.15.69:0/1382646804 shutdown_connections 2023-07-06T20:18:02.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:02.713+0000 7f92f8aa5700 1 -- 172.21.15.69:0/1382646804 wait complete. 2023-07-06T20:18:02.991 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:02 smithi069 ceph-mon[126399]: pgmap v287: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 639 B/s wr, 6 op/s; 13 B/s, 1 objects/s recovering 2023-07-06T20:18:03.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:02 smithi156 ceph-mon[129280]: pgmap v287: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 639 B/s wr, 6 op/s; 13 B/s, 1 objects/s recovering 2023-07-06T20:18:03.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:02 smithi098 ceph-mon[128968]: pgmap v287: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 639 B/s wr, 6 op/s; 13 B/s, 1 objects/s recovering 2023-07-06T20:18:03.593 INFO:teuthology.orchestra.run.smithi069.stdout:55834574912 2023-07-06T20:18:03.594 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-06T20:18:03.653 INFO:teuthology.orchestra.run.smithi069.stdout:163208757294 2023-07-06T20:18:03.653 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-06T20:18:03.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.751+0000 7feba2192700 1 -- 172.21.15.69:0/3145246510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb9c1008d0 msgr2=0x7feb9c1048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:03.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.751+0000 7feba2192700 1 --2- 172.21.15.69:0/3145246510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb9c1008d0 0x7feb9c1048a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb900099c0 tx=0x7feb90009cd0 comp rx=0 tx=0).stop 2023-07-06T20:18:03.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.751+0000 7feba2192700 1 -- 172.21.15.69:0/3145246510 shutdown_connections 2023-07-06T20:18:03.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.751+0000 7feba2192700 1 --2- 172.21.15.69:0/3145246510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb9c104fd0 0x7feb9c10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:03.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.751+0000 7feba2192700 1 --2- 172.21.15.69:0/3145246510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb9c0fff30 0x7feb9c100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:03.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.751+0000 7feba2192700 1 --2- 172.21.15.69:0/3145246510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb9c1008d0 0x7feb9c1048a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:03.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.751+0000 7feba2192700 1 -- 172.21.15.69:0/3145246510 >> 172.21.15.69:0/3145246510 conn(0x7feb9c0fb790 msgr2=0x7feb9c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:03.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.751+0000 7feba2192700 1 -- 172.21.15.69:0/3145246510 shutdown_connections 2023-07-06T20:18:03.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.751+0000 7feba2192700 1 -- 172.21.15.69:0/3145246510 wait complete. 2023-07-06T20:18:03.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.752+0000 7feba2192700 1 Processor -- start 2023-07-06T20:18:03.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.752+0000 7feba2192700 1 -- start start 2023-07-06T20:18:03.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feba2192700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb9c0fff30 0x7feb9c195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:03.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feba2192700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb9c1008d0 0x7feb9c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:03.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feba2192700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb9c104fd0 0x7feb9c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:03.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feba2192700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c19ada0 con 0x7feb9c104fd0 2023-07-06T20:18:03.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feba2192700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c19af10 con 0x7feb9c1008d0 2023-07-06T20:18:03.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feba2192700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c19b0d0 con 0x7feb9c0fff30 2023-07-06T20:18:03.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feb9b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb9c0fff30 0x7feb9c195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:03.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feb9bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb9c104fd0 0x7feb9c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:03.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feb9affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb9c1008d0 0x7feb9c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:03.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feb9affd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb9c1008d0 0x7feb9c196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:58860/0 (socket says 172.21.15.69:58860) 2023-07-06T20:18:03.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feb9b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb9c0fff30 0x7feb9c195de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:40722/0 (socket says 172.21.15.69:40722) 2023-07-06T20:18:03.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feb9affd700 1 -- 172.21.15.69:0/81007092 learned_addr learned my addr 172.21.15.69:0/81007092 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:03.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.753+0000 7feb9bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb9c104fd0 0x7feb9c19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43776/0 (socket says 172.21.15.69:43776) 2023-07-06T20:18:03.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.754+0000 7feb9bfff700 1 -- 172.21.15.69:0/81007092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb9c0fff30 msgr2=0x7feb9c195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:03.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.754+0000 7feb9bfff700 1 --2- 172.21.15.69:0/81007092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb9c0fff30 0x7feb9c195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:03.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.754+0000 7feb9bfff700 1 -- 172.21.15.69:0/81007092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb9c1008d0 msgr2=0x7feb9c196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:03.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.754+0000 7feb9bfff700 1 --2- 172.21.15.69:0/81007092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb9c1008d0 0x7feb9c196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:03.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.754+0000 7feb9bfff700 1 -- 172.21.15.69:0/81007092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb900095f0 con 0x7feb9c104fd0 2023-07-06T20:18:03.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.754+0000 7feb9bfff700 1 --2- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb9c104fd0 0x7feb9c19a790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb9400b330 tx=0x7feb9400b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:03.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.754+0000 7feb98ff9700 1 -- 172.21.15.69:0/81007092 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94015070 con 0x7feb9c104fd0 2023-07-06T20:18:03.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.754+0000 7feba2192700 1 -- 172.21.15.69:0/81007092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9c19b350 con 0x7feb9c104fd0 2023-07-06T20:18:03.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.754+0000 7feb98ff9700 1 -- 172.21.15.69:0/81007092 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb9400bdd0 con 0x7feb9c104fd0 2023-07-06T20:18:03.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.754+0000 7feb98ff9700 1 -- 172.21.15.69:0/81007092 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94007c20 con 0x7feb9c104fd0 2023-07-06T20:18:03.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.755+0000 7feba2192700 1 -- 172.21.15.69:0/81007092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9c072a40 con 0x7feb9c104fd0 2023-07-06T20:18:03.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.756+0000 7feba2192700 1 -- 172.21.15.69:0/81007092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feb9c04e340 con 0x7feb9c104fd0 2023-07-06T20:18:03.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.758+0000 7feb98ff9700 1 -- 172.21.15.69:0/81007092 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7feb94007500 con 0x7feb9c104fd0 2023-07-06T20:18:03.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.759+0000 7feb98ff9700 1 --2- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feb840a11e0 0x7feb840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:03.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.760+0000 7feb98ff9700 1 -- 172.21.15.69:0/81007092 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7feb940d0cc0 con 0x7feb9c104fd0 2023-07-06T20:18:03.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.760+0000 7feb9b7fe700 1 --2- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feb840a11e0 0x7feb840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:03.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.760+0000 7feb98ff9700 1 --2- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786] conn(0x7feb840aad90 0x7feb840ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:03.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.760+0000 7feb98ff9700 1 -- 172.21.15.69:0/81007092 --> [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb840ad8f0 con 0x7feb840aad90 2023-07-06T20:18:03.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.760+0000 7feb98ff9700 1 -- 172.21.15.69:0/81007092 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb940d1030 con 0x7feb9c104fd0 2023-07-06T20:18:03.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.760+0000 7feb9affd700 1 --2- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786] conn(0x7feb840aad90 0x7feb840ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:03.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.761+0000 7feb9b7fe700 1 --2- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feb840a11e0 0x7feb840a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009c60 tx=0x7feb8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:03.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.761+0000 7feb9affd700 1 --2- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786] conn(0x7feb840aad90 0x7feb840ad220 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:03.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.762+0000 7feb98ff9700 1 -- 172.21.15.69:0/81007092 <== osd.0 v2:172.21.15.69:6802/2593880786 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7feb840ad8f0 con 0x7feb840aad90 2023-07-06T20:18:03.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.779+0000 7feba2192700 1 -- 172.21.15.69:0/81007092 --> [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feb9c10fe80 con 0x7feb840aad90 2023-07-06T20:18:03.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.779+0000 7feb98ff9700 1 -- 172.21.15.69:0/81007092 <== osd.0 v2:172.21.15.69:6802/2593880786 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7feb9c10fe80 con 0x7feb840aad90 2023-07-06T20:18:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.780+0000 7feba2192700 1 -- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786] conn(0x7feb840aad90 msgr2=0x7feb840ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.780+0000 7feba2192700 1 --2- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786] conn(0x7feb840aad90 0x7feb840ad220 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.780+0000 7feba2192700 1 -- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feb840a11e0 msgr2=0x7feb840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.780+0000 7feba2192700 1 --2- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feb840a11e0 0x7feb840a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009c60 tx=0x7feb8c009260 comp rx=0 tx=0).stop 2023-07-06T20:18:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.780+0000 7feba2192700 1 -- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb9c104fd0 msgr2=0x7feb9c19a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.780+0000 7feba2192700 1 --2- 172.21.15.69:0/81007092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb9c104fd0 0x7feb9c19a790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb9400b330 tx=0x7feb9400b640 comp rx=0 tx=0).stop 2023-07-06T20:18:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.780+0000 7feb9bfff700 1 -- 172.21.15.69:0/81007092 reap_dead start 2023-07-06T20:18:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.780+0000 7feba2192700 1 -- 172.21.15.69:0/81007092 shutdown_connections 2023-07-06T20:18:03.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.780+0000 7feba2192700 1 -- 172.21.15.69:0/81007092 >> 172.21.15.69:0/81007092 conn(0x7feb9c0fb790 msgr2=0x7feb9c0fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:03.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.780+0000 7feba2192700 1 -- 172.21.15.69:0/81007092 shutdown_connections 2023-07-06T20:18:03.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.780+0000 7feba2192700 1 -- 172.21.15.69:0/81007092 wait complete. 2023-07-06T20:18:04.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.703+0000 7fc870261700 1 -- 172.21.15.69:0/981454847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868100f20 msgr2=0x7fc8681012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:04.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.703+0000 7fc870261700 1 --2- 172.21.15.69:0/981454847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868100f20 0x7fc8681012f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc864009990 tx=0x7fc864009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:04.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 -- 172.21.15.69:0/981454847 shutdown_connections 2023-07-06T20:18:04.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 --2- 172.21.15.69:0/981454847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc86810a270 0x7fc86810c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:04.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 --2- 172.21.15.69:0/981454847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868100f20 0x7fc8681012f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:04.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 --2- 172.21.15.69:0/981454847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc868101830 0x7fc868109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:04.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 -- 172.21.15.69:0/981454847 >> 172.21.15.69:0/981454847 conn(0x7fc868075ca0 msgr2=0x7fc8680760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:04.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 -- 172.21.15.69:0/981454847 shutdown_connections 2023-07-06T20:18:04.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 -- 172.21.15.69:0/981454847 wait complete. 2023-07-06T20:18:04.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 Processor -- start 2023-07-06T20:18:04.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 -- start start 2023-07-06T20:18:04.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc868100f20 0x7fc868106050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:04.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868101830 0x7fc868106590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:04.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86810a270 0x7fc868106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:04.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8681a8660 con 0x7fc86810a270 2023-07-06T20:18:04.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc868112f30 con 0x7fc868100f20 2023-07-06T20:18:04.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8681130f0 con 0x7fc868101830 2023-07-06T20:18:04.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc86d7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868101830 0x7fc868106590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:04.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc86d7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868101830 0x7fc868106590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:40706/0 (socket says 172.21.15.69:40706) 2023-07-06T20:18:04.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc86d7fc700 1 -- 172.21.15.69:0/2190718714 learned_addr learned my addr 172.21.15.69:0/2190718714 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:04.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc86dffd700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc868100f20 0x7fc868106050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:04.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc86d7fc700 1 -- 172.21.15.69:0/2190718714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc868100f20 msgr2=0x7fc868106050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:04.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc86d7fc700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc868100f20 0x7fc868106050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:04.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc86d7fc700 1 -- 172.21.15.69:0/2190718714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86810a270 msgr2=0x7fc868106ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:04.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc86d7fc700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86810a270 0x7fc868106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:04.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc86d7fc700 1 -- 172.21.15.69:0/2190718714 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8640095f0 con 0x7fc868101830 2023-07-06T20:18:04.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc86d7fc700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868101830 0x7fc868106590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc85800e9a0 tx=0x7fc85800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:04.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc85effd700 1 -- 172.21.15.69:0/2190718714 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858009790 con 0x7fc868101830 2023-07-06T20:18:04.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 -- 172.21.15.69:0/2190718714 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8681133d0 con 0x7fc868101830 2023-07-06T20:18:04.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc870261700 1 -- 172.21.15.69:0/2190718714 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc868113920 con 0x7fc868101830 2023-07-06T20:18:04.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc85effd700 1 -- 172.21.15.69:0/2190718714 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc85800cbd0 con 0x7fc868101830 2023-07-06T20:18:04.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.705+0000 7fc85effd700 1 -- 172.21.15.69:0/2190718714 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858018680 con 0x7fc868101830 2023-07-06T20:18:04.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.710+0000 7fc870261700 1 -- 172.21.15.69:0/2190718714 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc8680ffdd0 con 0x7fc868101830 2023-07-06T20:18:04.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.712+0000 7fc85effd700 1 -- 172.21.15.69:0/2190718714 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc8580187e0 con 0x7fc868101830 2023-07-06T20:18:04.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.712+0000 7fc85effd700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc8540a12e0 0x7fc8540a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:04.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.712+0000 7fc85effd700 1 -- 172.21.15.69:0/2190718714 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fc858014070 con 0x7fc868101830 2023-07-06T20:18:04.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.717+0000 7fc85effd700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162] conn(0x7fc8540aae90 0x7fc8540ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:04.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.717+0000 7fc85effd700 1 -- 172.21.15.69:0/2190718714 --> [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc8540ad9f0 con 0x7fc8540aae90 2023-07-06T20:18:04.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.717+0000 7fc85effd700 1 -- 172.21.15.69:0/2190718714 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8580cdb00 con 0x7fc868101830 2023-07-06T20:18:04.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.717+0000 7fc86e7fe700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162] conn(0x7fc8540aae90 0x7fc8540ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:04.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.717+0000 7fc86e7fe700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162] conn(0x7fc8540aae90 0x7fc8540ad320 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:04.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.719+0000 7fc85effd700 1 -- 172.21.15.69:0/2190718714 <== osd.4 v2:172.21.15.98:6808/3859428162 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc8540ad9f0 con 0x7fc8540aae90 2023-07-06T20:18:04.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.719+0000 7fc86dffd700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc8540a12e0 0x7fc8540a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:04.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.720+0000 7fc86dffd700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc8540a12e0 0x7fc8540a3790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc864006010 tx=0x7fc86400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:04.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.738+0000 7fc870261700 1 -- 172.21.15.69:0/2190718714 --> [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc86804e340 con 0x7fc8540aae90 2023-07-06T20:18:04.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.738+0000 7fc85effd700 1 -- 172.21.15.69:0/2190718714 <== osd.4 v2:172.21.15.98:6808/3859428162 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc86804e340 con 0x7fc8540aae90 2023-07-06T20:18:04.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.739+0000 7fc870261700 1 -- 172.21.15.69:0/2190718714 >> [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162] conn(0x7fc8540aae90 msgr2=0x7fc8540ad320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:04.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.739+0000 7fc870261700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162] conn(0x7fc8540aae90 0x7fc8540ad320 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:04.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.739+0000 7fc870261700 1 -- 172.21.15.69:0/2190718714 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc8540a12e0 msgr2=0x7fc8540a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:04.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.739+0000 7fc870261700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc8540a12e0 0x7fc8540a3790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc864006010 tx=0x7fc86400b260 comp rx=0 tx=0).stop 2023-07-06T20:18:04.112 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.739+0000 7fc870261700 1 -- 172.21.15.69:0/2190718714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868101830 msgr2=0x7fc868106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:04.112 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.739+0000 7fc870261700 1 --2- 172.21.15.69:0/2190718714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc868101830 0x7fc868106590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc85800e9a0 tx=0x7fc85800ed60 comp rx=0 tx=0).stop 2023-07-06T20:18:04.112 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.739+0000 7fc86e7fe700 1 -- 172.21.15.69:0/2190718714 reap_dead start 2023-07-06T20:18:04.112 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.739+0000 7fc870261700 1 -- 172.21.15.69:0/2190718714 shutdown_connections 2023-07-06T20:18:04.112 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.739+0000 7fc870261700 1 -- 172.21.15.69:0/2190718714 >> 172.21.15.69:0/2190718714 conn(0x7fc868075ca0 msgr2=0x7fc86810eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:04.112 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.739+0000 7fc870261700 1 -- 172.21.15.69:0/2190718714 shutdown_connections 2023-07-06T20:18:04.112 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:03.739+0000 7fc870261700 1 -- 172.21.15.69:0/2190718714 wait complete. 2023-07-06T20:18:04.748 INFO:teuthology.orchestra.run.smithi069.stdout:34359738438 2023-07-06T20:18:04.749 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-06T20:18:04.758 INFO:teuthology.orchestra.run.smithi069.stdout:141733920819 2023-07-06T20:18:04.759 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-06T20:18:04.777 INFO:teuthology.orchestra.run.smithi069.stdout:107374182455 2023-07-06T20:18:04.778 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-06T20:18:04.957 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:05.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:04 smithi069 ceph-mon[126399]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 13 B/s, 1 objects/s recovering 2023-07-06T20:18:05.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:04 smithi156 ceph-mon[129280]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 13 B/s, 1 objects/s recovering 2023-07-06T20:18:05.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:04 smithi098 ceph-mon[128968]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 13 B/s, 1 objects/s recovering 2023-07-06T20:18:05.149 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:05.162 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:05.169 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:05.312 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:05.591 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:05.592 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:05.826 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:06.034 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:07.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:06 smithi069 ceph-mon[126399]: pgmap v289: 97 pgs: 97 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s; 13 B/s, 1 objects/s recovering 2023-07-06T20:18:07.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:06 smithi156 ceph-mon[129280]: pgmap v289: 97 pgs: 97 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s; 13 B/s, 1 objects/s recovering 2023-07-06T20:18:07.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:06 smithi098 ceph-mon[128968]: pgmap v289: 97 pgs: 97 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s; 13 B/s, 1 objects/s recovering 2023-07-06T20:18:07.489 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:07.489 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:09.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:08 smithi156 ceph-mon[129280]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 209 B/s wr, 5 op/s; 11 B/s, 1 objects/s recovering 2023-07-06T20:18:09.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:08 smithi098 ceph-mon[128968]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 209 B/s wr, 5 op/s; 11 B/s, 1 objects/s recovering 2023-07-06T20:18:09.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:08 smithi069 ceph-mon[126399]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 209 B/s wr, 5 op/s; 11 B/s, 1 objects/s recovering 2023-07-06T20:18:11.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.214+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/3245624517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d40ffb30 msgr2=0x7fe8d40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:11.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.214+0000 7fe8d9eb5700 1 --2- 172.21.15.69:0/3245624517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d40ffb30 0x7fe8d40fffa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009990 tx=0x7fe8c0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.215+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/3245624517 shutdown_connections 2023-07-06T20:18:11.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.215+0000 7fe8d9eb5700 1 --2- 172.21.15.69:0/3245624517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8d41004e0 0x7fe8d410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.215+0000 7fe8d9eb5700 1 --2- 172.21.15.69:0/3245624517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8d4105b50 0x7fe8d4105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.215+0000 7fe8d9eb5700 1 --2- 172.21.15.69:0/3245624517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d40ffb30 0x7fe8d40fffa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.215+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/3245624517 >> 172.21.15.69:0/3245624517 conn(0x7fe8d40fb690 msgr2=0x7fe8d40fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:11.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.215+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/3245624517 shutdown_connections 2023-07-06T20:18:11.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.215+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/3245624517 wait complete. 2023-07-06T20:18:11.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.216+0000 7fe8d9eb5700 1 Processor -- start 2023-07-06T20:18:11.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.216+0000 7fe8d9eb5700 1 -- start start 2023-07-06T20:18:11.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.216+0000 7fe8d9eb5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8d40ffb30 0x7fe8d4072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:11.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.216+0000 7fe8d9eb5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8d41004e0 0x7fe8d4072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:11.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.216+0000 7fe8d9eb5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d4105b50 0x7fe8d4073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:11.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.216+0000 7fe8d9eb5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d41a4060 con 0x7fe8d4105b50 2023-07-06T20:18:11.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.216+0000 7fe8d9eb5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d41a41d0 con 0x7fe8d40ffb30 2023-07-06T20:18:11.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.216+0000 7fe8d9eb5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d41a4390 con 0x7fe8d41004e0 2023-07-06T20:18:11.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d96b4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d4105b50 0x7fe8d4073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:11.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d8eb3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8d40ffb30 0x7fe8d4072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:11.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d96b4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d4105b50 0x7fe8d4073400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43780/0 (socket says 172.21.15.69:43780) 2023-07-06T20:18:11.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d96b4700 1 -- 172.21.15.69:0/1246024899 learned_addr learned my addr 172.21.15.69:0/1246024899 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:11.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d3fff700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8d41004e0 0x7fe8d4072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:11.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d96b4700 1 -- 172.21.15.69:0/1246024899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8d41004e0 msgr2=0x7fe8d4072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:11.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d96b4700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8d41004e0 0x7fe8d4072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d96b4700 1 -- 172.21.15.69:0/1246024899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8d40ffb30 msgr2=0x7fe8d4072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:11.024 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d96b4700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8d40ffb30 0x7fe8d4072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.024 INFO:teuthology.orchestra.run.smithi069.stdout:249108103201 2023-07-06T20:18:11.024 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d96b4700 1 -- 172.21.15.69:0/1246024899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c00095f0 con 0x7fe8d4105b50 2023-07-06T20:18:11.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d96b4700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d4105b50 0x7fe8d4073400 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00ef30 tx=0x7fe8cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:11.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d1ffb700 1 -- 172.21.15.69:0/1246024899 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc018070 con 0x7fe8d4105b50 2023-07-06T20:18:11.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d1ffb700 1 -- 172.21.15.69:0/1246024899 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8cc00f040 con 0x7fe8d4105b50 2023-07-06T20:18:11.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/1246024899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d41a4610 con 0x7fe8d4105b50 2023-07-06T20:18:11.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.217+0000 7fe8d1ffb700 1 -- 172.21.15.69:0/1246024899 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00cd00 con 0x7fe8d4105b50 2023-07-06T20:18:11.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.218+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/1246024899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d41a4b10 con 0x7fe8d4105b50 2023-07-06T20:18:11.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.219+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/1246024899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d404eb90 con 0x7fe8d4105b50 2023-07-06T20:18:11.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.221+0000 7fe8d1ffb700 1 -- 172.21.15.69:0/1246024899 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe8cc003a00 con 0x7fe8d4105b50 2023-07-06T20:18:11.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.222+0000 7fe8d1ffb700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe8c40a11e0 0x7fe8c40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:11.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.222+0000 7fe8d1ffb700 1 -- 172.21.15.69:0/1246024899 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe8cc0cc630 con 0x7fe8d4105b50 2023-07-06T20:18:11.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.222+0000 7fe8d8eb3700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe8c40a11e0 0x7fe8c40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:11.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.223+0000 7fe8d8eb3700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe8c40a11e0 0x7fe8c40a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8005950 tx=0x7fe8c800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:11.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.226+0000 7fe8d1ffb700 1 -- 172.21.15.69:0/1246024899 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe8cc095c40 con 0x7fe8d4105b50 2023-07-06T20:18:11.027 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.431+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/1246024899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fe8d41a4e60 con 0x7fe8d4105b50 2023-07-06T20:18:11.027 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.431+0000 7fe8d1ffb700 1 -- 172.21.15.69:0/1246024899 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe8cc099260 con 0x7fe8d4105b50 2023-07-06T20:18:11.027 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.433+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/1246024899 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe8c40a11e0 msgr2=0x7fe8c40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:11.027 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.434+0000 7fe8d9eb5700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe8c40a11e0 0x7fe8c40a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8005950 tx=0x7fe8c800b410 comp rx=0 tx=0).stop 2023-07-06T20:18:11.027 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.434+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/1246024899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d4105b50 msgr2=0x7fe8d4073400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:11.027 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.434+0000 7fe8d9eb5700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d4105b50 0x7fe8d4073400 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00ef30 tx=0x7fe8cc00c3a0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.027 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.434+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/1246024899 shutdown_connections 2023-07-06T20:18:11.028 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.434+0000 7fe8d9eb5700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe8c40a11e0 0x7fe8c40a3690 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.028 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.434+0000 7fe8d9eb5700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d4105b50 0x7fe8d4073400 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.028 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.434+0000 7fe8d9eb5700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8d40ffb30 0x7fe8d4072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.028 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.434+0000 7fe8d9eb5700 1 --2- 172.21.15.69:0/1246024899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8d41004e0 0x7fe8d4072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.028 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.434+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/1246024899 >> 172.21.15.69:0/1246024899 conn(0x7fe8d40fb690 msgr2=0x7fe8d40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:11.028 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.434+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/1246024899 shutdown_connections 2023-07-06T20:18:11.029 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:10.434+0000 7fe8d9eb5700 1 -- 172.21.15.69:0/1246024899 wait complete. 2023-07-06T20:18:11.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:10 smithi156 ceph-mon[129280]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 189 B/s wr, 6 op/s; 10 B/s, 0 objects/s recovering 2023-07-06T20:18:11.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:10 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1246024899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-06T20:18:11.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:10 smithi069 ceph-mon[126399]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 189 B/s wr, 6 op/s; 10 B/s, 0 objects/s recovering 2023-07-06T20:18:11.119 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:10 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1246024899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-06T20:18:11.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:10 smithi098 ceph-mon[128968]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 189 B/s wr, 6 op/s; 10 B/s, 0 objects/s recovering 2023-07-06T20:18:11.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:10 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1246024899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-06T20:18:11.464 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.303+0000 7f70b0ae4700 1 -- 172.21.15.69:0/2896499563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70ac0fff30 msgr2=0x7f70ac100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:11.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.303+0000 7f70b0ae4700 1 --2- 172.21.15.69:0/2896499563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70ac0fff30 0x7f70ac100300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f709c0099c0 tx=0x7f709c009cd0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.304+0000 7f70b0ae4700 1 -- 172.21.15.69:0/2896499563 shutdown_connections 2023-07-06T20:18:11.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.304+0000 7f70b0ae4700 1 --2- 172.21.15.69:0/2896499563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70ac104fd0 0x7f70ac10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.304+0000 7f70b0ae4700 1 --2- 172.21.15.69:0/2896499563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70ac0fff30 0x7f70ac100300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.304+0000 7f70b0ae4700 1 --2- 172.21.15.69:0/2896499563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70ac1008d0 0x7f70ac1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.304+0000 7f70b0ae4700 1 -- 172.21.15.69:0/2896499563 >> 172.21.15.69:0/2896499563 conn(0x7f70ac0fb790 msgr2=0x7f70ac0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:11.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.304+0000 7f70b0ae4700 1 -- 172.21.15.69:0/2896499563 shutdown_connections 2023-07-06T20:18:11.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.304+0000 7f70b0ae4700 1 -- 172.21.15.69:0/2896499563 wait complete. 2023-07-06T20:18:11.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.305+0000 7f70b0ae4700 1 Processor -- start 2023-07-06T20:18:11.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.305+0000 7f70b0ae4700 1 -- start start 2023-07-06T20:18:11.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.306+0000 7f70b0ae4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70ac0fff30 0x7f70ac072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:11.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.306+0000 7f70b0ae4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70ac1008d0 0x7f70ac072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:11.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.306+0000 7f70b0ae4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70ac104fd0 0x7f70ac073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:11.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.306+0000 7f70b0ae4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac1a4270 con 0x7f70ac104fd0 2023-07-06T20:18:11.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.306+0000 7f70b0ae4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac1a43e0 con 0x7f70ac1008d0 2023-07-06T20:18:11.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.306+0000 7f70b0ae4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac1a45a0 con 0x7f70ac0fff30 2023-07-06T20:18:11.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.306+0000 7f70aa59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70ac0fff30 0x7f70ac072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:11.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.306+0000 7f70a9d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70ac1008d0 0x7f70ac072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:11.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.306+0000 7f70aa59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70ac0fff30 0x7f70ac072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:40758/0 (socket says 172.21.15.69:40758) 2023-07-06T20:18:11.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.306+0000 7f70aad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70ac104fd0 0x7f70ac073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:11.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.306+0000 7f70aa59c700 1 -- 172.21.15.69:0/583196393 learned_addr learned my addr 172.21.15.69:0/583196393 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:11.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f70aa59c700 1 -- 172.21.15.69:0/583196393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70ac1008d0 msgr2=0x7f70ac072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:11.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f70aa59c700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70ac1008d0 0x7f70ac072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f70aa59c700 1 -- 172.21.15.69:0/583196393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70ac104fd0 msgr2=0x7f70ac073400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:11.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f70aa59c700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70ac104fd0 0x7f70ac073400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f70aa59c700 1 -- 172.21.15.69:0/583196393 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0095f0 con 0x7f70ac0fff30 2023-07-06T20:18:11.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f70a9d9b700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70ac1008d0 0x7f70ac072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:11.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f70aad9d700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70ac104fd0 0x7f70ac073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:11.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f70aa59c700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70ac0fff30 0x7f70ac072940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f709c009990 tx=0x7f709c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:11.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f709b7fe700 1 -- 172.21.15.69:0/583196393 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c01d070 con 0x7f70ac0fff30 2023-07-06T20:18:11.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f70b0ae4700 1 -- 172.21.15.69:0/583196393 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac1a4820 con 0x7f70ac0fff30 2023-07-06T20:18:11.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f709b7fe700 1 -- 172.21.15.69:0/583196393 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c003f90 con 0x7f70ac0fff30 2023-07-06T20:18:11.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f709b7fe700 1 -- 172.21.15.69:0/583196393 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c022ba0 con 0x7f70ac0fff30 2023-07-06T20:18:11.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.307+0000 7f70b0ae4700 1 -- 172.21.15.69:0/583196393 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ac1a4c40 con 0x7f70ac0fff30 2023-07-06T20:18:11.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.309+0000 7f70b0ae4700 1 -- 172.21.15.69:0/583196393 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ac04e340 con 0x7f70ac0fff30 2023-07-06T20:18:11.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.311+0000 7f709b7fe700 1 -- 172.21.15.69:0/583196393 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f709c003ad0 con 0x7f70ac0fff30 2023-07-06T20:18:11.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.312+0000 7f709b7fe700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f70940a11e0 0x7f70940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:11.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.312+0000 7f709b7fe700 1 -- 172.21.15.69:0/583196393 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f709c0ce0d0 con 0x7f70ac0fff30 2023-07-06T20:18:11.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.312+0000 7f70a9d9b700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f70940a11e0 0x7f70940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:11.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.313+0000 7f70a9d9b700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f70940a11e0 0x7f70940a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f70ac074130 tx=0x7f70a000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:11.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.314+0000 7f709b7fe700 1 -- 172.21.15.69:0/583196393 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f709c097740 con 0x7f70ac0fff30 2023-07-06T20:18:11.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.530+0000 7f70b0ae4700 1 -- 172.21.15.69:0/583196393 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f70ac073e70 con 0x7f70ac0fff30 2023-07-06T20:18:11.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.530+0000 7f709b7fe700 1 -- 172.21.15.69:0/583196393 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f709c09ad60 con 0x7f70ac0fff30 2023-07-06T20:18:11.533 INFO:teuthology.orchestra.run.smithi069.stdout:219043332134 2023-07-06T20:18:11.535 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.533+0000 7f70b0ae4700 1 -- 172.21.15.69:0/583196393 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f70940a11e0 msgr2=0x7f70940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:11.535 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.533+0000 7f70b0ae4700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f70940a11e0 0x7f70940a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f70ac074130 tx=0x7f70a000a300 comp rx=0 tx=0).stop 2023-07-06T20:18:11.535 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.533+0000 7f70b0ae4700 1 -- 172.21.15.69:0/583196393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70ac0fff30 msgr2=0x7f70ac072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:11.535 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.533+0000 7f70b0ae4700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70ac0fff30 0x7f70ac072940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f709c009990 tx=0x7f709c005740 comp rx=0 tx=0).stop 2023-07-06T20:18:11.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.533+0000 7f70b0ae4700 1 -- 172.21.15.69:0/583196393 shutdown_connections 2023-07-06T20:18:11.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.533+0000 7f70b0ae4700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f70940a11e0 0x7f70940a3690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.533+0000 7f70b0ae4700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70ac104fd0 0x7f70ac073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.533+0000 7f70b0ae4700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70ac0fff30 0x7f70ac072940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.533+0000 7f70b0ae4700 1 --2- 172.21.15.69:0/583196393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70ac1008d0 0x7f70ac072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:11.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.533+0000 7f70b0ae4700 1 -- 172.21.15.69:0/583196393 >> 172.21.15.69:0/583196393 conn(0x7f70ac0fb790 msgr2=0x7f70ac0fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:11.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.533+0000 7f70b0ae4700 1 -- 172.21.15.69:0/583196393 shutdown_connections 2023-07-06T20:18:11.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.533+0000 7f70b0ae4700 1 -- 172.21.15.69:0/583196393 wait complete. 2023-07-06T20:18:12.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:11 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/583196393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-06T20:18:12.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:11 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/583196393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-06T20:18:12.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:11 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/583196393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-06T20:18:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.974+0000 7f33cd829700 1 -- 172.21.15.69:0/3362774470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33c80fff30 msgr2=0x7f33c8100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.974+0000 7f33cd829700 1 --2- 172.21.15.69:0/3362774470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33c80fff30 0x7f33c8100300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f33b80099c0 tx=0x7f33b8009cd0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.975+0000 7f33cd829700 1 -- 172.21.15.69:0/3362774470 shutdown_connections 2023-07-06T20:18:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.975+0000 7f33cd829700 1 --2- 172.21.15.69:0/3362774470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c8104fd0 0x7f33c810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.975+0000 7f33cd829700 1 --2- 172.21.15.69:0/3362774470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33c80fff30 0x7f33c8100300 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.975+0000 7f33cd829700 1 --2- 172.21.15.69:0/3362774470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33c81008d0 0x7f33c81048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.975+0000 7f33cd829700 1 -- 172.21.15.69:0/3362774470 >> 172.21.15.69:0/3362774470 conn(0x7f33c80fb790 msgr2=0x7f33c80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.975+0000 7f33cd829700 1 -- 172.21.15.69:0/3362774470 shutdown_connections 2023-07-06T20:18:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.975+0000 7f33cd829700 1 -- 172.21.15.69:0/3362774470 wait complete. 2023-07-06T20:18:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.976+0000 7f33cd829700 1 Processor -- start 2023-07-06T20:18:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.976+0000 7f33cd829700 1 -- start start 2023-07-06T20:18:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.977+0000 7f33cd829700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33c80fff30 0x7f33c8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:12.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.977+0000 7f33cd829700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c81008d0 0x7f33c81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.977+0000 7f33cd829700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33c8104fd0 0x7f33c819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.977+0000 7f33cd829700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c819cf70 con 0x7f33c8104fd0 2023-07-06T20:18:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.977+0000 7f33cd829700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c819d0e0 con 0x7f33c80fff30 2023-07-06T20:18:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.977+0000 7f33cd829700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c819d2a0 con 0x7f33c81008d0 2023-07-06T20:18:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.977+0000 7f33c67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c81008d0 0x7f33c81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.977+0000 7f33c67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c81008d0 0x7f33c81984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:40802/0 (socket says 172.21.15.69:40802) 2023-07-06T20:18:12.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.977+0000 7f33c67fc700 1 -- 172.21.15.69:0/3950363208 learned_addr learned my addr 172.21.15.69:0/3950363208 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.977+0000 7f33c77fe700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33c8104fd0 0x7f33c819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:12.706 INFO:teuthology.orchestra.run.smithi069.stdout:283467841564 2023-07-06T20:18:12.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.978+0000 7f33c67fc700 1 -- 172.21.15.69:0/3950363208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33c80fff30 msgr2=0x7f33c8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.978+0000 7f33c67fc700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33c80fff30 0x7f33c8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.978+0000 7f33c67fc700 1 -- 172.21.15.69:0/3950363208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33c8104fd0 msgr2=0x7f33c819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.978+0000 7f33c67fc700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33c8104fd0 0x7f33c819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.978+0000 7f33c67fc700 1 -- 172.21.15.69:0/3950363208 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc009520 con 0x7f33c81008d0 2023-07-06T20:18:12.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.978+0000 7f33c67fc700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c81008d0 0x7f33c81984f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00ea30 tx=0x7f33bc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:12.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.978+0000 7f33cc827700 1 -- 172.21.15.69:0/3950363208 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc009820 con 0x7f33c81008d0 2023-07-06T20:18:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.978+0000 7f33cc827700 1 -- 172.21.15.69:0/3950363208 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33bc0044e0 con 0x7f33c81008d0 2023-07-06T20:18:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.978+0000 7f33cd829700 1 -- 172.21.15.69:0/3950363208 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b80095f0 con 0x7f33c81008d0 2023-07-06T20:18:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.978+0000 7f33cc827700 1 -- 172.21.15.69:0/3950363208 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc003e80 con 0x7f33c81008d0 2023-07-06T20:18:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.978+0000 7f33cd829700 1 -- 172.21.15.69:0/3950363208 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c81a4c10 con 0x7f33c81008d0 2023-07-06T20:18:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.979+0000 7f33cd829700 1 -- 172.21.15.69:0/3950363208 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c804e340 con 0x7f33c81008d0 2023-07-06T20:18:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.982+0000 7f33cc827700 1 -- 172.21.15.69:0/3950363208 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f33bc004000 con 0x7f33c81008d0 2023-07-06T20:18:12.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.983+0000 7f33cc827700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f33b00a1260 0x7f33b00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:12.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.983+0000 7f33cc827700 1 -- 172.21.15.69:0/3950363208 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f33bc014070 con 0x7f33c81008d0 2023-07-06T20:18:12.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.983+0000 7f33c6ffd700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f33b00a1260 0x7f33b00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:12.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.984+0000 7f33c6ffd700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f33b00a1260 0x7f33b00a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f33b80098e0 tx=0x7f33b8003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:12.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.985+0000 7f33cc827700 1 -- 172.21.15.69:0/3950363208 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f33bc096ba0 con 0x7f33c81008d0 2023-07-06T20:18:12.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.198+0000 7f33cd829700 1 -- 172.21.15.69:0/3950363208 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f33c819d460 con 0x7f33c81008d0 2023-07-06T20:18:12.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.199+0000 7f33cc827700 1 -- 172.21.15.69:0/3950363208 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f33bc09a1c0 con 0x7f33c81008d0 2023-07-06T20:18:12.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.201+0000 7f33cd829700 1 -- 172.21.15.69:0/3950363208 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f33b00a1260 msgr2=0x7f33b00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.201+0000 7f33cd829700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f33b00a1260 0x7f33b00a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f33b80098e0 tx=0x7f33b8003680 comp rx=0 tx=0).stop 2023-07-06T20:18:12.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.202+0000 7f33cd829700 1 -- 172.21.15.69:0/3950363208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c81008d0 msgr2=0x7f33c81984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.202+0000 7f33cd829700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c81008d0 0x7f33c81984f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00ea30 tx=0x7f33bc00edf0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.202+0000 7f33cd829700 1 -- 172.21.15.69:0/3950363208 shutdown_connections 2023-07-06T20:18:12.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.202+0000 7f33cd829700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f33b00a1260 0x7f33b00a3710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.202+0000 7f33cd829700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33c8104fd0 0x7f33c819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.202+0000 7f33cd829700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33c80fff30 0x7f33c8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.202+0000 7f33cd829700 1 --2- 172.21.15.69:0/3950363208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33c81008d0 0x7f33c81984f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.202+0000 7f33cd829700 1 -- 172.21.15.69:0/3950363208 >> 172.21.15.69:0/3950363208 conn(0x7f33c80fb790 msgr2=0x7f33c80fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:12.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.202+0000 7f33cd829700 1 -- 172.21.15.69:0/3950363208 shutdown_connections 2023-07-06T20:18:12.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.202+0000 7f33cd829700 1 -- 172.21.15.69:0/3950363208 wait complete. 2023-07-06T20:18:12.778 INFO:teuthology.orchestra.run.smithi069.stdout:304942678041 2023-07-06T20:18:12.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.490+0000 7efd39498700 1 -- 172.21.15.69:0/1255923937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd341008d0 msgr2=0x7efd341048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.490+0000 7efd39498700 1 --2- 172.21.15.69:0/1255923937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd341008d0 0x7efd341048a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd280099c0 tx=0x7efd28009cd0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.490+0000 7efd39498700 1 -- 172.21.15.69:0/1255923937 shutdown_connections 2023-07-06T20:18:12.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.490+0000 7efd39498700 1 --2- 172.21.15.69:0/1255923937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd34104fd0 0x7efd3410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.490+0000 7efd39498700 1 --2- 172.21.15.69:0/1255923937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd340fff30 0x7efd34100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.490+0000 7efd39498700 1 --2- 172.21.15.69:0/1255923937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd341008d0 0x7efd341048a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.490+0000 7efd39498700 1 -- 172.21.15.69:0/1255923937 >> 172.21.15.69:0/1255923937 conn(0x7efd340fb790 msgr2=0x7efd340fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:12.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.490+0000 7efd39498700 1 -- 172.21.15.69:0/1255923937 shutdown_connections 2023-07-06T20:18:12.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.490+0000 7efd39498700 1 -- 172.21.15.69:0/1255923937 wait complete. 2023-07-06T20:18:12.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.491+0000 7efd39498700 1 Processor -- start 2023-07-06T20:18:12.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.491+0000 7efd39498700 1 -- start start 2023-07-06T20:18:12.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.491+0000 7efd39498700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd340fff30 0x7efd34072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:12.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd39498700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd341008d0 0x7efd34072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:12.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd39498700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd34104fd0 0x7efd34073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:12.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd39498700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd341a42b0 con 0x7efd340fff30 2023-07-06T20:18:12.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd39498700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd341a4420 con 0x7efd34104fd0 2023-07-06T20:18:12.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd39498700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd341a45e0 con 0x7efd341008d0 2023-07-06T20:18:12.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd32ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd340fff30 0x7efd34072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:12.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd32ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd340fff30 0x7efd34072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43846/0 (socket says 172.21.15.69:43846) 2023-07-06T20:18:12.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd32ffd700 1 -- 172.21.15.69:0/1627107714 learned_addr learned my addr 172.21.15.69:0/1627107714 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:12.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd337fe700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd34104fd0 0x7efd34073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:12.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd327fc700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd341008d0 0x7efd34072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:12.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd32ffd700 1 -- 172.21.15.69:0/1627107714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd341008d0 msgr2=0x7efd34072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd32ffd700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd341008d0 0x7efd34072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd32ffd700 1 -- 172.21.15.69:0/1627107714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd34104fd0 msgr2=0x7efd34073400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd32ffd700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd34104fd0 0x7efd34073400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.492+0000 7efd32ffd700 1 -- 172.21.15.69:0/1627107714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd24009520 con 0x7efd340fff30 2023-07-06T20:18:12.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.493+0000 7efd32ffd700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd340fff30 0x7efd34072940 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efd2400eb30 tx=0x7efd2400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:12.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.493+0000 7efd1bfff700 1 -- 172.21.15.69:0/1627107714 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24009880 con 0x7efd340fff30 2023-07-06T20:18:12.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.493+0000 7efd39498700 1 -- 172.21.15.69:0/1627107714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd280095f0 con 0x7efd340fff30 2023-07-06T20:18:12.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.493+0000 7efd1bfff700 1 -- 172.21.15.69:0/1627107714 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd240044e0 con 0x7efd340fff30 2023-07-06T20:18:12.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.493+0000 7efd1bfff700 1 -- 172.21.15.69:0/1627107714 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24003d80 con 0x7efd340fff30 2023-07-06T20:18:12.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.493+0000 7efd39498700 1 -- 172.21.15.69:0/1627107714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd341a4c50 con 0x7efd340fff30 2023-07-06T20:18:12.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.494+0000 7efd39498700 1 -- 172.21.15.69:0/1627107714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3404e340 con 0x7efd340fff30 2023-07-06T20:18:12.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.498+0000 7efd1bfff700 1 -- 172.21.15.69:0/1627107714 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efd24003f20 con 0x7efd340fff30 2023-07-06T20:18:12.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.499+0000 7efd1bfff700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7efd1c0a11e0 0x7efd1c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:12.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.500+0000 7efd1bfff700 1 -- 172.21.15.69:0/1627107714 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7efd24014070 con 0x7efd340fff30 2023-07-06T20:18:12.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.500+0000 7efd327fc700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7efd1c0a11e0 0x7efd1c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:12.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.500+0000 7efd1bfff700 1 -- 172.21.15.69:0/1627107714 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7efd24096790 con 0x7efd340fff30 2023-07-06T20:18:12.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.500+0000 7efd327fc700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7efd1c0a11e0 0x7efd1c0a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efd28009fd0 tx=0x7efd28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:12.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.739+0000 7efd39498700 1 -- 172.21.15.69:0/1627107714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7efd341a4ed0 con 0x7efd340fff30 2023-07-06T20:18:12.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.739+0000 7efd1bfff700 1 -- 172.21.15.69:0/1627107714 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7efd24099db0 con 0x7efd340fff30 2023-07-06T20:18:12.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.742+0000 7efd39498700 1 -- 172.21.15.69:0/1627107714 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7efd1c0a11e0 msgr2=0x7efd1c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.742+0000 7efd39498700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7efd1c0a11e0 0x7efd1c0a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efd28009fd0 tx=0x7efd28005fb0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.742+0000 7efd39498700 1 -- 172.21.15.69:0/1627107714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd340fff30 msgr2=0x7efd34072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.742+0000 7efd39498700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd340fff30 0x7efd34072940 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efd2400eb30 tx=0x7efd2400ee40 comp rx=0 tx=0).stop 2023-07-06T20:18:12.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.742+0000 7efd39498700 1 -- 172.21.15.69:0/1627107714 shutdown_connections 2023-07-06T20:18:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.742+0000 7efd39498700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7efd1c0a11e0 0x7efd1c0a3690 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efd28009fd0 tx=0x7efd28005fb0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.742+0000 7efd39498700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd34104fd0 0x7efd34073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.742+0000 7efd39498700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd340fff30 0x7efd34072940 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.742+0000 7efd39498700 1 --2- 172.21.15.69:0/1627107714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd341008d0 0x7efd34072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.742+0000 7efd39498700 1 -- 172.21.15.69:0/1627107714 >> 172.21.15.69:0/1627107714 conn(0x7efd340fb790 msgr2=0x7efd340fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.744+0000 7efd39498700 1 -- 172.21.15.69:0/1627107714 shutdown_connections 2023-07-06T20:18:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.744+0000 7efd39498700 1 -- 172.21.15.69:0/1627107714 wait complete. 2023-07-06T20:18:12.786 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103199 got 249108103201 for osd.8 2023-07-06T20:18:12.786 DEBUG:teuthology.parallel:result is None 2023-07-06T20:18:12.796 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332132 got 219043332134 for osd.7 2023-07-06T20:18:12.796 DEBUG:teuthology.parallel:result is None 2023-07-06T20:18:12.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.710+0000 7fd16d583700 1 -- 172.21.15.69:0/3116618198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1681008d0 msgr2=0x7fd1681048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.710+0000 7fd16d583700 1 --2- 172.21.15.69:0/3116618198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1681008d0 0x7fd1681048a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0099c0 tx=0x7fd15c009cd0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.710+0000 7fd16d583700 1 -- 172.21.15.69:0/3116618198 shutdown_connections 2023-07-06T20:18:12.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.710+0000 7fd16d583700 1 --2- 172.21.15.69:0/3116618198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd168104fd0 0x7fd16810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.710+0000 7fd16d583700 1 --2- 172.21.15.69:0/3116618198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1680fff30 0x7fd168100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.710+0000 7fd16d583700 1 --2- 172.21.15.69:0/3116618198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1681008d0 0x7fd1681048a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.710+0000 7fd16d583700 1 -- 172.21.15.69:0/3116618198 >> 172.21.15.69:0/3116618198 conn(0x7fd1680fb790 msgr2=0x7fd1680fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.711+0000 7fd16d583700 1 -- 172.21.15.69:0/3116618198 shutdown_connections 2023-07-06T20:18:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.711+0000 7fd16d583700 1 -- 172.21.15.69:0/3116618198 wait complete. 2023-07-06T20:18:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.712+0000 7fd16d583700 1 Processor -- start 2023-07-06T20:18:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.712+0000 7fd16d583700 1 -- start start 2023-07-06T20:18:12.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.712+0000 7fd16d583700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1680fff30 0x7fd168197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:12.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.712+0000 7fd16d583700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1681008d0 0x7fd1681984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:12.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.712+0000 7fd16d583700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd168104fd0 0x7fd16819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:12.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.712+0000 7fd16d583700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16819cf50 con 0x7fd1680fff30 2023-07-06T20:18:12.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.712+0000 7fd16d583700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16819d0c0 con 0x7fd1681008d0 2023-07-06T20:18:12.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.712+0000 7fd16d583700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16819d280 con 0x7fd168104fd0 2023-07-06T20:18:12.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.713+0000 7fd166ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1680fff30 0x7fd168197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:12.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.713+0000 7fd166ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1680fff30 0x7fd168197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43814/0 (socket says 172.21.15.69:43814) 2023-07-06T20:18:12.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.713+0000 7fd1667fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1681008d0 0x7fd1681984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:12.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.713+0000 7fd166ffd700 1 -- 172.21.15.69:0/3513868593 learned_addr learned my addr 172.21.15.69:0/3513868593 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:12.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.713+0000 7fd1677fe700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd168104fd0 0x7fd16819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:12.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.713+0000 7fd166ffd700 1 -- 172.21.15.69:0/3513868593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd168104fd0 msgr2=0x7fd16819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.713+0000 7fd166ffd700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd168104fd0 0x7fd16819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.713+0000 7fd166ffd700 1 -- 172.21.15.69:0/3513868593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1681008d0 msgr2=0x7fd1681984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.713+0000 7fd166ffd700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1681008d0 0x7fd1681984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.713+0000 7fd166ffd700 1 -- 172.21.15.69:0/3513868593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c0095f0 con 0x7fd1680fff30 2023-07-06T20:18:12.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.713+0000 7fd1667fc700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1681008d0 0x7fd1681984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:12.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.713+0000 7fd166ffd700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1680fff30 0x7fd168197f90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd15800e9a0 tx=0x7fd15800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:12.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.714+0000 7fd14ffff700 1 -- 172.21.15.69:0/3513868593 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158009790 con 0x7fd1680fff30 2023-07-06T20:18:12.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.714+0000 7fd16d583700 1 -- 172.21.15.69:0/3513868593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16819d560 con 0x7fd1680fff30 2023-07-06T20:18:12.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.714+0000 7fd14ffff700 1 -- 172.21.15.69:0/3513868593 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd158010430 con 0x7fd1680fff30 2023-07-06T20:18:12.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.714+0000 7fd14ffff700 1 -- 172.21.15.69:0/3513868593 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158018680 con 0x7fd1680fff30 2023-07-06T20:18:12.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.714+0000 7fd16d583700 1 -- 172.21.15.69:0/3513868593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1681a4dc0 con 0x7fd1680fff30 2023-07-06T20:18:12.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.715+0000 7fd16d583700 1 -- 172.21.15.69:0/3513868593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16804e340 con 0x7fd1680fff30 2023-07-06T20:18:12.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.717+0000 7fd14ffff700 1 -- 172.21.15.69:0/3513868593 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd1580105a0 con 0x7fd1680fff30 2023-07-06T20:18:12.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.719+0000 7fd14ffff700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd1500a11e0 0x7fd1500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:12.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.719+0000 7fd14ffff700 1 -- 172.21.15.69:0/3513868593 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fd158014070 con 0x7fd1680fff30 2023-07-06T20:18:12.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.719+0000 7fd1667fc700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd1500a11e0 0x7fd1500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:12.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.720+0000 7fd1667fc700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd1500a11e0 0x7fd1500a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009fd0 tx=0x7fd15c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:12.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.721+0000 7fd14ffff700 1 -- 172.21.15.69:0/3513868593 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd158096440 con 0x7fd1680fff30 2023-07-06T20:18:12.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.979+0000 7fd16d583700 1 -- 172.21.15.69:0/3513868593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd1681a5050 con 0x7fd1680fff30 2023-07-06T20:18:12.825 INFO:teuthology.orchestra.run.smithi069.stdout:77309411391 2023-07-06T20:18:12.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.980+0000 7fd14ffff700 1 -- 172.21.15.69:0/3513868593 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd158026090 con 0x7fd1680fff30 2023-07-06T20:18:12.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.982+0000 7fd16d583700 1 -- 172.21.15.69:0/3513868593 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd1500a11e0 msgr2=0x7fd1500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.982+0000 7fd16d583700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd1500a11e0 0x7fd1500a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009fd0 tx=0x7fd15c01a040 comp rx=0 tx=0).stop 2023-07-06T20:18:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.982+0000 7fd16d583700 1 -- 172.21.15.69:0/3513868593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1680fff30 msgr2=0x7fd168197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.982+0000 7fd16d583700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1680fff30 0x7fd168197f90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd15800e9a0 tx=0x7fd15800ecb0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.983+0000 7fd16d583700 1 -- 172.21.15.69:0/3513868593 shutdown_connections 2023-07-06T20:18:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.983+0000 7fd16d583700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd1500a11e0 0x7fd1500a3690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.983+0000 7fd16d583700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd168104fd0 0x7fd16819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.983+0000 7fd16d583700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1680fff30 0x7fd168197f90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.983+0000 7fd16d583700 1 --2- 172.21.15.69:0/3513868593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1681008d0 0x7fd1681984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:12.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.983+0000 7fd16d583700 1 -- 172.21.15.69:0/3513868593 >> 172.21.15.69:0/3513868593 conn(0x7fd1680fb790 msgr2=0x7fd1680fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:12.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.983+0000 7fd16d583700 1 -- 172.21.15.69:0/3513868593 shutdown_connections 2023-07-06T20:18:12.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:11.983+0000 7fd16d583700 1 -- 172.21.15.69:0/3513868593 wait complete. 2023-07-06T20:18:12.940 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:12 smithi069 ceph-mon[126399]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2023-07-06T20:18:12.940 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:12 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3513868593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-06T20:18:12.940 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:12 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3950363208' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-06T20:18:12.941 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:12 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1627107714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-06T20:18:12.941 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:12 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1439653179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-06T20:18:13.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:12 smithi156 ceph-mon[129280]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2023-07-06T20:18:13.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:12 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3513868593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-06T20:18:13.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:12 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3950363208' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-06T20:18:13.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:12 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1627107714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-06T20:18:13.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:12 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1439653179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-06T20:18:13.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:12 smithi098 ceph-mon[128968]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2023-07-06T20:18:13.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:12 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3513868593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-06T20:18:13.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:12 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3950363208' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-06T20:18:13.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:12 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1627107714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-06T20:18:13.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:12 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1439653179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-06T20:18:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.732+0000 7f2651bf2700 1 -- 172.21.15.69:0/3237888937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f264c1004e0 msgr2=0x7f264c10c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.732+0000 7f2651bf2700 1 --2- 172.21.15.69:0/3237888937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f264c1004e0 0x7f264c10c920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f264400b0a0 tx=0x7f264400b3b0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.732+0000 7f2651bf2700 1 -- 172.21.15.69:0/3237888937 shutdown_connections 2023-07-06T20:18:13.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.732+0000 7f2651bf2700 1 --2- 172.21.15.69:0/3237888937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f264c1004e0 0x7f264c10c920 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.732+0000 7f2651bf2700 1 --2- 172.21.15.69:0/3237888937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f264c105b50 0x7f264c105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.732+0000 7f2651bf2700 1 --2- 172.21.15.69:0/3237888937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f264c0ffb30 0x7f264c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.732+0000 7f2651bf2700 1 -- 172.21.15.69:0/3237888937 >> 172.21.15.69:0/3237888937 conn(0x7f264c0fb690 msgr2=0x7f264c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:13.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.732+0000 7f2651bf2700 1 -- 172.21.15.69:0/3237888937 shutdown_connections 2023-07-06T20:18:13.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.732+0000 7f2651bf2700 1 -- 172.21.15.69:0/3237888937 wait complete. 2023-07-06T20:18:13.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.734+0000 7f2651bf2700 1 Processor -- start 2023-07-06T20:18:13.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.734+0000 7f2651bf2700 1 -- start start 2023-07-06T20:18:13.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.734+0000 7f2651bf2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f264c0ffb30 0x7f264c072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:13.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2651bf2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f264c1004e0 0x7f264c072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:13.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2651bf2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f264c105b50 0x7f264c073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:13.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2651bf2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1a41e0 con 0x7f264c105b50 2023-07-06T20:18:13.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2651bf2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1a4350 con 0x7f264c1004e0 2023-07-06T20:18:13.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2651bf2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1a4510 con 0x7f264c0ffb30 2023-07-06T20:18:13.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2650bf0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f264c0ffb30 0x7f264c072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:13.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2650bf0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f264c0ffb30 0x7f264c072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:40852/0 (socket says 172.21.15.69:40852) 2023-07-06T20:18:13.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2650bf0700 1 -- 172.21.15.69:0/347806673 learned_addr learned my addr 172.21.15.69:0/347806673 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:13.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f264bfff700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f264c1004e0 0x7f264c072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:13.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2650bf0700 1 -- 172.21.15.69:0/347806673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f264c1004e0 msgr2=0x7f264c072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:13.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2650bf0700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f264c1004e0 0x7f264c072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2650bf0700 1 -- 172.21.15.69:0/347806673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f264c105b50 msgr2=0x7f264c073400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:13.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2650bf0700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f264c105b50 0x7f264c073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.735+0000 7f2650bf0700 1 -- 172.21.15.69:0/347806673 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2638009520 con 0x7f264c0ffb30 2023-07-06T20:18:13.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.736+0000 7f2650bf0700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f264c0ffb30 0x7f264c072940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f264000b530 tx=0x7f264000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:13.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.736+0000 7f2649ffb700 1 -- 172.21.15.69:0/347806673 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640011830 con 0x7f264c0ffb30 2023-07-06T20:18:13.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.736+0000 7f2651bf2700 1 -- 172.21.15.69:0/347806673 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2644009d00 con 0x7f264c0ffb30 2023-07-06T20:18:13.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.737+0000 7f2649ffb700 1 -- 172.21.15.69:0/347806673 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2640011e50 con 0x7f264c0ffb30 2023-07-06T20:18:13.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.737+0000 7f2649ffb700 1 -- 172.21.15.69:0/347806673 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264000f2e0 con 0x7f264c0ffb30 2023-07-06T20:18:13.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.737+0000 7f2651bf2700 1 -- 172.21.15.69:0/347806673 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c1a4bb0 con 0x7f264c0ffb30 2023-07-06T20:18:13.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.738+0000 7f2651bf2700 1 -- 172.21.15.69:0/347806673 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264c04eb90 con 0x7f264c0ffb30 2023-07-06T20:18:13.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.741+0000 7f2649ffb700 1 -- 172.21.15.69:0/347806673 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2640010380 con 0x7f264c0ffb30 2023-07-06T20:18:13.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.743+0000 7f2649ffb700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f263c0a1260 0x7f263c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:13.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.744+0000 7f2649ffb700 1 -- 172.21.15.69:0/347806673 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f26400cca20 con 0x7f264c0ffb30 2023-07-06T20:18:13.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.744+0000 7f264bfff700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f263c0a1260 0x7f263c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:13.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.744+0000 7f2649ffb700 1 -- 172.21.15.69:0/347806673 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f26400992c0 con 0x7f264c0ffb30 2023-07-06T20:18:13.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.744+0000 7f264bfff700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f263c0a1260 0x7f263c0a3710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f264c074130 tx=0x7f2638009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:13.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.010+0000 7f2651bf2700 1 -- 172.21.15.69:0/347806673 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f264c073e70 con 0x7f264c0ffb30 2023-07-06T20:18:13.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.011+0000 7f2649ffb700 1 -- 172.21.15.69:0/347806673 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2640099550 con 0x7f264c0ffb30 2023-07-06T20:18:13.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.013+0000 7f2651bf2700 1 -- 172.21.15.69:0/347806673 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f263c0a1260 msgr2=0x7f263c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:13.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.013+0000 7f2651bf2700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f263c0a1260 0x7f263c0a3710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f264c074130 tx=0x7f2638009260 comp rx=0 tx=0).stop 2023-07-06T20:18:13.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.013+0000 7f2651bf2700 1 -- 172.21.15.69:0/347806673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f264c0ffb30 msgr2=0x7f264c072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:13.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.013+0000 7f2651bf2700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f264c0ffb30 0x7f264c072940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f264000b530 tx=0x7f264000b840 comp rx=0 tx=0).stop 2023-07-06T20:18:13.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.013+0000 7f2651bf2700 1 -- 172.21.15.69:0/347806673 shutdown_connections 2023-07-06T20:18:13.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.013+0000 7f2651bf2700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f263c0a1260 0x7f263c0a3710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.013+0000 7f2651bf2700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f264c105b50 0x7f264c073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.013+0000 7f2651bf2700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f264c0ffb30 0x7f264c072940 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.013+0000 7f2651bf2700 1 --2- 172.21.15.69:0/347806673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f264c1004e0 0x7f264c072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.013+0000 7f2651bf2700 1 -- 172.21.15.69:0/347806673 >> 172.21.15.69:0/347806673 conn(0x7f264c0fb690 msgr2=0x7f264c103650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:13.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.013+0000 7f2651bf2700 1 -- 172.21.15.69:0/347806673 shutdown_connections 2023-07-06T20:18:13.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.013+0000 7f2651bf2700 1 -- 172.21.15.69:0/347806673 wait complete. 2023-07-06T20:18:13.204 INFO:teuthology.orchestra.run.smithi069.stdout:163208757296 2023-07-06T20:18:13.293 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678038 got 304942678041 for osd.10 2023-07-06T20:18:13.294 DEBUG:teuthology.parallel:result is None 2023-07-06T20:18:13.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.535+0000 7f52c81b3700 1 -- 172.21.15.69:0/2914944743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52c0073290 msgr2=0x7f52c010c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:13.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.535+0000 7f52c81b3700 1 --2- 172.21.15.69:0/2914944743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52c0073290 0x7f52c010c040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009990 tx=0x7f52b0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.535+0000 7f52c81b3700 1 -- 172.21.15.69:0/2914944743 shutdown_connections 2023-07-06T20:18:13.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.535+0000 7f52c81b3700 1 --2- 172.21.15.69:0/2914944743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c010c580 0x7f52c010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.535+0000 7f52c81b3700 1 --2- 172.21.15.69:0/2914944743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c0072980 0x7f52c0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.535+0000 7f52c81b3700 1 --2- 172.21.15.69:0/2914944743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52c0073290 0x7f52c010c040 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.536+0000 7f52c81b3700 1 -- 172.21.15.69:0/2914944743 >> 172.21.15.69:0/2914944743 conn(0x7f52c00fb7c0 msgr2=0x7f52c00fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:13.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.536+0000 7f52c81b3700 1 -- 172.21.15.69:0/2914944743 shutdown_connections 2023-07-06T20:18:13.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.536+0000 7f52c81b3700 1 -- 172.21.15.69:0/2914944743 wait complete. 2023-07-06T20:18:13.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.537+0000 7f52c81b3700 1 Processor -- start 2023-07-06T20:18:13.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.537+0000 7f52c81b3700 1 -- start start 2023-07-06T20:18:13.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.538+0000 7f52c81b3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c0072980 0x7f52c00ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:13.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.538+0000 7f52c81b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52c0073290 0x7f52c01001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:13.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.538+0000 7f52c81b3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c010c580 0x7f52c0100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:13.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.538+0000 7f52c81b3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c01a85f0 con 0x7f52c0072980 2023-07-06T20:18:13.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.538+0000 7f52c81b3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c01a8760 con 0x7f52c0073290 2023-07-06T20:18:13.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.538+0000 7f52c81b3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c01a8920 con 0x7f52c010c580 2023-07-06T20:18:13.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.538+0000 7f52c5f4f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c0072980 0x7f52c00ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:13.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.538+0000 7f52c5f4f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c0072980 0x7f52c00ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43870/0 (socket says 172.21.15.69:43870) 2023-07-06T20:18:13.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.538+0000 7f52c5f4f700 1 -- 172.21.15.69:0/1439653179 learned_addr learned my addr 172.21.15.69:0/1439653179 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:13.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.538+0000 7f52c6750700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c010c580 0x7f52c0100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:13.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.538+0000 7f52c574e700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52c0073290 0x7f52c01001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:13.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.539+0000 7f52c574e700 1 -- 172.21.15.69:0/1439653179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c010c580 msgr2=0x7f52c0100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:13.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.539+0000 7f52c574e700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c010c580 0x7f52c0100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.539+0000 7f52c574e700 1 -- 172.21.15.69:0/1439653179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c0072980 msgr2=0x7f52c00ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:13.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.539+0000 7f52c574e700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c0072980 0x7f52c00ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.539+0000 7f52c574e700 1 -- 172.21.15.69:0/1439653179 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b00095f0 con 0x7f52c0073290 2023-07-06T20:18:13.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.539+0000 7f52c5f4f700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c0072980 0x7f52c00ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:13.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.539+0000 7f52c574e700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52c0073290 0x7f52c01001e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f52b0000c00 tx=0x7f52b0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:13.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.539+0000 7f52b6ffd700 1 -- 172.21.15.69:0/1439653179 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b001d070 con 0x7f52c0073290 2023-07-06T20:18:13.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.539+0000 7f52b6ffd700 1 -- 172.21.15.69:0/1439653179 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b0022470 con 0x7f52c0073290 2023-07-06T20:18:13.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.539+0000 7f52c81b3700 1 -- 172.21.15.69:0/1439653179 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c01a8ba0 con 0x7f52c0073290 2023-07-06T20:18:13.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.539+0000 7f52b6ffd700 1 -- 172.21.15.69:0/1439653179 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0017650 con 0x7f52c0073290 2023-07-06T20:18:13.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.539+0000 7f52c81b3700 1 -- 172.21.15.69:0/1439653179 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c01a9040 con 0x7f52c0073290 2023-07-06T20:18:13.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.540+0000 7f52c81b3700 1 -- 172.21.15.69:0/1439653179 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c0106ea0 con 0x7f52c0073290 2023-07-06T20:18:13.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.542+0000 7f52b6ffd700 1 -- 172.21.15.69:0/1439653179 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f52b00225e0 con 0x7f52c0073290 2023-07-06T20:18:13.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.544+0000 7f52b6ffd700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f52ac0a11e0 0x7f52ac0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:13.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.544+0000 7f52b6ffd700 1 -- 172.21.15.69:0/1439653179 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f52b00cdbc0 con 0x7f52c0073290 2023-07-06T20:18:13.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.544+0000 7f52c5f4f700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f52ac0a11e0 0x7f52ac0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:13.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.545+0000 7f52c5f4f700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f52ac0a11e0 0x7f52ac0a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f52bc011550 tx=0x7f52bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:13.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.546+0000 7f52b6ffd700 1 -- 172.21.15.69:0/1439653179 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f52b00972f0 con 0x7f52c0073290 2023-07-06T20:18:13.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.806+0000 7f52c81b3700 1 -- 172.21.15.69:0/1439653179 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f52c004e340 con 0x7f52c0073290 2023-07-06T20:18:13.608 INFO:teuthology.orchestra.run.smithi069.stdout:335007449109 2023-07-06T20:18:13.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.807+0000 7f52b6ffd700 1 -- 172.21.15.69:0/1439653179 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f52b0027020 con 0x7f52c0073290 2023-07-06T20:18:13.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.810+0000 7f52c81b3700 1 -- 172.21.15.69:0/1439653179 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f52ac0a11e0 msgr2=0x7f52ac0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:13.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.810+0000 7f52c81b3700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f52ac0a11e0 0x7f52ac0a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f52bc011550 tx=0x7f52bc009260 comp rx=0 tx=0).stop 2023-07-06T20:18:13.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.810+0000 7f52c81b3700 1 -- 172.21.15.69:0/1439653179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52c0073290 msgr2=0x7f52c01001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:13.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.810+0000 7f52c81b3700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52c0073290 0x7f52c01001e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f52b0000c00 tx=0x7f52b0009580 comp rx=0 tx=0).stop 2023-07-06T20:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.810+0000 7f52c81b3700 1 -- 172.21.15.69:0/1439653179 shutdown_connections 2023-07-06T20:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.810+0000 7f52c81b3700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f52ac0a11e0 0x7f52ac0a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.810+0000 7f52c81b3700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c010c580 0x7f52c0100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.810+0000 7f52c81b3700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c0072980 0x7f52c00ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.810+0000 7f52c81b3700 1 --2- 172.21.15.69:0/1439653179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52c0073290 0x7f52c01001e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.810+0000 7f52c81b3700 1 -- 172.21.15.69:0/1439653179 >> 172.21.15.69:0/1439653179 conn(0x7f52c00fb7c0 msgr2=0x7f52c0103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.810+0000 7f52c81b3700 1 -- 172.21.15.69:0/1439653179 shutdown_connections 2023-07-06T20:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:12.810+0000 7f52c81b3700 1 -- 172.21.15.69:0/1439653179 wait complete. 2023-07-06T20:18:13.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:13 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/347806673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-06T20:18:14.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.997+0000 7f506f58e700 1 -- 172.21.15.69:0/1975824613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5068104fd0 msgr2=0x7f506810cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:14.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.997+0000 7f506f58e700 1 --2- 172.21.15.69:0/1975824613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5068104fd0 0x7f506810cb50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f506000b0d0 tx=0x7f506000b3e0 comp rx=0 tx=0).stop 2023-07-06T20:18:14.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.997+0000 7f506f58e700 1 -- 172.21.15.69:0/1975824613 shutdown_connections 2023-07-06T20:18:14.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.997+0000 7f506f58e700 1 --2- 172.21.15.69:0/1975824613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5068104fd0 0x7f506810cb50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:14.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.997+0000 7f506f58e700 1 --2- 172.21.15.69:0/1975824613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50680fff30 0x7f5068100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:14.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.997+0000 7f506f58e700 1 --2- 172.21.15.69:0/1975824613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50681008d0 0x7f50681048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:14.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.997+0000 7f506f58e700 1 -- 172.21.15.69:0/1975824613 >> 172.21.15.69:0/1975824613 conn(0x7f50680fb790 msgr2=0x7f50680fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:14.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.997+0000 7f506f58e700 1 -- 172.21.15.69:0/1975824613 shutdown_connections 2023-07-06T20:18:14.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.998+0000 7f506f58e700 1 -- 172.21.15.69:0/1975824613 wait complete. 2023-07-06T20:18:14.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.999+0000 7f506f58e700 1 Processor -- start 2023-07-06T20:18:14.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:13.999+0000 7f506f58e700 1 -- start start 2023-07-06T20:18:14.003 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506f58e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50680fff30 0x7f5068072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:14.003 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506f58e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50681008d0 0x7f5068072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:14.003 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506f58e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5068104fd0 0x7f5068073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:14.003 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506f58e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50681a4300 con 0x7f50680fff30 2023-07-06T20:18:14.003 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506f58e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50681a4470 con 0x7f50681008d0 2023-07-06T20:18:14.003 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506f58e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50681a4630 con 0x7f5068104fd0 2023-07-06T20:18:14.003 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506d32a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50680fff30 0x7f5068072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:14.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506d32a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50680fff30 0x7f5068072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36864/0 (socket says 172.21.15.69:36864) 2023-07-06T20:18:14.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506d32a700 1 -- 172.21.15.69:0/37590804 learned_addr learned my addr 172.21.15.69:0/37590804 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:14.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506cb29700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50681008d0 0x7f5068072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:14.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506db2b700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5068104fd0 0x7f5068073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:14.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506d32a700 1 -- 172.21.15.69:0/37590804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5068104fd0 msgr2=0x7f5068073400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:14.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506d32a700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5068104fd0 0x7f5068073400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:14.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506d32a700 1 -- 172.21.15.69:0/37590804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50681008d0 msgr2=0x7f5068072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:14.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506d32a700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50681008d0 0x7f5068072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:14.005 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.000+0000 7f506d32a700 1 -- 172.21.15.69:0/37590804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5060009d00 con 0x7f50680fff30 2023-07-06T20:18:14.005 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.001+0000 7f506d32a700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50680fff30 0x7f5068072940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f506400e9a0 tx=0x7f506400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:14.005 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.001+0000 7f505e7fc700 1 -- 172.21.15.69:0/37590804 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064009790 con 0x7f50680fff30 2023-07-06T20:18:14.006 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.001+0000 7f505e7fc700 1 -- 172.21.15.69:0/37590804 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5064010430 con 0x7f50680fff30 2023-07-06T20:18:14.006 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.001+0000 7f506f58e700 1 -- 172.21.15.69:0/37590804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50681a4910 con 0x7f50680fff30 2023-07-06T20:18:14.006 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.001+0000 7f505e7fc700 1 -- 172.21.15.69:0/37590804 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064018680 con 0x7f50680fff30 2023-07-06T20:18:14.006 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.001+0000 7f506f58e700 1 -- 172.21.15.69:0/37590804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50681a4e60 con 0x7f50680fff30 2023-07-06T20:18:14.006 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.002+0000 7f506f58e700 1 -- 172.21.15.69:0/37590804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506804e340 con 0x7f50680fff30 2023-07-06T20:18:14.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.004+0000 7f505e7fc700 1 -- 172.21.15.69:0/37590804 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f50640105a0 con 0x7f50680fff30 2023-07-06T20:18:14.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.005+0000 7f505e7fc700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f50540a1260 0x7f50540a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:14.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.005+0000 7f505e7fc700 1 -- 172.21.15.69:0/37590804 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f5064014070 con 0x7f50680fff30 2023-07-06T20:18:14.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.006+0000 7f506cb29700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f50540a1260 0x7f50540a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:14.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.006+0000 7f506cb29700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f50540a1260 0x7f50540a3710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5068074130 tx=0x7f5058009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:14.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.009+0000 7f505e7fc700 1 -- 172.21.15.69:0/37590804 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5064051770 con 0x7f50680fff30 2023-07-06T20:18:14.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:13 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/347806673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-06T20:18:14.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:13 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/347806673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-06T20:18:14.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.211+0000 7f506f58e700 1 -- 172.21.15.69:0/37590804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f50681a5180 con 0x7f50680fff30 2023-07-06T20:18:14.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.211+0000 7f505e7fc700 1 -- 172.21.15.69:0/37590804 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f50640961e0 con 0x7f50680fff30 2023-07-06T20:18:14.214 INFO:teuthology.orchestra.run.smithi069.stdout:197568495659 2023-07-06T20:18:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.214+0000 7f506f58e700 1 -- 172.21.15.69:0/37590804 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f50540a1260 msgr2=0x7f50540a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.214+0000 7f506f58e700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f50540a1260 0x7f50540a3710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5068074130 tx=0x7f5058009260 comp rx=0 tx=0).stop 2023-07-06T20:18:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.214+0000 7f506f58e700 1 -- 172.21.15.69:0/37590804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50680fff30 msgr2=0x7f5068072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.214+0000 7f506f58e700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50680fff30 0x7f5068072940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f506400e9a0 tx=0x7f506400ecb0 comp rx=0 tx=0).stop 2023-07-06T20:18:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.214+0000 7f506f58e700 1 -- 172.21.15.69:0/37590804 shutdown_connections 2023-07-06T20:18:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.214+0000 7f506f58e700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f50540a1260 0x7f50540a3710 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.214+0000 7f506f58e700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5068104fd0 0x7f5068073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.214+0000 7f506f58e700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50680fff30 0x7f5068072940 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.214+0000 7f506f58e700 1 --2- 172.21.15.69:0/37590804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50681008d0 0x7f5068072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.214+0000 7f506f58e700 1 -- 172.21.15.69:0/37590804 >> 172.21.15.69:0/37590804 conn(0x7f50680fb790 msgr2=0x7f50680fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.214+0000 7f506f58e700 1 -- 172.21.15.69:0/37590804 shutdown_connections 2023-07-06T20:18:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:14.214+0000 7f506f58e700 1 -- 172.21.15.69:0/37590804 wait complete. 2023-07-06T20:18:14.342 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:14.557 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757294 got 163208757296 for osd.5 2023-07-06T20:18:14.557 DEBUG:teuthology.parallel:result is None 2023-07-06T20:18:14.570 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841561 got 283467841564 for osd.9 2023-07-06T20:18:14.570 DEBUG:teuthology.parallel:result is None 2023-07-06T20:18:15.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:14 smithi156 ceph-mon[129280]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-06T20:18:15.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:14 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/37590804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-06T20:18:15.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:14 smithi098 ceph-mon[128968]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-06T20:18:15.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:14 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/37590804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-06T20:18:15.156 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:14 smithi069 ceph-mon[126399]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-06T20:18:15.156 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:14 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/37590804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-06T20:18:15.336 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411388 got 77309411391 for osd.2 2023-07-06T20:18:15.337 DEBUG:teuthology.parallel:result is None 2023-07-06T20:18:15.340 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449107 got 335007449109 for osd.11 2023-07-06T20:18:15.340 DEBUG:teuthology.parallel:result is None 2023-07-06T20:18:15.392 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495657 got 197568495659 for osd.6 2023-07-06T20:18:15.393 DEBUG:teuthology.parallel:result is None 2023-07-06T20:18:15.492 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.490+0000 7fe38e9cb700 1 -- 172.21.15.69:0/2757832971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe388104fd0 msgr2=0x7fe38810cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.490+0000 7fe38e9cb700 1 --2- 172.21.15.69:0/2757832971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe388104fd0 0x7fe38810cb50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe38000b0d0 tx=0x7fe38000b3e0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.490+0000 7fe38e9cb700 1 -- 172.21.15.69:0/2757832971 shutdown_connections 2023-07-06T20:18:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.490+0000 7fe38e9cb700 1 --2- 172.21.15.69:0/2757832971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe388104fd0 0x7fe38810cb50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.490+0000 7fe38e9cb700 1 --2- 172.21.15.69:0/2757832971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3880fff30 0x7fe388100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.490+0000 7fe38e9cb700 1 --2- 172.21.15.69:0/2757832971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3881008d0 0x7fe3881048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.490+0000 7fe38e9cb700 1 -- 172.21.15.69:0/2757832971 >> 172.21.15.69:0/2757832971 conn(0x7fe3880fb790 msgr2=0x7fe3880fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:15.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.490+0000 7fe38e9cb700 1 -- 172.21.15.69:0/2757832971 shutdown_connections 2023-07-06T20:18:15.494 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.490+0000 7fe38e9cb700 1 -- 172.21.15.69:0/2757832971 wait complete. 2023-07-06T20:18:15.495 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.491+0000 7fe38e9cb700 1 Processor -- start 2023-07-06T20:18:15.495 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.491+0000 7fe38e9cb700 1 -- start start 2023-07-06T20:18:15.495 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe38e9cb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3880fff30 0x7fe388195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:15.495 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe38e9cb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3881008d0 0x7fe3881963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:15.495 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe38e9cb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe388104fd0 0x7fe38819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:15.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe38e9cb700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38819ae20 con 0x7fe3881008d0 2023-07-06T20:18:15.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe38e9cb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38819af90 con 0x7fe388104fd0 2023-07-06T20:18:15.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe38e9cb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38819b150 con 0x7fe3880fff30 2023-07-06T20:18:15.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe3877fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3881008d0 0x7fe3881963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:15.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe3877fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3881008d0 0x7fe3881963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36894/0 (socket says 172.21.15.69:36894) 2023-07-06T20:18:15.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe38cf68700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe388104fd0 0x7fe38819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:15.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe3877fe700 1 -- 172.21.15.69:0/217604707 learned_addr learned my addr 172.21.15.69:0/217604707 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe387fff700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3880fff30 0x7fe388195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe3877fe700 1 -- 172.21.15.69:0/217604707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3880fff30 msgr2=0x7fe388195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe3877fe700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3880fff30 0x7fe388195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe3877fe700 1 -- 172.21.15.69:0/217604707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe388104fd0 msgr2=0x7fe38819a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe3877fe700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe388104fd0 0x7fe38819a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe3877fe700 1 -- 172.21.15.69:0/217604707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe380009d00 con 0x7fe3881008d0 2023-07-06T20:18:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.492+0000 7fe38cf68700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe388104fd0 0x7fe38819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:15.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.493+0000 7fe3877fe700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3881008d0 0x7fe3881963a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00e970 tx=0x7fe37c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.493+0000 7fe3857fa700 1 -- 172.21.15.69:0/217604707 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c009790 con 0x7fe3881008d0 2023-07-06T20:18:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.493+0000 7fe3857fa700 1 -- 172.21.15.69:0/217604707 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37c00c7e0 con 0x7fe3881008d0 2023-07-06T20:18:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.493+0000 7fe38e9cb700 1 -- 172.21.15.69:0/217604707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38819b430 con 0x7fe3881008d0 2023-07-06T20:18:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.493+0000 7fe3857fa700 1 -- 172.21.15.69:0/217604707 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c0054e0 con 0x7fe3881008d0 2023-07-06T20:18:15.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.493+0000 7fe38e9cb700 1 -- 172.21.15.69:0/217604707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3881a2c90 con 0x7fe3881008d0 2023-07-06T20:18:15.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.494+0000 7fe38e9cb700 1 -- 172.21.15.69:0/217604707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38804e340 con 0x7fe3881008d0 2023-07-06T20:18:15.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.496+0000 7fe3857fa700 1 -- 172.21.15.69:0/217604707 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe37c00c950 con 0x7fe3881008d0 2023-07-06T20:18:15.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.497+0000 7fe3857fa700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe3700a11e0 0x7fe3700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.497+0000 7fe3857fa700 1 -- 172.21.15.69:0/217604707 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe37c014070 con 0x7fe3881008d0 2023-07-06T20:18:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.497+0000 7fe387fff700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe3700a11e0 0x7fe3700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:15.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.498+0000 7fe387fff700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe3700a11e0 0x7fe3700a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe378009c60 tx=0x7fe378009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:15.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.501+0000 7fe3857fa700 1 -- 172.21.15.69:0/217604707 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe37c0962b0 con 0x7fe3881008d0 2023-07-06T20:18:15.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.199+0000 7f3a763cf700 1 -- 172.21.15.69:0/1511557629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a7010a0a0 msgr2=0x7f3a7010c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:15.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.199+0000 7f3a763cf700 1 --2- 172.21.15.69:0/1511557629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a7010a0a0 0x7f3a7010c500 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800b0a0 tx=0x7f3a6800b3b0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.199+0000 7f3a763cf700 1 -- 172.21.15.69:0/1511557629 shutdown_connections 2023-07-06T20:18:15.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.199+0000 7f3a763cf700 1 --2- 172.21.15.69:0/1511557629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a7010a0a0 0x7f3a7010c500 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.199+0000 7f3a763cf700 1 --2- 172.21.15.69:0/1511557629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a70100cc0 0x7f3a70101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.199+0000 7f3a763cf700 1 --2- 172.21.15.69:0/1511557629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a701015d0 0x7f3a70109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.199+0000 7f3a763cf700 1 -- 172.21.15.69:0/1511557629 >> 172.21.15.69:0/1511557629 conn(0x7f3a700fa580 msgr2=0x7f3a700fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:15.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.199+0000 7f3a763cf700 1 -- 172.21.15.69:0/1511557629 shutdown_connections 2023-07-06T20:18:15.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.199+0000 7f3a763cf700 1 -- 172.21.15.69:0/1511557629 wait complete. 2023-07-06T20:18:15.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.200+0000 7f3a763cf700 1 Processor -- start 2023-07-06T20:18:15.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.200+0000 7f3a763cf700 1 -- start start 2023-07-06T20:18:15.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.201+0000 7f3a763cf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a70100cc0 0x7f3a7019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:15.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.201+0000 7f3a763cf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a701015d0 0x7f3a7019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:15.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.201+0000 7f3a763cf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a7010a0a0 0x7f3a70195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:15.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.201+0000 7f3a763cf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a701961b0 con 0x7f3a7010a0a0 2023-07-06T20:18:15.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.201+0000 7f3a763cf700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a70196320 con 0x7f3a70100cc0 2023-07-06T20:18:15.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.201+0000 7f3a763cf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a701964e0 con 0x7f3a701015d0 2023-07-06T20:18:15.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.201+0000 7f3a75bce700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a7010a0a0 0x7f3a70195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:15.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.201+0000 7f3a753cd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a70100cc0 0x7f3a7019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:15.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.201+0000 7f3a75bce700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a7010a0a0 0x7f3a70195b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36880/0 (socket says 172.21.15.69:36880) 2023-07-06T20:18:15.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.201+0000 7f3a75bce700 1 -- 172.21.15.69:0/4220335716 learned_addr learned my addr 172.21.15.69:0/4220335716 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:15.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.201+0000 7f3a74bcc700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a701015d0 0x7f3a7019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:15.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.201+0000 7f3a75bce700 1 -- 172.21.15.69:0/4220335716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a701015d0 msgr2=0x7f3a7019c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:15.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.202+0000 7f3a75bce700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a701015d0 0x7f3a7019c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.202+0000 7f3a75bce700 1 -- 172.21.15.69:0/4220335716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a70100cc0 msgr2=0x7f3a7019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:15.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.202+0000 7f3a75bce700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a70100cc0 0x7f3a7019bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.202+0000 7f3a75bce700 1 -- 172.21.15.69:0/4220335716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68009d00 con 0x7f3a7010a0a0 2023-07-06T20:18:15.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.202+0000 7f3a753cd700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a70100cc0 0x7f3a7019bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:15.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.202+0000 7f3a75bce700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a7010a0a0 0x7f3a70195b40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009fd0 tx=0x7f3a6800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:15.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.202+0000 7f3a667fc700 1 -- 172.21.15.69:0/4220335716 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800e040 con 0x7f3a7010a0a0 2023-07-06T20:18:15.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.202+0000 7f3a667fc700 1 -- 172.21.15.69:0/4220335716 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a680094c0 con 0x7f3a7010a0a0 2023-07-06T20:18:15.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.202+0000 7f3a763cf700 1 -- 172.21.15.69:0/4220335716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a70196760 con 0x7f3a7010a0a0 2023-07-06T20:18:15.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.202+0000 7f3a667fc700 1 -- 172.21.15.69:0/4220335716 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a680052a0 con 0x7f3a7010a0a0 2023-07-06T20:18:15.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.202+0000 7f3a763cf700 1 -- 172.21.15.69:0/4220335716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a701a2ab0 con 0x7f3a7010a0a0 2023-07-06T20:18:15.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.203+0000 7f3a763cf700 1 -- 172.21.15.69:0/4220335716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a70104900 con 0x7f3a7010a0a0 2023-07-06T20:18:15.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.205+0000 7f3a667fc700 1 -- 172.21.15.69:0/4220335716 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3a6801d470 con 0x7f3a7010a0a0 2023-07-06T20:18:15.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.206+0000 7f3a667fc700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3a5c0a1260 0x7f3a5c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:15.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.207+0000 7f3a667fc700 1 -- 172.21.15.69:0/4220335716 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f3a680cdbd0 con 0x7f3a7010a0a0 2023-07-06T20:18:15.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.207+0000 7f3a753cd700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3a5c0a1260 0x7f3a5c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:15.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.207+0000 7f3a753cd700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3a5c0a1260 0x7f3a5c0a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c011490 tx=0x7f3a6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:15.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.211+0000 7f3a667fc700 1 -- 172.21.15.69:0/4220335716 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3a68097240 con 0x7f3a7010a0a0 2023-07-06T20:18:15.530 INFO:teuthology.orchestra.run.smithi069.stdout:55834574915 2023-07-06T20:18:15.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.450+0000 7f3a763cf700 1 -- 172.21.15.69:0/4220335716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3a7004eb90 con 0x7f3a7010a0a0 2023-07-06T20:18:15.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.450+0000 7f3a667fc700 1 -- 172.21.15.69:0/4220335716 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a6809a860 con 0x7f3a7010a0a0 2023-07-06T20:18:15.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.453+0000 7f3a763cf700 1 -- 172.21.15.69:0/4220335716 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3a5c0a1260 msgr2=0x7f3a5c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:15.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.453+0000 7f3a763cf700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3a5c0a1260 0x7f3a5c0a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c011490 tx=0x7f3a6c009260 comp rx=0 tx=0).stop 2023-07-06T20:18:15.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.453+0000 7f3a763cf700 1 -- 172.21.15.69:0/4220335716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a7010a0a0 msgr2=0x7f3a70195b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:15.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.453+0000 7f3a763cf700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a7010a0a0 0x7f3a70195b40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009fd0 tx=0x7f3a6800bce0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.453+0000 7f3a763cf700 1 -- 172.21.15.69:0/4220335716 shutdown_connections 2023-07-06T20:18:15.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.453+0000 7f3a763cf700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3a5c0a1260 0x7f3a5c0a3710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.453+0000 7f3a763cf700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a7010a0a0 0x7f3a70195b40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.453+0000 7f3a763cf700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a70100cc0 0x7f3a7019bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.453+0000 7f3a763cf700 1 --2- 172.21.15.69:0/4220335716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a701015d0 0x7f3a7019c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.453+0000 7f3a763cf700 1 -- 172.21.15.69:0/4220335716 >> 172.21.15.69:0/4220335716 conn(0x7f3a700fa580 msgr2=0x7f3a70101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:15.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.453+0000 7f3a763cf700 1 -- 172.21.15.69:0/4220335716 shutdown_connections 2023-07-06T20:18:15.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.453+0000 7f3a763cf700 1 -- 172.21.15.69:0/4220335716 wait complete. 2023-07-06T20:18:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.743+0000 7fe38e9cb700 1 -- 172.21.15.69:0/217604707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe3881a2f50 con 0x7fe3881008d0 2023-07-06T20:18:15.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.743+0000 7fe3857fa700 1 -- 172.21.15.69:0/217604707 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe37c0998d0 con 0x7fe3881008d0 2023-07-06T20:18:15.746 INFO:teuthology.orchestra.run.smithi069.stdout:34359738440 2023-07-06T20:18:15.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.746+0000 7fe38e9cb700 1 -- 172.21.15.69:0/217604707 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe3700a11e0 msgr2=0x7fe3700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:15.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.746+0000 7fe38e9cb700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe3700a11e0 0x7fe3700a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe378009c60 tx=0x7fe378009260 comp rx=0 tx=0).stop 2023-07-06T20:18:15.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.746+0000 7fe38e9cb700 1 -- 172.21.15.69:0/217604707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3881008d0 msgr2=0x7fe3881963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:15.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.746+0000 7fe38e9cb700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3881008d0 0x7fe3881963a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00e970 tx=0x7fe37c00ec80 comp rx=0 tx=0).stop 2023-07-06T20:18:15.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.746+0000 7fe38e9cb700 1 -- 172.21.15.69:0/217604707 shutdown_connections 2023-07-06T20:18:15.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.746+0000 7fe38e9cb700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe3700a11e0 0x7fe3700a3690 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.746+0000 7fe38e9cb700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe388104fd0 0x7fe38819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.746+0000 7fe38e9cb700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3880fff30 0x7fe388195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.746+0000 7fe38e9cb700 1 --2- 172.21.15.69:0/217604707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3881008d0 0x7fe3881963a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:15.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.746+0000 7fe38e9cb700 1 -- 172.21.15.69:0/217604707 >> 172.21.15.69:0/217604707 conn(0x7fe3880fb790 msgr2=0x7fe3880fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:15.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.746+0000 7fe38e9cb700 1 -- 172.21.15.69:0/217604707 shutdown_connections 2023-07-06T20:18:15.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.746+0000 7fe38e9cb700 1 -- 172.21.15.69:0/217604707 wait complete. 2023-07-06T20:18:16.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:15 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/4220335716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-06T20:18:16.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:15 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/217604707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-06T20:18:16.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:15 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/4220335716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-06T20:18:16.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:15 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/217604707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-06T20:18:16.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.962+0000 7f14cd818700 1 -- 172.21.15.69:0/1086612751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14c80732f0 msgr2=0x7f14c81107a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:16.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.962+0000 7f14cd818700 1 --2- 172.21.15.69:0/1086612751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14c80732f0 0x7f14c81107a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009990 tx=0x7f14b0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:16.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.963+0000 7f14cd818700 1 -- 172.21.15.69:0/1086612751 shutdown_connections 2023-07-06T20:18:16.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.963+0000 7f14cd818700 1 --2- 172.21.15.69:0/1086612751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14c8110ce0 0x7f14c81130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:16.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.963+0000 7f14cd818700 1 --2- 172.21.15.69:0/1086612751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c80729e0 0x7f14c8072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:16.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.963+0000 7f14cd818700 1 --2- 172.21.15.69:0/1086612751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14c80732f0 0x7f14c81107a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:16.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.963+0000 7f14cd818700 1 -- 172.21.15.69:0/1086612751 >> 172.21.15.69:0/1086612751 conn(0x7f14c80fb790 msgr2=0x7f14c80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:16.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.963+0000 7f14cd818700 1 -- 172.21.15.69:0/1086612751 shutdown_connections 2023-07-06T20:18:16.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.963+0000 7f14cd818700 1 -- 172.21.15.69:0/1086612751 wait complete. 2023-07-06T20:18:16.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.964+0000 7f14cd818700 1 Processor -- start 2023-07-06T20:18:16.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.964+0000 7f14cd818700 1 -- start start 2023-07-06T20:18:16.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14cd818700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c80729e0 0x7f14c819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:16.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14cd818700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14c80732f0 0x7f14c819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:16.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14cd818700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14c8110ce0 0x7f14c81a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:16.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14cd818700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c81a13a0 con 0x7f14c80732f0 2023-07-06T20:18:16.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14cd818700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c81a1510 con 0x7f14c8110ce0 2023-07-06T20:18:16.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14cd818700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c81a16d0 con 0x7f14c80729e0 2023-07-06T20:18:16.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14c77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14c8110ce0 0x7f14c81a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:16.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14be7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14c80732f0 0x7f14c819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:16.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14c77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14c8110ce0 0x7f14c81a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:44732/0 (socket says 172.21.15.69:44732) 2023-07-06T20:18:16.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14c77fe700 1 -- 172.21.15.69:0/568080717 learned_addr learned my addr 172.21.15.69:0/568080717 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:16.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14c6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c80729e0 0x7f14c819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:16.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14c77fe700 1 -- 172.21.15.69:0/568080717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c80729e0 msgr2=0x7f14c819c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:16.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.965+0000 7f14c77fe700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c80729e0 0x7f14c819c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:16.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.966+0000 7f14c77fe700 1 -- 172.21.15.69:0/568080717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14c80732f0 msgr2=0x7f14c819c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:16.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.966+0000 7f14c77fe700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14c80732f0 0x7f14c819c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:16.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.966+0000 7f14c77fe700 1 -- 172.21.15.69:0/568080717 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b00095f0 con 0x7f14c8110ce0 2023-07-06T20:18:16.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.966+0000 7f14be7fc700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14c80732f0 0x7f14c819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:16.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.966+0000 7f14c77fe700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14c8110ce0 0x7f14c81a0d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f14c000e970 tx=0x7f14c000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:16.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.966+0000 7f14cc816700 1 -- 172.21.15.69:0/568080717 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0009750 con 0x7f14c8110ce0 2023-07-06T20:18:16.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.966+0000 7f14cd818700 1 -- 172.21.15.69:0/568080717 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c81a19b0 con 0x7f14c8110ce0 2023-07-06T20:18:16.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.966+0000 7f14cc816700 1 -- 172.21.15.69:0/568080717 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14c0005020 con 0x7f14c8110ce0 2023-07-06T20:18:16.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.966+0000 7f14cc816700 1 -- 172.21.15.69:0/568080717 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c000cbf0 con 0x7f14c8110ce0 2023-07-06T20:18:16.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.966+0000 7f14cd818700 1 -- 172.21.15.69:0/568080717 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c81a9210 con 0x7f14c8110ce0 2023-07-06T20:18:16.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.967+0000 7f14cd818700 1 -- 172.21.15.69:0/568080717 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c81031c0 con 0x7f14c8110ce0 2023-07-06T20:18:16.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.969+0000 7f14cc816700 1 -- 172.21.15.69:0/568080717 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f14c0005470 con 0x7f14c8110ce0 2023-07-06T20:18:16.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.970+0000 7f14cc816700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f14b40a11e0 0x7f14b40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:16.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.970+0000 7f14cc816700 1 -- 172.21.15.69:0/568080717 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f14c0014070 con 0x7f14c8110ce0 2023-07-06T20:18:16.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.971+0000 7f14c6ffd700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f14b40a11e0 0x7f14b40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:16.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.971+0000 7f14c6ffd700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f14b40a11e0 0x7f14b40a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009550 tx=0x7f14b80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:16.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:15.972+0000 7f14cc816700 1 -- 172.21.15.69:0/568080717 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f14c00968d0 con 0x7f14c8110ce0 2023-07-06T20:18:16.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.192+0000 7f14cd818700 1 -- 172.21.15.69:0/568080717 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f14c804e340 con 0x7f14c8110ce0 2023-07-06T20:18:16.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.192+0000 7f14cc816700 1 -- 172.21.15.69:0/568080717 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f14c0099ef0 con 0x7f14c8110ce0 2023-07-06T20:18:16.198 INFO:teuthology.orchestra.run.smithi069.stdout:141733920821 2023-07-06T20:18:16.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.195+0000 7f14cd818700 1 -- 172.21.15.69:0/568080717 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f14b40a11e0 msgr2=0x7f14b40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:16.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.195+0000 7f14cd818700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f14b40a11e0 0x7f14b40a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009550 tx=0x7f14b80091b0 comp rx=0 tx=0).stop 2023-07-06T20:18:16.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.195+0000 7f14cd818700 1 -- 172.21.15.69:0/568080717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14c8110ce0 msgr2=0x7f14c81a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:16.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.195+0000 7f14cd818700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14c8110ce0 0x7f14c81a0d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f14c000e970 tx=0x7f14c000ec80 comp rx=0 tx=0).stop 2023-07-06T20:18:16.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.195+0000 7f14cd818700 1 -- 172.21.15.69:0/568080717 shutdown_connections 2023-07-06T20:18:16.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.195+0000 7f14cd818700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f14b40a11e0 0x7f14b40a3690 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:16.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.195+0000 7f14cd818700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14c8110ce0 0x7f14c81a0d90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:16.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.195+0000 7f14cd818700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c80729e0 0x7f14c819c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:16.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.195+0000 7f14cd818700 1 --2- 172.21.15.69:0/568080717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14c80732f0 0x7f14c819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:16.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.195+0000 7f14cd818700 1 -- 172.21.15.69:0/568080717 >> 172.21.15.69:0/568080717 conn(0x7f14c80fb790 msgr2=0x7f14c80fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:16.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.195+0000 7f14cd818700 1 -- 172.21.15.69:0/568080717 shutdown_connections 2023-07-06T20:18:16.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:16.195+0000 7f14cd818700 1 -- 172.21.15.69:0/568080717 wait complete. 2023-07-06T20:18:16.218 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:15 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/4220335716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-06T20:18:16.218 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:15 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/217604707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-06T20:18:16.627 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574912 got 55834574915 for osd.1 2023-07-06T20:18:16.627 DEBUG:teuthology.parallel:result is None 2023-07-06T20:18:16.691 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738438 got 34359738440 for osd.0 2023-07-06T20:18:16.691 DEBUG:teuthology.parallel:result is None 2023-07-06T20:18:17.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:16 smithi156 ceph-mon[129280]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-06T20:18:17.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:16 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/568080717' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-06T20:18:17.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:16 smithi098 ceph-mon[128968]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-06T20:18:17.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:16 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/568080717' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-06T20:18:17.143 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:16 smithi069 ceph-mon[126399]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-06T20:18:17.144 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:16 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/568080717' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-06T20:18:17.202 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920819 got 141733920821 for osd.4 2023-07-06T20:18:17.202 DEBUG:teuthology.parallel:result is None 2023-07-06T20:18:17.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.904+0000 7f613ea1e700 1 -- 172.21.15.69:0/2824965151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61380732f0 msgr2=0x7f613810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:17.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.904+0000 7f613ea1e700 1 --2- 172.21.15.69:0/2824965151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61380732f0 0x7f613810bfd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f612c009990 tx=0x7f612c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:17.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.904+0000 7f613ea1e700 1 -- 172.21.15.69:0/2824965151 shutdown_connections 2023-07-06T20:18:17.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.904+0000 7f613ea1e700 1 --2- 172.21.15.69:0/2824965151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f613810c510 0x7f613810e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.904+0000 7f613ea1e700 1 --2- 172.21.15.69:0/2824965151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61380729e0 0x7f6138072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.904+0000 7f613ea1e700 1 --2- 172.21.15.69:0/2824965151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61380732f0 0x7f613810bfd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.904+0000 7f613ea1e700 1 -- 172.21.15.69:0/2824965151 >> 172.21.15.69:0/2824965151 conn(0x7f61380fb790 msgr2=0x7f61380fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.904+0000 7f613ea1e700 1 -- 172.21.15.69:0/2824965151 shutdown_connections 2023-07-06T20:18:17.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.904+0000 7f613ea1e700 1 -- 172.21.15.69:0/2824965151 wait complete. 2023-07-06T20:18:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.905+0000 7f613ea1e700 1 Processor -- start 2023-07-06T20:18:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.905+0000 7f613ea1e700 1 -- start start 2023-07-06T20:18:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.906+0000 7f613ea1e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61380729e0 0x7f6138197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.906+0000 7f613ea1e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61380732f0 0x7f6138198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.906+0000 7f613ea1e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f613810c510 0x7f613819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:17.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.906+0000 7f613ea1e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613819cf30 con 0x7f613810c510 2023-07-06T20:18:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.906+0000 7f613ea1e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613819d0a0 con 0x7f61380732f0 2023-07-06T20:18:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.906+0000 7f613ea1e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613819d260 con 0x7f61380729e0 2023-07-06T20:18:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.906+0000 7f613cfbb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f613810c510 0x7f613819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.906+0000 7f613cfbb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f613810c510 0x7f613819c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36922/0 (socket says 172.21.15.69:36922) 2023-07-06T20:18:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.906+0000 7f613cfbb700 1 -- 172.21.15.69:0/187514083 learned_addr learned my addr 172.21.15.69:0/187514083 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:17.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.906+0000 7f61377fe700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61380732f0 0x7f6138198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:17.912 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.906+0000 7f6137fff700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61380729e0 0x7f6138197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:17.912 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.907+0000 7f613cfbb700 1 -- 172.21.15.69:0/187514083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61380729e0 msgr2=0x7f6138197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:17.912 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.907+0000 7f613cfbb700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61380729e0 0x7f6138197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:17.912 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.907+0000 7f613cfbb700 1 -- 172.21.15.69:0/187514083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61380732f0 msgr2=0x7f6138198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:17.912 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.907+0000 7f613cfbb700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61380732f0 0x7f6138198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:17.912 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.907+0000 7f613cfbb700 1 -- 172.21.15.69:0/187514083 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c0095f0 con 0x7f613810c510 2023-07-06T20:18:17.912 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.907+0000 7f61377fe700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61380732f0 0x7f6138198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.907+0000 7f613cfbb700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f613810c510 0x7f613819c890 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f613000ef30 tx=0x7f613000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.907+0000 7f61357fa700 1 -- 172.21.15.69:0/187514083 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130018070 con 0x7f613810c510 2023-07-06T20:18:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.907+0000 7f613ea1e700 1 -- 172.21.15.69:0/187514083 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613819d4e0 con 0x7f613810c510 2023-07-06T20:18:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.907+0000 7f61357fa700 1 -- 172.21.15.69:0/187514083 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f613000f040 con 0x7f613810c510 2023-07-06T20:18:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.907+0000 7f61357fa700 1 -- 172.21.15.69:0/187514083 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613000cd00 con 0x7f613810c510 2023-07-06T20:18:17.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.907+0000 7f613ea1e700 1 -- 172.21.15.69:0/187514083 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61381a4d10 con 0x7f613810c510 2023-07-06T20:18:17.914 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.908+0000 7f613ea1e700 1 -- 172.21.15.69:0/187514083 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613804e340 con 0x7f613810c510 2023-07-06T20:18:17.914 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.910+0000 7f61357fa700 1 -- 172.21.15.69:0/187514083 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6130003a00 con 0x7f613810c510 2023-07-06T20:18:17.915 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.911+0000 7f61357fa700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f61200a11e0 0x7f61200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:17.915 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.912+0000 7f61357fa700 1 -- 172.21.15.69:0/187514083 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f61300cc5e0 con 0x7f613810c510 2023-07-06T20:18:17.915 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.912+0000 7f6137fff700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f61200a11e0 0x7f61200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:17.915 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.912+0000 7f6137fff700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f61200a11e0 0x7f61200a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6128009c60 tx=0x7f6128009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:17.917 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:17.915+0000 7f61357fa700 1 -- 172.21.15.69:0/187514083 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6130095bf0 con 0x7f613810c510 2023-07-06T20:18:18.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.114+0000 7f613ea1e700 1 -- 172.21.15.69:0/187514083 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f61381a4fd0 con 0x7f613810c510 2023-07-06T20:18:18.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.114+0000 7f61357fa700 1 -- 172.21.15.69:0/187514083 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6130099210 con 0x7f613810c510 2023-07-06T20:18:18.117 INFO:teuthology.orchestra.run.smithi069.stdout:107374182458 2023-07-06T20:18:18.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.116+0000 7f613ea1e700 1 -- 172.21.15.69:0/187514083 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f61200a11e0 msgr2=0x7f61200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:18.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.116+0000 7f613ea1e700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f61200a11e0 0x7f61200a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6128009c60 tx=0x7f6128009260 comp rx=0 tx=0).stop 2023-07-06T20:18:18.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.116+0000 7f613ea1e700 1 -- 172.21.15.69:0/187514083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f613810c510 msgr2=0x7f613819c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:18.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.116+0000 7f613ea1e700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f613810c510 0x7f613819c890 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f613000ef30 tx=0x7f613000c3a0 comp rx=0 tx=0).stop 2023-07-06T20:18:18.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.116+0000 7f613ea1e700 1 -- 172.21.15.69:0/187514083 shutdown_connections 2023-07-06T20:18:18.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.116+0000 7f613ea1e700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f61200a11e0 0x7f61200a3690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:18.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.116+0000 7f613ea1e700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f613810c510 0x7f613819c890 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:18.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.116+0000 7f613ea1e700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61380729e0 0x7f6138197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:18.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.116+0000 7f613ea1e700 1 --2- 172.21.15.69:0/187514083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61380732f0 0x7f6138198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:18.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.116+0000 7f613ea1e700 1 -- 172.21.15.69:0/187514083 >> 172.21.15.69:0/187514083 conn(0x7f61380fb790 msgr2=0x7f613810e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:18.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.117+0000 7f613ea1e700 1 -- 172.21.15.69:0/187514083 shutdown_connections 2023-07-06T20:18:18.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:18.117+0000 7f613ea1e700 1 -- 172.21.15.69:0/187514083 wait complete. 2023-07-06T20:18:18.577 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182455 got 107374182458 for osd.3 2023-07-06T20:18:18.577 DEBUG:teuthology.parallel:result is None 2023-07-06T20:18:18.578 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-06T20:18:18.578 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph pg dump --format=json 2023-07-06T20:18:19.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:18 smithi098 ceph-mon[128968]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:19.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:18 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/187514083' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-06T20:18:19.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:18 smithi069 ceph-mon[126399]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:19.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:18 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/187514083' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-06T20:18:19.305 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:19.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:18 smithi156 ceph-mon[129280]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:19.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:18 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/187514083' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-06T20:18:20.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.785+0000 7f45e30c4700 1 -- 172.21.15.69:0/2663769149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45dc104fd0 msgr2=0x7f45dc10cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:20.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.785+0000 7f45e30c4700 1 --2- 172.21.15.69:0/2663769149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45dc104fd0 0x7f45dc10cb50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f45d400b0d0 tx=0x7f45d400b3e0 comp rx=0 tx=0).stop 2023-07-06T20:18:20.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.785+0000 7f45e30c4700 1 -- 172.21.15.69:0/2663769149 shutdown_connections 2023-07-06T20:18:20.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.785+0000 7f45e30c4700 1 --2- 172.21.15.69:0/2663769149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45dc104fd0 0x7f45dc10cb50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:20.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.785+0000 7f45e30c4700 1 --2- 172.21.15.69:0/2663769149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45dc0fff30 0x7f45dc100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:20.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.785+0000 7f45e30c4700 1 --2- 172.21.15.69:0/2663769149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45dc1008d0 0x7f45dc1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:20.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.785+0000 7f45e30c4700 1 -- 172.21.15.69:0/2663769149 >> 172.21.15.69:0/2663769149 conn(0x7f45dc0fb790 msgr2=0x7f45dc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:20.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.785+0000 7f45e30c4700 1 -- 172.21.15.69:0/2663769149 shutdown_connections 2023-07-06T20:18:20.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.786+0000 7f45e30c4700 1 -- 172.21.15.69:0/2663769149 wait complete. 2023-07-06T20:18:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.786+0000 7f45e30c4700 1 Processor -- start 2023-07-06T20:18:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.787+0000 7f45e30c4700 1 -- start start 2023-07-06T20:18:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.787+0000 7f45e30c4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45dc0fff30 0x7f45dc1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.787+0000 7f45e30c4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45dc1008d0 0x7f45dc198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.787+0000 7f45e30c4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45dc104fd0 0x7f45dc19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.787+0000 7f45e30c4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc19d0b0 con 0x7f45dc0fff30 2023-07-06T20:18:20.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.787+0000 7f45e30c4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc19d220 con 0x7f45dc1008d0 2023-07-06T20:18:20.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.787+0000 7f45e30c4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc19d3e0 con 0x7f45dc104fd0 2023-07-06T20:18:20.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.787+0000 7f45e0e60700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45dc0fff30 0x7f45dc1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:20.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.787+0000 7f45e0e60700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45dc0fff30 0x7f45dc1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36932/0 (socket says 172.21.15.69:36932) 2023-07-06T20:18:20.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.787+0000 7f45e1661700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45dc104fd0 0x7f45dc19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:20.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.787+0000 7f45e0e60700 1 -- 172.21.15.69:0/3279811524 learned_addr learned my addr 172.21.15.69:0/3279811524 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:20.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.787+0000 7f45dbfff700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45dc1008d0 0x7f45dc198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:20.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.788+0000 7f45e1661700 1 -- 172.21.15.69:0/3279811524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45dc1008d0 msgr2=0x7f45dc198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:20.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.788+0000 7f45e1661700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45dc1008d0 0x7f45dc198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:20.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.788+0000 7f45e1661700 1 -- 172.21.15.69:0/3279811524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45dc0fff30 msgr2=0x7f45dc1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:20.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.788+0000 7f45e1661700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45dc0fff30 0x7f45dc1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:20.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.788+0000 7f45e1661700 1 -- 172.21.15.69:0/3279811524 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d4009d00 con 0x7f45dc104fd0 2023-07-06T20:18:20.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.788+0000 7f45e0e60700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45dc0fff30 0x7f45dc1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:20.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.788+0000 7f45e1661700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45dc104fd0 0x7f45dc19caa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45d400b820 tx=0x7f45d4007870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:20.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.788+0000 7f45d9ffb700 1 -- 172.21.15.69:0/3279811524 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d400e040 con 0x7f45dc104fd0 2023-07-06T20:18:20.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.788+0000 7f45e30c4700 1 -- 172.21.15.69:0/3279811524 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45dc19d660 con 0x7f45dc104fd0 2023-07-06T20:18:20.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.788+0000 7f45d9ffb700 1 -- 172.21.15.69:0/3279811524 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45d401dd50 con 0x7f45dc104fd0 2023-07-06T20:18:20.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.788+0000 7f45d9ffb700 1 -- 172.21.15.69:0/3279811524 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d401bd60 con 0x7f45dc104fd0 2023-07-06T20:18:20.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.788+0000 7f45e30c4700 1 -- 172.21.15.69:0/3279811524 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc1a4f20 con 0x7f45dc104fd0 2023-07-06T20:18:20.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.789+0000 7f45e30c4700 1 -- 172.21.15.69:0/3279811524 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45dc04e340 con 0x7f45dc104fd0 2023-07-06T20:18:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.792+0000 7f45d9ffb700 1 -- 172.21.15.69:0/3279811524 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f45d401b450 con 0x7f45dc104fd0 2023-07-06T20:18:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.793+0000 7f45d9ffb700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f45cc0a11e0 0x7f45cc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.793+0000 7f45d9ffb700 1 -- 172.21.15.69:0/3279811524 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f45d40ceac0 con 0x7f45dc104fd0 2023-07-06T20:18:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.793+0000 7f45e0e60700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f45cc0a11e0 0x7f45cc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:20.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.794+0000 7f45e0e60700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f45cc0a11e0 0x7f45cc0a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f45d0005950 tx=0x7f45d000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.795+0000 7f45d9ffb700 1 -- 172.21.15.69:0/3279811524 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f45d40980d0 con 0x7f45dc104fd0 2023-07-06T20:18:20.994 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:20.992+0000 7f45e30c4700 1 -- 172.21.15.69:0/3279811524 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45dc0fc8c0 con 0x7f45cc0a11e0 2023-07-06T20:18:21.005 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.003+0000 7f45d9ffb700 1 -- 172.21.15.69:0/3279811524 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339936 (secure 0 0 0) 0x7f45dc0fc8c0 con 0x7f45cc0a11e0 2023-07-06T20:18:21.010 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:21.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.009+0000 7f45e30c4700 1 -- 172.21.15.69:0/3279811524 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f45cc0a11e0 msgr2=0x7f45cc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:21.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.009+0000 7f45e30c4700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f45cc0a11e0 0x7f45cc0a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f45d0005950 tx=0x7f45d000b500 comp rx=0 tx=0).stop 2023-07-06T20:18:21.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.009+0000 7f45e30c4700 1 -- 172.21.15.69:0/3279811524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45dc104fd0 msgr2=0x7f45dc19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:21.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.009+0000 7f45e30c4700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45dc104fd0 0x7f45dc19caa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45d400b820 tx=0x7f45d4007870 comp rx=0 tx=0).stop 2023-07-06T20:18:21.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.009+0000 7f45e30c4700 1 -- 172.21.15.69:0/3279811524 shutdown_connections 2023-07-06T20:18:21.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.009+0000 7f45e30c4700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f45cc0a11e0 0x7f45cc0a3690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:21.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.009+0000 7f45e30c4700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45dc104fd0 0x7f45dc19caa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:21.013 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.009+0000 7f45e30c4700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45dc0fff30 0x7f45dc1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:21.013 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.009+0000 7f45e30c4700 1 --2- 172.21.15.69:0/3279811524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45dc1008d0 0x7f45dc198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:21.013 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.009+0000 7f45e30c4700 1 -- 172.21.15.69:0/3279811524 >> 172.21.15.69:0/3279811524 conn(0x7f45dc0fb790 msgr2=0x7f45dc0fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:21.013 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.009+0000 7f45e30c4700 1 -- 172.21.15.69:0/3279811524 shutdown_connections 2023-07-06T20:18:21.013 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:21.009+0000 7f45e30c4700 1 -- 172.21.15.69:0/3279811524 wait complete. 2023-07-06T20:18:21.013 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-07-06T20:18:21.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:20 smithi098 ceph-mon[128968]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-06T20:18:21.144 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:20 smithi069 ceph-mon[126399]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-06T20:18:21.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:20 smithi156 ceph-mon[129280]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-06T20:18:21.475 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-07-06T20:18:19.532416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653435,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":417,"num_read_kb":320,"num_write":333,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336388,"kb_used_data":12964,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605436,"statfs":{"total":1151940427776,"available":1151595966464,"internally_reserved":0,"allocated":13275136,"data_stored":7279293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003237"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744044+0000","last_change":"2023-07-06T20:17:28.575520+0000","last_active":"2023-07-06T20:17:58.744044+0000","last_peered":"2023-07-06T20:17:58.744044+0000","last_clean":"2023-07-06T20:17:58.744044+0000","last_became_active":"2023-07-06T20:17:28.575209+0000","last_became_peered":"2023-07-06T20:17:28.575209+0000","last_unstale":"2023-07-06T20:17:58.744044+0000","last_undegraded":"2023-07-06T20:17:58.744044+0000","last_fullsized":"2023-07-06T20:17:58.744044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:52:25.496969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744067+0000","last_change":"2023-07-06T20:17:28.572547+0000","last_active":"2023-07-06T20:17:58.744067+0000","last_peered":"2023-07-06T20:17:58.744067+0000","last_clean":"2023-07-06T20:17:58.744067+0000","last_became_active":"2023-07-06T20:17:28.572150+0000","last_became_peered":"2023-07-06T20:17:28.572150+0000","last_unstale":"2023-07-06T20:17:58.744067+0000","last_undegraded":"2023-07-06T20:17:58.744067+0000","last_fullsized":"2023-07-06T20:17:58.744067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:12:53.896783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744407+0000","last_change":"2023-07-06T20:17:28.574926+0000","last_active":"2023-07-06T20:17:58.744407+0000","last_peered":"2023-07-06T20:17:58.744407+0000","last_clean":"2023-07-06T20:17:58.744407+0000","last_became_active":"2023-07-06T20:17:28.574602+0000","last_became_peered":"2023-07-06T20:17:28.574602+0000","last_unstale":"2023-07-06T20:17:58.744407+0000","last_undegraded":"2023-07-06T20:17:58.744407+0000","last_fullsized":"2023-07-06T20:17:58.744407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:36:52.873371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862515+0000","last_change":"2023-07-06T20:17:28.571533+0000","last_active":"2023-07-06T20:17:58.862515+0000","last_peered":"2023-07-06T20:17:58.862515+0000","last_clean":"2023-07-06T20:17:58.862515+0000","last_became_active":"2023-07-06T20:17:28.571099+0000","last_became_peered":"2023-07-06T20:17:28.571099+0000","last_unstale":"2023-07-06T20:17:58.862515+0000","last_undegraded":"2023-07-06T20:17:58.862515+0000","last_fullsized":"2023-07-06T20:17:58.862515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:34:07.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746458+0000","last_change":"2023-07-06T20:17:28.572100+0000","last_active":"2023-07-06T20:17:58.746458+0000","last_peered":"2023-07-06T20:17:58.746458+0000","last_clean":"2023-07-06T20:17:58.746458+0000","last_became_active":"2023-07-06T20:17:28.571772+0000","last_became_peered":"2023-07-06T20:17:28.571772+0000","last_unstale":"2023-07-06T20:17:58.746458+0000","last_undegraded":"2023-07-06T20:17:58.746458+0000","last_fullsized":"2023-07-06T20:17:58.746458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:50.574562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744230+0000","last_change":"2023-07-06T20:17:28.577816+0000","last_active":"2023-07-06T20:17:58.744230+0000","last_peered":"2023-07-06T20:17:58.744230+0000","last_clean":"2023-07-06T20:17:58.744230+0000","last_became_active":"2023-07-06T20:17:28.577382+0000","last_became_peered":"2023-07-06T20:17:28.577382+0000","last_unstale":"2023-07-06T20:17:58.744230+0000","last_undegraded":"2023-07-06T20:17:58.744230+0000","last_fullsized":"2023-07-06T20:17:58.744230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:46:12.848233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743783+0000","last_change":"2023-07-06T20:17:28.570415+0000","last_active":"2023-07-06T20:17:58.743783+0000","last_peered":"2023-07-06T20:17:58.743783+0000","last_clean":"2023-07-06T20:17:58.743783+0000","last_became_active":"2023-07-06T20:17:28.570091+0000","last_became_peered":"2023-07-06T20:17:28.570091+0000","last_unstale":"2023-07-06T20:17:58.743783+0000","last_undegraded":"2023-07-06T20:17:58.743783+0000","last_fullsized":"2023-07-06T20:17:58.743783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:19:38.692740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861426+0000","last_change":"2023-07-06T20:17:28.568370+0000","last_active":"2023-07-06T20:17:58.861426+0000","last_peered":"2023-07-06T20:17:58.861426+0000","last_clean":"2023-07-06T20:17:58.861426+0000","last_became_active":"2023-07-06T20:17:28.568032+0000","last_became_peered":"2023-07-06T20:17:28.568032+0000","last_unstale":"2023-07-06T20:17:58.861426+0000","last_undegraded":"2023-07-06T20:17:58.861426+0000","last_fullsized":"2023-07-06T20:17:58.861426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:56:58.507364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747148+0000","last_change":"2023-07-06T20:17:28.563386+0000","last_active":"2023-07-06T20:17:58.747148+0000","last_peered":"2023-07-06T20:17:58.747148+0000","last_clean":"2023-07-06T20:17:58.747148+0000","last_became_active":"2023-07-06T20:17:28.562864+0000","last_became_peered":"2023-07-06T20:17:28.562864+0000","last_unstale":"2023-07-06T20:17:58.747148+0000","last_undegraded":"2023-07-06T20:17:58.747148+0000","last_fullsized":"2023-07-06T20:17:58.747148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:38:18.267804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743951+0000","last_change":"2023-07-06T20:17:28.578016+0000","last_active":"2023-07-06T20:17:58.743951+0000","last_peered":"2023-07-06T20:17:58.743951+0000","last_clean":"2023-07-06T20:17:58.743951+0000","last_became_active":"2023-07-06T20:17:28.577305+0000","last_became_peered":"2023-07-06T20:17:28.577305+0000","last_unstale":"2023-07-06T20:17:58.743951+0000","last_undegraded":"2023-07-06T20:17:58.743951+0000","last_fullsized":"2023-07-06T20:17:58.743951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:27:34.666203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744374+0000","last_change":"2023-07-06T20:17:28.569683+0000","last_active":"2023-07-06T20:17:58.744374+0000","last_peered":"2023-07-06T20:17:58.744374+0000","last_clean":"2023-07-06T20:17:58.744374+0000","last_became_active":"2023-07-06T20:17:28.569070+0000","last_became_peered":"2023-07-06T20:17:28.569070+0000","last_unstale":"2023-07-06T20:17:58.744374+0000","last_undegraded":"2023-07-06T20:17:58.744374+0000","last_fullsized":"2023-07-06T20:17:58.744374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:38:56.624764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862482+0000","last_change":"2023-07-06T20:17:28.579463+0000","last_active":"2023-07-06T20:17:58.862482+0000","last_peered":"2023-07-06T20:17:58.862482+0000","last_clean":"2023-07-06T20:17:58.862482+0000","last_became_active":"2023-07-06T20:17:28.579141+0000","last_became_peered":"2023-07-06T20:17:28.579141+0000","last_unstale":"2023-07-06T20:17:58.862482+0000","last_undegraded":"2023-07-06T20:17:58.862482+0000","last_fullsized":"2023-07-06T20:17:58.862482+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:11.840039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.864016+0000","last_change":"2023-07-06T20:17:58.864016+0000","last_active":"2023-07-06T20:17:58.864016+0000","last_peered":"2023-07-06T20:17:58.864016+0000","last_clean":"2023-07-06T20:17:58.864016+0000","last_became_active":"2023-07-06T20:17:58.863583+0000","last_became_peered":"2023-07-06T20:17:58.863583+0000","last_unstale":"2023-07-06T20:17:58.864016+0000","last_undegraded":"2023-07-06T20:17:58.864016+0000","last_fullsized":"2023-07-06T20:17:58.864016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:18:13.895396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743986+0000","last_change":"2023-07-06T20:17:28.572197+0000","last_active":"2023-07-06T20:17:58.743986+0000","last_peered":"2023-07-06T20:17:58.743986+0000","last_clean":"2023-07-06T20:17:58.743986+0000","last_became_active":"2023-07-06T20:17:28.571816+0000","last_became_peered":"2023-07-06T20:17:28.571816+0000","last_unstale":"2023-07-06T20:17:58.743986+0000","last_undegraded":"2023-07-06T20:17:58.743986+0000","last_fullsized":"2023-07-06T20:17:58.743986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:03:41.788066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746966+0000","last_change":"2023-07-06T20:17:58.746966+0000","last_active":"2023-07-06T20:17:58.746966+0000","last_peered":"2023-07-06T20:17:58.746966+0000","last_clean":"2023-07-06T20:17:58.746966+0000","last_became_active":"2023-07-06T20:17:58.746598+0000","last_became_peered":"2023-07-06T20:17:58.746598+0000","last_unstale":"2023-07-06T20:17:58.746966+0000","last_undegraded":"2023-07-06T20:17:58.746966+0000","last_fullsized":"2023-07-06T20:17:58.746966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:46:09.676858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747319+0000","last_change":"2023-07-06T20:17:28.573552+0000","last_active":"2023-07-06T20:17:58.747319+0000","last_peered":"2023-07-06T20:17:58.747319+0000","last_clean":"2023-07-06T20:17:58.747319+0000","last_became_active":"2023-07-06T20:17:28.573253+0000","last_became_peered":"2023-07-06T20:17:28.573253+0000","last_unstale":"2023-07-06T20:17:58.747319+0000","last_undegraded":"2023-07-06T20:17:58.747319+0000","last_fullsized":"2023-07-06T20:17:58.747319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:41:53.685379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744159+0000","last_change":"2023-07-06T20:17:28.572425+0000","last_active":"2023-07-06T20:17:58.744159+0000","last_peered":"2023-07-06T20:17:58.744159+0000","last_clean":"2023-07-06T20:17:58.744159+0000","last_became_active":"2023-07-06T20:17:28.572104+0000","last_became_peered":"2023-07-06T20:17:28.572104+0000","last_unstale":"2023-07-06T20:17:58.744159+0000","last_undegraded":"2023-07-06T20:17:58.744159+0000","last_fullsized":"2023-07-06T20:17:58.744159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:54:48.624955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862859+0000","last_change":"2023-07-06T20:17:28.573814+0000","last_active":"2023-07-06T20:17:58.862859+0000","last_peered":"2023-07-06T20:17:58.862859+0000","last_clean":"2023-07-06T20:17:58.862859+0000","last_became_active":"2023-07-06T20:17:28.573470+0000","last_became_peered":"2023-07-06T20:17:28.573470+0000","last_unstale":"2023-07-06T20:17:58.862859+0000","last_undegraded":"2023-07-06T20:17:58.862859+0000","last_fullsized":"2023-07-06T20:17:58.862859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:03:31.057892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.865802+0000","last_change":"2023-07-06T20:17:58.865802+0000","last_active":"2023-07-06T20:17:58.865802+0000","last_peered":"2023-07-06T20:17:58.865802+0000","last_clean":"2023-07-06T20:17:58.865802+0000","last_became_active":"2023-07-06T20:17:58.865290+0000","last_became_peered":"2023-07-06T20:17:58.865290+0000","last_unstale":"2023-07-06T20:17:58.865802+0000","last_undegraded":"2023-07-06T20:17:58.865802+0000","last_fullsized":"2023-07-06T20:17:58.865802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:40:17.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.863042+0000","last_change":"2023-07-06T20:17:27.603771+0000","last_active":"2023-07-06T20:17:58.863042+0000","last_peered":"2023-07-06T20:17:58.863042+0000","last_clean":"2023-07-06T20:17:58.863042+0000","last_became_active":"2023-07-06T20:17:27.603404+0000","last_became_peered":"2023-07-06T20:17:27.603404+0000","last_unstale":"2023-07-06T20:17:58.863042+0000","last_undegraded":"2023-07-06T20:17:58.863042+0000","last_fullsized":"2023-07-06T20:17:58.863042+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:17:53.507946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862034+0000","last_change":"2023-07-06T20:17:28.568371+0000","last_active":"2023-07-06T20:17:58.862034+0000","last_peered":"2023-07-06T20:17:58.862034+0000","last_clean":"2023-07-06T20:17:58.862034+0000","last_became_active":"2023-07-06T20:17:28.568034+0000","last_became_peered":"2023-07-06T20:17:28.568034+0000","last_unstale":"2023-07-06T20:17:58.862034+0000","last_undegraded":"2023-07-06T20:17:58.862034+0000","last_fullsized":"2023-07-06T20:17:58.862034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:37:29.119572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862283+0000","last_change":"2023-07-06T20:17:27.609833+0000","last_active":"2023-07-06T20:17:58.862283+0000","last_peered":"2023-07-06T20:17:58.862283+0000","last_clean":"2023-07-06T20:17:58.862283+0000","last_became_active":"2023-07-06T20:17:27.609520+0000","last_became_peered":"2023-07-06T20:17:27.609520+0000","last_unstale":"2023-07-06T20:17:58.862283+0000","last_undegraded":"2023-07-06T20:17:58.862283+0000","last_fullsized":"2023-07-06T20:17:58.862283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:05:16.374974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743868+0000","last_change":"2023-07-06T20:17:28.574520+0000","last_active":"2023-07-06T20:17:58.743868+0000","last_peered":"2023-07-06T20:17:58.743868+0000","last_clean":"2023-07-06T20:17:58.743868+0000","last_became_active":"2023-07-06T20:17:28.574158+0000","last_became_peered":"2023-07-06T20:17:28.574158+0000","last_unstale":"2023-07-06T20:17:58.743868+0000","last_undegraded":"2023-07-06T20:17:58.743868+0000","last_fullsized":"2023-07-06T20:17:58.743868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:10:19.258285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747197+0000","last_change":"2023-07-06T20:17:27.601612+0000","last_active":"2023-07-06T20:17:58.747197+0000","last_peered":"2023-07-06T20:17:58.747197+0000","last_clean":"2023-07-06T20:17:58.747197+0000","last_became_active":"2023-07-06T20:17:27.601313+0000","last_became_peered":"2023-07-06T20:17:27.601313+0000","last_unstale":"2023-07-06T20:17:58.747197+0000","last_undegraded":"2023-07-06T20:17:58.747197+0000","last_fullsized":"2023-07-06T20:17:58.747197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:40:08.926500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746884+0000","last_change":"2023-07-06T20:17:28.566389+0000","last_active":"2023-07-06T20:17:58.746884+0000","last_peered":"2023-07-06T20:17:58.746884+0000","last_clean":"2023-07-06T20:17:58.746884+0000","last_became_active":"2023-07-06T20:17:28.566094+0000","last_became_peered":"2023-07-06T20:17:28.566094+0000","last_unstale":"2023-07-06T20:17:58.746884+0000","last_undegraded":"2023-07-06T20:17:58.746884+0000","last_fullsized":"2023-07-06T20:17:58.746884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:15:29.583885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746810+0000","last_change":"2023-07-06T20:17:27.599893+0000","last_active":"2023-07-06T20:17:58.746810+0000","last_peered":"2023-07-06T20:17:58.746810+0000","last_clean":"2023-07-06T20:17:58.746810+0000","last_became_active":"2023-07-06T20:17:27.599544+0000","last_became_peered":"2023-07-06T20:17:27.599544+0000","last_unstale":"2023-07-06T20:17:58.746810+0000","last_undegraded":"2023-07-06T20:17:58.746810+0000","last_fullsized":"2023-07-06T20:17:58.746810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:52:55.097220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861850+0000","last_change":"2023-07-06T20:17:28.575794+0000","last_active":"2023-07-06T20:17:58.861850+0000","last_peered":"2023-07-06T20:17:58.861850+0000","last_clean":"2023-07-06T20:17:58.861850+0000","last_became_active":"2023-07-06T20:17:28.575456+0000","last_became_peered":"2023-07-06T20:17:28.575456+0000","last_unstale":"2023-07-06T20:17:58.861850+0000","last_undegraded":"2023-07-06T20:17:58.861850+0000","last_fullsized":"2023-07-06T20:17:58.861850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:39:54.919014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":72,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:18:15.627355+0000","last_change":"2023-07-06T20:17:27.596351+0000","last_active":"2023-07-06T20:18:15.627355+0000","last_peered":"2023-07-06T20:18:15.627355+0000","last_clean":"2023-07-06T20:18:15.627355+0000","last_became_active":"2023-07-06T20:17:27.595982+0000","last_became_peered":"2023-07-06T20:17:27.595982+0000","last_unstale":"2023-07-06T20:18:15.627355+0000","last_undegraded":"2023-07-06T20:18:15.627355+0000","last_fullsized":"2023-07-06T20:18:15.627355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:15:37.772241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746328+0000","last_change":"2023-07-06T20:17:28.570018+0000","last_active":"2023-07-06T20:17:58.746328+0000","last_peered":"2023-07-06T20:17:58.746328+0000","last_clean":"2023-07-06T20:17:58.746328+0000","last_became_active":"2023-07-06T20:17:28.569674+0000","last_became_peered":"2023-07-06T20:17:28.569674+0000","last_unstale":"2023-07-06T20:17:58.746328+0000","last_undegraded":"2023-07-06T20:17:58.746328+0000","last_fullsized":"2023-07-06T20:17:58.746328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:24:04.467030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.866940+0000","last_change":"2023-07-06T20:17:58.866940+0000","last_active":"2023-07-06T20:17:58.866940+0000","last_peered":"2023-07-06T20:17:58.866940+0000","last_clean":"2023-07-06T20:17:58.866940+0000","last_became_active":"2023-07-06T20:17:58.863435+0000","last_became_peered":"2023-07-06T20:17:58.863435+0000","last_unstale":"2023-07-06T20:17:58.866940+0000","last_undegraded":"2023-07-06T20:17:58.866940+0000","last_fullsized":"2023-07-06T20:17:58.866940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:32:08.233817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861923+0000","last_change":"2023-07-06T20:17:28.575249+0000","last_active":"2023-07-06T20:17:58.861923+0000","last_peered":"2023-07-06T20:17:58.861923+0000","last_clean":"2023-07-06T20:17:58.861923+0000","last_became_active":"2023-07-06T20:17:28.574954+0000","last_became_peered":"2023-07-06T20:17:28.574954+0000","last_unstale":"2023-07-06T20:17:58.861923+0000","last_undegraded":"2023-07-06T20:17:58.861923+0000","last_fullsized":"2023-07-06T20:17:58.861923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:27:13.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.740609+0000","last_change":"2023-07-06T20:17:27.600552+0000","last_active":"2023-07-06T20:17:58.740609+0000","last_peered":"2023-07-06T20:17:58.740609+0000","last_clean":"2023-07-06T20:17:58.740609+0000","last_became_active":"2023-07-06T20:17:27.600016+0000","last_became_peered":"2023-07-06T20:17:27.600016+0000","last_unstale":"2023-07-06T20:17:58.740609+0000","last_undegraded":"2023-07-06T20:17:58.740609+0000","last_fullsized":"2023-07-06T20:17:58.740609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:11:39.398698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861595+0000","last_change":"2023-07-06T20:17:28.566364+0000","last_active":"2023-07-06T20:17:58.861595+0000","last_peered":"2023-07-06T20:17:58.861595+0000","last_clean":"2023-07-06T20:17:58.861595+0000","last_became_active":"2023-07-06T20:17:28.565474+0000","last_became_peered":"2023-07-06T20:17:28.565474+0000","last_unstale":"2023-07-06T20:17:58.861595+0000","last_undegraded":"2023-07-06T20:17:58.861595+0000","last_fullsized":"2023-07-06T20:17:58.861595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:49:42.093269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861332+0000","last_change":"2023-07-06T20:17:27.598606+0000","last_active":"2023-07-06T20:17:58.861332+0000","last_peered":"2023-07-06T20:17:58.861332+0000","last_clean":"2023-07-06T20:17:58.861332+0000","last_became_active":"2023-07-06T20:17:27.598292+0000","last_became_peered":"2023-07-06T20:17:27.598292+0000","last_unstale":"2023-07-06T20:17:58.861332+0000","last_undegraded":"2023-07-06T20:17:58.861332+0000","last_fullsized":"2023-07-06T20:17:58.861332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:42:01.805838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862700+0000","last_change":"2023-07-06T20:17:28.571246+0000","last_active":"2023-07-06T20:17:58.862700+0000","last_peered":"2023-07-06T20:17:58.862700+0000","last_clean":"2023-07-06T20:17:58.862700+0000","last_became_active":"2023-07-06T20:17:28.570846+0000","last_became_peered":"2023-07-06T20:17:28.570846+0000","last_unstale":"2023-07-06T20:17:58.862700+0000","last_undegraded":"2023-07-06T20:17:58.862700+0000","last_fullsized":"2023-07-06T20:17:58.862700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:45:47.944924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:18:13.623341+0000","last_change":"2023-07-06T20:17:27.595879+0000","last_active":"2023-07-06T20:18:13.623341+0000","last_peered":"2023-07-06T20:18:13.623341+0000","last_clean":"2023-07-06T20:18:13.623341+0000","last_became_active":"2023-07-06T20:17:27.595553+0000","last_became_peered":"2023-07-06T20:17:27.595553+0000","last_unstale":"2023-07-06T20:18:13.623341+0000","last_undegraded":"2023-07-06T20:18:13.623341+0000","last_fullsized":"2023-07-06T20:18:13.623341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:12:02.846365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747175+0000","last_change":"2023-07-06T20:17:28.566816+0000","last_active":"2023-07-06T20:17:58.747175+0000","last_peered":"2023-07-06T20:17:58.747175+0000","last_clean":"2023-07-06T20:17:58.747175+0000","last_became_active":"2023-07-06T20:17:28.566565+0000","last_became_peered":"2023-07-06T20:17:28.566565+0000","last_unstale":"2023-07-06T20:17:58.747175+0000","last_undegraded":"2023-07-06T20:17:58.747175+0000","last_fullsized":"2023-07-06T20:17:58.747175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:50:37.443167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.868853+0000","last_change":"2023-07-06T20:17:58.868853+0000","last_active":"2023-07-06T20:17:58.868853+0000","last_peered":"2023-07-06T20:17:58.868853+0000","last_clean":"2023-07-06T20:17:58.868853+0000","last_became_active":"2023-07-06T20:17:58.863304+0000","last_became_peered":"2023-07-06T20:17:58.863304+0000","last_unstale":"2023-07-06T20:17:58.868853+0000","last_undegraded":"2023-07-06T20:17:58.868853+0000","last_fullsized":"2023-07-06T20:17:58.868853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:16:21.561050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743734+0000","last_change":"2023-07-06T20:17:28.572400+0000","last_active":"2023-07-06T20:17:58.743734+0000","last_peered":"2023-07-06T20:17:58.743734+0000","last_clean":"2023-07-06T20:17:58.743734+0000","last_became_active":"2023-07-06T20:17:28.572047+0000","last_became_peered":"2023-07-06T20:17:28.572047+0000","last_unstale":"2023-07-06T20:17:58.743734+0000","last_undegraded":"2023-07-06T20:17:58.743734+0000","last_fullsized":"2023-07-06T20:17:58.743734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:46:01.868129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862013+0000","last_change":"2023-07-06T20:17:27.603224+0000","last_active":"2023-07-06T20:17:58.862013+0000","last_peered":"2023-07-06T20:17:58.862013+0000","last_clean":"2023-07-06T20:17:58.862013+0000","last_became_active":"2023-07-06T20:17:27.602864+0000","last_became_peered":"2023-07-06T20:17:27.602864+0000","last_unstale":"2023-07-06T20:17:58.862013+0000","last_undegraded":"2023-07-06T20:17:58.862013+0000","last_fullsized":"2023-07-06T20:17:58.862013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:19:13.048990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747083+0000","last_change":"2023-07-06T20:17:28.572683+0000","last_active":"2023-07-06T20:17:58.747083+0000","last_peered":"2023-07-06T20:17:58.747083+0000","last_clean":"2023-07-06T20:17:58.747083+0000","last_became_active":"2023-07-06T20:17:28.572352+0000","last_became_peered":"2023-07-06T20:17:28.572352+0000","last_unstale":"2023-07-06T20:17:58.747083+0000","last_undegraded":"2023-07-06T20:17:58.747083+0000","last_fullsized":"2023-07-06T20:17:58.747083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:24:33.562887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'27","reported_seq":139,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:18:14.630295+0000","last_change":"2023-07-06T20:17:27.597285+0000","last_active":"2023-07-06T20:18:14.630295+0000","last_peered":"2023-07-06T20:18:14.630295+0000","last_clean":"2023-07-06T20:18:14.630295+0000","last_became_active":"2023-07-06T20:17:27.596850+0000","last_became_peered":"2023-07-06T20:17:27.596850+0000","last_unstale":"2023-07-06T20:18:14.630295+0000","last_undegraded":"2023-07-06T20:18:14.630295+0000","last_fullsized":"2023-07-06T20:18:14.630295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:41:53.734853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58527,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":94,"num_read_kb":103,"num_write":28,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746860+0000","last_change":"2023-07-06T20:17:28.569855+0000","last_active":"2023-07-06T20:17:58.746860+0000","last_peered":"2023-07-06T20:17:58.746860+0000","last_clean":"2023-07-06T20:17:58.746860+0000","last_became_active":"2023-07-06T20:17:28.569495+0000","last_became_peered":"2023-07-06T20:17:28.569495+0000","last_unstale":"2023-07-06T20:17:58.746860+0000","last_undegraded":"2023-07-06T20:17:58.746860+0000","last_fullsized":"2023-07-06T20:17:58.746860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:53:51.114104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.741227+0000","last_change":"2023-07-06T20:17:27.605898+0000","last_active":"2023-07-06T20:17:58.741227+0000","last_peered":"2023-07-06T20:17:58.741227+0000","last_clean":"2023-07-06T20:17:58.741227+0000","last_became_active":"2023-07-06T20:17:27.605605+0000","last_became_peered":"2023-07-06T20:17:27.605605+0000","last_unstale":"2023-07-06T20:17:58.741227+0000","last_undegraded":"2023-07-06T20:17:58.741227+0000","last_fullsized":"2023-07-06T20:17:58.741227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:27:10.201428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.740770+0000","last_change":"2023-07-06T20:17:28.579236+0000","last_active":"2023-07-06T20:17:58.740770+0000","last_peered":"2023-07-06T20:17:58.740770+0000","last_clean":"2023-07-06T20:17:58.740770+0000","last_became_active":"2023-07-06T20:17:28.578854+0000","last_became_peered":"2023-07-06T20:17:28.578854+0000","last_unstale":"2023-07-06T20:17:58.740770+0000","last_undegraded":"2023-07-06T20:17:58.740770+0000","last_fullsized":"2023-07-06T20:17:58.740770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:27:54.674535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861554+0000","last_change":"2023-07-06T20:17:27.595724+0000","last_active":"2023-07-06T20:17:58.861554+0000","last_peered":"2023-07-06T20:17:58.861554+0000","last_clean":"2023-07-06T20:17:58.861554+0000","last_became_active":"2023-07-06T20:17:27.595360+0000","last_became_peered":"2023-07-06T20:17:27.595360+0000","last_unstale":"2023-07-06T20:17:58.861554+0000","last_undegraded":"2023-07-06T20:17:58.861554+0000","last_fullsized":"2023-07-06T20:17:58.861554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:26:42.090999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744193+0000","last_change":"2023-07-06T20:17:28.573447+0000","last_active":"2023-07-06T20:17:58.744193+0000","last_peered":"2023-07-06T20:17:58.744193+0000","last_clean":"2023-07-06T20:17:58.744193+0000","last_became_active":"2023-07-06T20:17:28.573161+0000","last_became_peered":"2023-07-06T20:17:28.573161+0000","last_unstale":"2023-07-06T20:17:58.744193+0000","last_undegraded":"2023-07-06T20:17:58.744193+0000","last_fullsized":"2023-07-06T20:17:58.744193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:19:39.016518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":69,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:18:15.637169+0000","last_change":"2023-07-06T20:17:27.596254+0000","last_active":"2023-07-06T20:18:15.637169+0000","last_peered":"2023-07-06T20:18:15.637169+0000","last_clean":"2023-07-06T20:18:15.637169+0000","last_became_active":"2023-07-06T20:17:27.595178+0000","last_became_peered":"2023-07-06T20:17:27.595178+0000","last_unstale":"2023-07-06T20:18:15.637169+0000","last_undegraded":"2023-07-06T20:18:15.637169+0000","last_fullsized":"2023-07-06T20:18:15.637169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:19:24.488674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743517+0000","last_change":"2023-07-06T20:17:28.569745+0000","last_active":"2023-07-06T20:17:58.743517+0000","last_peered":"2023-07-06T20:17:58.743517+0000","last_clean":"2023-07-06T20:17:58.743517+0000","last_became_active":"2023-07-06T20:17:28.569468+0000","last_became_peered":"2023-07-06T20:17:28.569468+0000","last_unstale":"2023-07-06T20:17:58.743517+0000","last_undegraded":"2023-07-06T20:17:58.743517+0000","last_fullsized":"2023-07-06T20:17:58.743517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:43:29.072902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.772326+0000","last_change":"2023-07-06T20:17:58.771936+0000","last_active":"2023-07-06T20:17:58.772326+0000","last_peered":"2023-07-06T20:17:58.772326+0000","last_clean":"2023-07-06T20:17:58.772326+0000","last_became_active":"2023-07-06T20:17:58.748196+0000","last_became_peered":"2023-07-06T20:17:58.748196+0000","last_unstale":"2023-07-06T20:17:58.772326+0000","last_undegraded":"2023-07-06T20:17:58.772326+0000","last_fullsized":"2023-07-06T20:17:58.772326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:51:51.798767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861791+0000","last_change":"2023-07-06T20:17:28.575390+0000","last_active":"2023-07-06T20:17:58.861791+0000","last_peered":"2023-07-06T20:17:58.861791+0000","last_clean":"2023-07-06T20:17:58.861791+0000","last_became_active":"2023-07-06T20:17:28.575083+0000","last_became_peered":"2023-07-06T20:17:28.575083+0000","last_unstale":"2023-07-06T20:17:58.861791+0000","last_undegraded":"2023-07-06T20:17:58.861791+0000","last_fullsized":"2023-07-06T20:17:58.861791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:36.857081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747087+0000","last_change":"2023-07-06T20:17:27.600891+0000","last_active":"2023-07-06T20:17:58.747087+0000","last_peered":"2023-07-06T20:17:58.747087+0000","last_clean":"2023-07-06T20:17:58.747087+0000","last_became_active":"2023-07-06T20:17:27.600533+0000","last_became_peered":"2023-07-06T20:17:27.600533+0000","last_unstale":"2023-07-06T20:17:58.747087+0000","last_undegraded":"2023-07-06T20:17:58.747087+0000","last_fullsized":"2023-07-06T20:17:58.747087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:32:22.858270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.863073+0000","last_change":"2023-07-06T20:17:58.863073+0000","last_active":"2023-07-06T20:17:58.863073+0000","last_peered":"2023-07-06T20:17:58.863073+0000","last_clean":"2023-07-06T20:17:58.863073+0000","last_became_active":"2023-07-06T20:17:58.862613+0000","last_became_peered":"2023-07-06T20:17:58.862613+0000","last_unstale":"2023-07-06T20:17:58.863073+0000","last_undegraded":"2023-07-06T20:17:58.863073+0000","last_fullsized":"2023-07-06T20:17:58.863073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:51:25.715070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862771+0000","last_change":"2023-07-06T20:17:27.604464+0000","last_active":"2023-07-06T20:17:58.862771+0000","last_peered":"2023-07-06T20:17:58.862771+0000","last_clean":"2023-07-06T20:17:58.862771+0000","last_became_active":"2023-07-06T20:17:27.604036+0000","last_became_peered":"2023-07-06T20:17:27.604036+0000","last_unstale":"2023-07-06T20:17:58.862771+0000","last_undegraded":"2023-07-06T20:17:58.862771+0000","last_fullsized":"2023-07-06T20:17:58.862771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:14:15.333486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743848+0000","last_change":"2023-07-06T20:17:28.577302+0000","last_active":"2023-07-06T20:17:58.743848+0000","last_peered":"2023-07-06T20:17:58.743848+0000","last_clean":"2023-07-06T20:17:58.743848+0000","last_became_active":"2023-07-06T20:17:28.576998+0000","last_became_peered":"2023-07-06T20:17:28.576998+0000","last_unstale":"2023-07-06T20:17:58.743848+0000","last_undegraded":"2023-07-06T20:17:58.743848+0000","last_fullsized":"2023-07-06T20:17:58.743848+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:52:54.469918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744736+0000","last_change":"2023-07-06T20:17:27.608391+0000","last_active":"2023-07-06T20:17:58.744736+0000","last_peered":"2023-07-06T20:17:58.744736+0000","last_clean":"2023-07-06T20:17:58.744736+0000","last_became_active":"2023-07-06T20:17:27.608073+0000","last_became_peered":"2023-07-06T20:17:27.608073+0000","last_unstale":"2023-07-06T20:17:58.744736+0000","last_undegraded":"2023-07-06T20:17:58.744736+0000","last_fullsized":"2023-07-06T20:17:58.744736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:46:04.264772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744267+0000","last_change":"2023-07-06T20:17:28.577554+0000","last_active":"2023-07-06T20:17:58.744267+0000","last_peered":"2023-07-06T20:17:58.744267+0000","last_clean":"2023-07-06T20:17:58.744267+0000","last_became_active":"2023-07-06T20:17:28.577208+0000","last_became_peered":"2023-07-06T20:17:28.577208+0000","last_unstale":"2023-07-06T20:17:58.744267+0000","last_undegraded":"2023-07-06T20:17:58.744267+0000","last_fullsized":"2023-07-06T20:17:58.744267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:26:43.899763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861568+0000","last_change":"2023-07-06T20:17:28.571678+0000","last_active":"2023-07-06T20:17:58.861568+0000","last_peered":"2023-07-06T20:17:58.861568+0000","last_clean":"2023-07-06T20:17:58.861568+0000","last_became_active":"2023-07-06T20:17:28.570974+0000","last_became_peered":"2023-07-06T20:17:28.570974+0000","last_unstale":"2023-07-06T20:17:58.861568+0000","last_undegraded":"2023-07-06T20:17:58.861568+0000","last_fullsized":"2023-07-06T20:17:58.861568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:56:31.725747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:18:17.639991+0000","last_change":"2023-07-06T20:17:27.607172+0000","last_active":"2023-07-06T20:18:17.639991+0000","last_peered":"2023-07-06T20:18:17.639991+0000","last_clean":"2023-07-06T20:18:17.639991+0000","last_became_active":"2023-07-06T20:17:27.606841+0000","last_became_peered":"2023-07-06T20:17:27.606841+0000","last_unstale":"2023-07-06T20:18:17.639991+0000","last_undegraded":"2023-07-06T20:18:17.639991+0000","last_fullsized":"2023-07-06T20:18:17.639991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:41:58.034920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861784+0000","last_change":"2023-07-06T20:17:28.567707+0000","last_active":"2023-07-06T20:17:58.861784+0000","last_peered":"2023-07-06T20:17:58.861784+0000","last_clean":"2023-07-06T20:17:58.861784+0000","last_became_active":"2023-07-06T20:17:28.567394+0000","last_became_peered":"2023-07-06T20:17:28.567394+0000","last_unstale":"2023-07-06T20:17:58.861784+0000","last_undegraded":"2023-07-06T20:17:58.861784+0000","last_fullsized":"2023-07-06T20:17:58.861784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:22:52.559395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862245+0000","last_change":"2023-07-06T20:17:27.595746+0000","last_active":"2023-07-06T20:17:58.862245+0000","last_peered":"2023-07-06T20:17:58.862245+0000","last_clean":"2023-07-06T20:17:58.862245+0000","last_became_active":"2023-07-06T20:17:27.595039+0000","last_became_peered":"2023-07-06T20:17:27.595039+0000","last_unstale":"2023-07-06T20:17:58.862245+0000","last_undegraded":"2023-07-06T20:17:58.862245+0000","last_fullsized":"2023-07-06T20:17:58.862245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:51:26.692902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":337,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743684+0000","last_change":"2023-07-06T20:16:23.542324+0000","last_active":"2023-07-06T20:17:58.743684+0000","last_peered":"2023-07-06T20:17:58.743684+0000","last_clean":"2023-07-06T20:17:58.743684+0000","last_became_active":"2023-07-06T20:16:23.541832+0000","last_became_peered":"2023-07-06T20:16:23.541832+0000","last_unstale":"2023-07-06T20:17:58.743684+0000","last_undegraded":"2023-07-06T20:17:58.743684+0000","last_fullsized":"2023-07-06T20:17:58.743684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_clean_scrub_stamp":"2023-07-06T20:13:14.072924+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:43:32.297556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.740438+0000","last_change":"2023-07-06T20:17:28.577309+0000","last_active":"2023-07-06T20:17:58.740438+0000","last_peered":"2023-07-06T20:17:58.740438+0000","last_clean":"2023-07-06T20:17:58.740438+0000","last_became_active":"2023-07-06T20:17:28.576962+0000","last_became_peered":"2023-07-06T20:17:28.576962+0000","last_unstale":"2023-07-06T20:17:58.740438+0000","last_undegraded":"2023-07-06T20:17:58.740438+0000","last_fullsized":"2023-07-06T20:17:58.740438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:27:20.743560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743813+0000","last_change":"2023-07-06T20:17:27.601487+0000","last_active":"2023-07-06T20:17:58.743813+0000","last_peered":"2023-07-06T20:17:58.743813+0000","last_clean":"2023-07-06T20:17:58.743813+0000","last_became_active":"2023-07-06T20:17:27.601195+0000","last_became_peered":"2023-07-06T20:17:27.601195+0000","last_unstale":"2023-07-06T20:17:58.743813+0000","last_undegraded":"2023-07-06T20:17:58.743813+0000","last_fullsized":"2023-07-06T20:17:58.743813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:22:55.958720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747176+0000","last_change":"2023-07-06T20:17:27.610374+0000","last_active":"2023-07-06T20:17:58.747176+0000","last_peered":"2023-07-06T20:17:58.747176+0000","last_clean":"2023-07-06T20:17:58.747176+0000","last_became_active":"2023-07-06T20:17:27.610104+0000","last_became_peered":"2023-07-06T20:17:27.610104+0000","last_unstale":"2023-07-06T20:17:58.747176+0000","last_undegraded":"2023-07-06T20:17:58.747176+0000","last_fullsized":"2023-07-06T20:17:58.747176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:25:52.553271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.740620+0000","last_change":"2023-07-06T20:17:28.570146+0000","last_active":"2023-07-06T20:17:58.740620+0000","last_peered":"2023-07-06T20:17:58.740620+0000","last_clean":"2023-07-06T20:17:58.740620+0000","last_became_active":"2023-07-06T20:17:28.569341+0000","last_became_peered":"2023-07-06T20:17:28.569341+0000","last_unstale":"2023-07-06T20:17:58.740620+0000","last_undegraded":"2023-07-06T20:17:58.740620+0000","last_fullsized":"2023-07-06T20:17:58.740620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:47:51.639184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862987+0000","last_change":"2023-07-06T20:17:27.604630+0000","last_active":"2023-07-06T20:17:58.862987+0000","last_peered":"2023-07-06T20:17:58.862987+0000","last_clean":"2023-07-06T20:17:58.862987+0000","last_became_active":"2023-07-06T20:17:27.604118+0000","last_became_peered":"2023-07-06T20:17:27.604118+0000","last_unstale":"2023-07-06T20:17:58.862987+0000","last_undegraded":"2023-07-06T20:17:58.862987+0000","last_fullsized":"2023-07-06T20:17:58.862987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:02:10.943054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862071+0000","last_change":"2023-07-06T20:17:28.571753+0000","last_active":"2023-07-06T20:17:58.862071+0000","last_peered":"2023-07-06T20:17:58.862071+0000","last_clean":"2023-07-06T20:17:58.862071+0000","last_became_active":"2023-07-06T20:17:28.571073+0000","last_became_peered":"2023-07-06T20:17:28.571073+0000","last_unstale":"2023-07-06T20:17:58.862071+0000","last_undegraded":"2023-07-06T20:17:58.862071+0000","last_fullsized":"2023-07-06T20:17:58.862071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:43:01.162549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862660+0000","last_change":"2023-07-06T20:17:27.594354+0000","last_active":"2023-07-06T20:17:58.862660+0000","last_peered":"2023-07-06T20:17:58.862660+0000","last_clean":"2023-07-06T20:17:58.862660+0000","last_became_active":"2023-07-06T20:17:27.594042+0000","last_became_peered":"2023-07-06T20:17:27.594042+0000","last_unstale":"2023-07-06T20:17:58.862660+0000","last_undegraded":"2023-07-06T20:17:58.862660+0000","last_fullsized":"2023-07-06T20:17:58.862660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:16:57.362089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744208+0000","last_change":"2023-07-06T20:17:28.576170+0000","last_active":"2023-07-06T20:17:58.744208+0000","last_peered":"2023-07-06T20:17:58.744208+0000","last_clean":"2023-07-06T20:17:58.744208+0000","last_became_active":"2023-07-06T20:17:28.575493+0000","last_became_peered":"2023-07-06T20:17:28.575493+0000","last_unstale":"2023-07-06T20:17:58.744208+0000","last_undegraded":"2023-07-06T20:17:58.744208+0000","last_fullsized":"2023-07-06T20:17:58.744208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:03:49.657987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747401+0000","last_change":"2023-07-06T20:17:27.602984+0000","last_active":"2023-07-06T20:17:58.747401+0000","last_peered":"2023-07-06T20:17:58.747401+0000","last_clean":"2023-07-06T20:17:58.747401+0000","last_became_active":"2023-07-06T20:17:27.602720+0000","last_became_peered":"2023-07-06T20:17:27.602720+0000","last_unstale":"2023-07-06T20:17:58.747401+0000","last_undegraded":"2023-07-06T20:17:58.747401+0000","last_fullsized":"2023-07-06T20:17:58.747401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:38:05.396361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.745930+0000","last_change":"2023-07-06T20:17:28.565954+0000","last_active":"2023-07-06T20:17:58.745930+0000","last_peered":"2023-07-06T20:17:58.745930+0000","last_clean":"2023-07-06T20:17:58.745930+0000","last_became_active":"2023-07-06T20:17:28.565300+0000","last_became_peered":"2023-07-06T20:17:28.565300+0000","last_unstale":"2023-07-06T20:17:58.745930+0000","last_undegraded":"2023-07-06T20:17:58.745930+0000","last_fullsized":"2023-07-06T20:17:58.745930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:18:37.459355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861635+0000","last_change":"2023-07-06T20:17:27.597859+0000","last_active":"2023-07-06T20:17:58.861635+0000","last_peered":"2023-07-06T20:17:58.861635+0000","last_clean":"2023-07-06T20:17:58.861635+0000","last_became_active":"2023-07-06T20:17:27.597479+0000","last_became_peered":"2023-07-06T20:17:27.597479+0000","last_unstale":"2023-07-06T20:17:58.861635+0000","last_undegraded":"2023-07-06T20:17:58.861635+0000","last_fullsized":"2023-07-06T20:17:58.861635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:12:44.598603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744672+0000","last_change":"2023-07-06T20:17:28.573069+0000","last_active":"2023-07-06T20:17:58.744672+0000","last_peered":"2023-07-06T20:17:58.744672+0000","last_clean":"2023-07-06T20:17:58.744672+0000","last_became_active":"2023-07-06T20:17:28.572742+0000","last_became_peered":"2023-07-06T20:17:28.572742+0000","last_unstale":"2023-07-06T20:17:58.744672+0000","last_undegraded":"2023-07-06T20:17:58.744672+0000","last_fullsized":"2023-07-06T20:17:58.744672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:52:06.992269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.741766+0000","last_change":"2023-07-06T20:17:27.599259+0000","last_active":"2023-07-06T20:17:58.741766+0000","last_peered":"2023-07-06T20:17:58.741766+0000","last_clean":"2023-07-06T20:17:58.741766+0000","last_became_active":"2023-07-06T20:17:27.598883+0000","last_became_peered":"2023-07-06T20:17:27.598883+0000","last_unstale":"2023-07-06T20:17:58.741766+0000","last_undegraded":"2023-07-06T20:17:58.741766+0000","last_fullsized":"2023-07-06T20:17:58.741766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:05:19.031900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861912+0000","last_change":"2023-07-06T20:17:28.573874+0000","last_active":"2023-07-06T20:17:58.861912+0000","last_peered":"2023-07-06T20:17:58.861912+0000","last_clean":"2023-07-06T20:17:58.861912+0000","last_became_active":"2023-07-06T20:17:28.573379+0000","last_became_peered":"2023-07-06T20:17:28.573379+0000","last_unstale":"2023-07-06T20:17:58.861912+0000","last_undegraded":"2023-07-06T20:17:58.861912+0000","last_fullsized":"2023-07-06T20:17:58.861912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:45:45.647998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746518+0000","last_change":"2023-07-06T20:17:27.601474+0000","last_active":"2023-07-06T20:17:58.746518+0000","last_peered":"2023-07-06T20:17:58.746518+0000","last_clean":"2023-07-06T20:17:58.746518+0000","last_became_active":"2023-07-06T20:17:27.601099+0000","last_became_peered":"2023-07-06T20:17:27.601099+0000","last_unstale":"2023-07-06T20:17:58.746518+0000","last_undegraded":"2023-07-06T20:17:58.746518+0000","last_fullsized":"2023-07-06T20:17:58.746518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:59:28.147471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744872+0000","last_change":"2023-07-06T20:17:28.571170+0000","last_active":"2023-07-06T20:17:58.744872+0000","last_peered":"2023-07-06T20:17:58.744872+0000","last_clean":"2023-07-06T20:17:58.744872+0000","last_became_active":"2023-07-06T20:17:28.570874+0000","last_became_peered":"2023-07-06T20:17:28.570874+0000","last_unstale":"2023-07-06T20:17:58.744872+0000","last_undegraded":"2023-07-06T20:17:58.744872+0000","last_fullsized":"2023-07-06T20:17:58.744872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:52:55.391441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.741459+0000","last_change":"2023-07-06T20:17:27.603604+0000","last_active":"2023-07-06T20:17:58.741459+0000","last_peered":"2023-07-06T20:17:58.741459+0000","last_clean":"2023-07-06T20:17:58.741459+0000","last_became_active":"2023-07-06T20:17:27.603015+0000","last_became_peered":"2023-07-06T20:17:27.603015+0000","last_unstale":"2023-07-06T20:17:58.741459+0000","last_undegraded":"2023-07-06T20:17:58.741459+0000","last_fullsized":"2023-07-06T20:17:58.741459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:09:30.856672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744507+0000","last_change":"2023-07-06T20:17:28.567906+0000","last_active":"2023-07-06T20:17:58.744507+0000","last_peered":"2023-07-06T20:17:58.744507+0000","last_clean":"2023-07-06T20:17:58.744507+0000","last_became_active":"2023-07-06T20:17:28.567509+0000","last_became_peered":"2023-07-06T20:17:28.567509+0000","last_unstale":"2023-07-06T20:17:58.744507+0000","last_undegraded":"2023-07-06T20:17:58.744507+0000","last_fullsized":"2023-07-06T20:17:58.744507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:36:49.402373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861781+0000","last_change":"2023-07-06T20:17:27.598323+0000","last_active":"2023-07-06T20:17:58.861781+0000","last_peered":"2023-07-06T20:17:58.861781+0000","last_clean":"2023-07-06T20:17:58.861781+0000","last_became_active":"2023-07-06T20:17:27.598032+0000","last_became_peered":"2023-07-06T20:17:27.598032+0000","last_unstale":"2023-07-06T20:17:58.861781+0000","last_undegraded":"2023-07-06T20:17:58.861781+0000","last_fullsized":"2023-07-06T20:17:58.861781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:54:23.185321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743964+0000","last_change":"2023-07-06T20:17:28.571756+0000","last_active":"2023-07-06T20:17:58.743964+0000","last_peered":"2023-07-06T20:17:58.743964+0000","last_clean":"2023-07-06T20:17:58.743964+0000","last_became_active":"2023-07-06T20:17:28.571440+0000","last_became_peered":"2023-07-06T20:17:28.571440+0000","last_unstale":"2023-07-06T20:17:58.743964+0000","last_undegraded":"2023-07-06T20:17:58.743964+0000","last_fullsized":"2023-07-06T20:17:58.743964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:13:22.304976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.741065+0000","last_change":"2023-07-06T20:17:27.602620+0000","last_active":"2023-07-06T20:17:58.741065+0000","last_peered":"2023-07-06T20:17:58.741065+0000","last_clean":"2023-07-06T20:17:58.741065+0000","last_became_active":"2023-07-06T20:17:27.602224+0000","last_became_peered":"2023-07-06T20:17:27.602224+0000","last_unstale":"2023-07-06T20:17:58.741065+0000","last_undegraded":"2023-07-06T20:17:58.741065+0000","last_fullsized":"2023-07-06T20:17:58.741065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:43:07.550009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744425+0000","last_change":"2023-07-06T20:17:28.569203+0000","last_active":"2023-07-06T20:17:58.744425+0000","last_peered":"2023-07-06T20:17:58.744425+0000","last_clean":"2023-07-06T20:17:58.744425+0000","last_became_active":"2023-07-06T20:17:28.568948+0000","last_became_peered":"2023-07-06T20:17:28.568948+0000","last_unstale":"2023-07-06T20:17:58.744425+0000","last_undegraded":"2023-07-06T20:17:58.744425+0000","last_fullsized":"2023-07-06T20:17:58.744425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:47:14.103672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744725+0000","last_change":"2023-07-06T20:17:28.568829+0000","last_active":"2023-07-06T20:17:58.744725+0000","last_peered":"2023-07-06T20:17:58.744725+0000","last_clean":"2023-07-06T20:17:58.744725+0000","last_became_active":"2023-07-06T20:17:28.568511+0000","last_became_peered":"2023-07-06T20:17:28.568511+0000","last_unstale":"2023-07-06T20:17:58.744725+0000","last_undegraded":"2023-07-06T20:17:58.744725+0000","last_fullsized":"2023-07-06T20:17:58.744725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:13:19.422184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.863188+0000","last_change":"2023-07-06T20:17:58.863188+0000","last_active":"2023-07-06T20:17:58.863188+0000","last_peered":"2023-07-06T20:17:58.863188+0000","last_clean":"2023-07-06T20:17:58.863188+0000","last_became_active":"2023-07-06T20:17:58.862743+0000","last_became_peered":"2023-07-06T20:17:58.862743+0000","last_unstale":"2023-07-06T20:17:58.863188+0000","last_undegraded":"2023-07-06T20:17:58.863188+0000","last_fullsized":"2023-07-06T20:17:58.863188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:08:47.107647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.741447+0000","last_change":"2023-07-06T20:17:28.572264+0000","last_active":"2023-07-06T20:17:58.741447+0000","last_peered":"2023-07-06T20:17:58.741447+0000","last_clean":"2023-07-06T20:17:58.741447+0000","last_became_active":"2023-07-06T20:17:28.571914+0000","last_became_peered":"2023-07-06T20:17:28.571914+0000","last_unstale":"2023-07-06T20:17:58.741447+0000","last_undegraded":"2023-07-06T20:17:58.741447+0000","last_fullsized":"2023-07-06T20:17:58.741447+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:54:59.445149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744193+0000","last_change":"2023-07-06T20:17:28.581374+0000","last_active":"2023-07-06T20:17:58.744193+0000","last_peered":"2023-07-06T20:17:58.744193+0000","last_clean":"2023-07-06T20:17:58.744193+0000","last_became_active":"2023-07-06T20:17:28.580995+0000","last_became_peered":"2023-07-06T20:17:28.580995+0000","last_unstale":"2023-07-06T20:17:58.744193+0000","last_undegraded":"2023-07-06T20:17:58.744193+0000","last_fullsized":"2023-07-06T20:17:58.744193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:55:00.813142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.740889+0000","last_change":"2023-07-06T20:17:28.578926+0000","last_active":"2023-07-06T20:17:58.740889+0000","last_peered":"2023-07-06T20:17:58.740889+0000","last_clean":"2023-07-06T20:17:58.740889+0000","last_became_active":"2023-07-06T20:17:28.578620+0000","last_became_peered":"2023-07-06T20:17:28.578620+0000","last_unstale":"2023-07-06T20:17:58.740889+0000","last_undegraded":"2023-07-06T20:17:58.740889+0000","last_fullsized":"2023-07-06T20:17:58.740889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:02:08.450840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.864937+0000","last_change":"2023-07-06T20:17:58.864937+0000","last_active":"2023-07-06T20:17:58.864937+0000","last_peered":"2023-07-06T20:17:58.864937+0000","last_clean":"2023-07-06T20:17:58.864937+0000","last_became_active":"2023-07-06T20:17:58.864457+0000","last_became_peered":"2023-07-06T20:17:58.864457+0000","last_unstale":"2023-07-06T20:17:58.864937+0000","last_undegraded":"2023-07-06T20:17:58.864937+0000","last_fullsized":"2023-07-06T20:17:58.864937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:13:23.476623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.741781+0000","last_change":"2023-07-06T20:17:28.567965+0000","last_active":"2023-07-06T20:17:58.741781+0000","last_peered":"2023-07-06T20:17:58.741781+0000","last_clean":"2023-07-06T20:17:58.741781+0000","last_became_active":"2023-07-06T20:17:28.567618+0000","last_became_peered":"2023-07-06T20:17:28.567618+0000","last_unstale":"2023-07-06T20:17:58.741781+0000","last_undegraded":"2023-07-06T20:17:58.741781+0000","last_fullsized":"2023-07-06T20:17:58.741781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:30:26.630851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862035+0000","last_change":"2023-07-06T20:17:28.571934+0000","last_active":"2023-07-06T20:17:58.862035+0000","last_peered":"2023-07-06T20:17:58.862035+0000","last_clean":"2023-07-06T20:17:58.862035+0000","last_became_active":"2023-07-06T20:17:28.571472+0000","last_became_peered":"2023-07-06T20:17:28.571472+0000","last_unstale":"2023-07-06T20:17:58.862035+0000","last_undegraded":"2023-07-06T20:17:58.862035+0000","last_fullsized":"2023-07-06T20:17:58.862035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:29.285045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861417+0000","last_change":"2023-07-06T20:17:28.571904+0000","last_active":"2023-07-06T20:17:58.861417+0000","last_peered":"2023-07-06T20:17:58.861417+0000","last_clean":"2023-07-06T20:17:58.861417+0000","last_became_active":"2023-07-06T20:17:28.571375+0000","last_became_peered":"2023-07-06T20:17:28.571375+0000","last_unstale":"2023-07-06T20:17:58.861417+0000","last_undegraded":"2023-07-06T20:17:58.861417+0000","last_fullsized":"2023-07-06T20:17:58.861417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:33:16.020700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861467+0000","last_change":"2023-07-06T20:17:28.569139+0000","last_active":"2023-07-06T20:17:58.861467+0000","last_peered":"2023-07-06T20:17:58.861467+0000","last_clean":"2023-07-06T20:17:58.861467+0000","last_became_active":"2023-07-06T20:17:28.568828+0000","last_became_peered":"2023-07-06T20:17:28.568828+0000","last_unstale":"2023-07-06T20:17:58.861467+0000","last_undegraded":"2023-07-06T20:17:58.861467+0000","last_fullsized":"2023-07-06T20:17:58.861467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:41:29.170094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746782+0000","last_change":"2023-07-06T20:17:28.562352+0000","last_active":"2023-07-06T20:17:58.746782+0000","last_peered":"2023-07-06T20:17:58.746782+0000","last_clean":"2023-07-06T20:17:58.746782+0000","last_became_active":"2023-07-06T20:17:28.562128+0000","last_became_peered":"2023-07-06T20:17:28.562128+0000","last_unstale":"2023-07-06T20:17:58.746782+0000","last_undegraded":"2023-07-06T20:17:58.746782+0000","last_fullsized":"2023-07-06T20:17:58.746782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:47.370429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.740766+0000","last_change":"2023-07-06T20:17:28.578940+0000","last_active":"2023-07-06T20:17:58.740766+0000","last_peered":"2023-07-06T20:17:58.740766+0000","last_clean":"2023-07-06T20:17:58.740766+0000","last_became_active":"2023-07-06T20:17:28.578660+0000","last_became_peered":"2023-07-06T20:17:28.578660+0000","last_unstale":"2023-07-06T20:17:58.740766+0000","last_undegraded":"2023-07-06T20:17:58.740766+0000","last_fullsized":"2023-07-06T20:17:58.740766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:52:29.675150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743663+0000","last_change":"2023-07-06T20:17:28.573964+0000","last_active":"2023-07-06T20:17:58.743663+0000","last_peered":"2023-07-06T20:17:58.743663+0000","last_clean":"2023-07-06T20:17:58.743663+0000","last_became_active":"2023-07-06T20:17:28.573618+0000","last_became_peered":"2023-07-06T20:17:28.573618+0000","last_unstale":"2023-07-06T20:17:58.743663+0000","last_undegraded":"2023-07-06T20:17:58.743663+0000","last_fullsized":"2023-07-06T20:17:58.743663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:55:42.131166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62821,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":291,"num_read_kb":211,"num_write":150,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":188463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449111,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.60499999999999998}]},{"osd":2,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.92800000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.95599999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":9,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.0329999999999999}]},{"osd":10,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.97099999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678042,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1622016,"data_stored":1094238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.375},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.47799999999999998}]},{"osd":2,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.874}]},{"osd":4,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.442},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.496}]},{"osd":6,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.627}]},{"osd":9,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":5.4219999999999997,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.65200000000000002}]},{"osd":11,"last update":"Thu Jul 6 20:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.70499999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841566,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":444118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.505,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":5.7919999999999998,"5min":5.7919999999999998,"15min":5.7919999999999998},"last":0.33200000000000002}]},{"osd":1,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71199999999999997,"15min":0.71699999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.93799999999999994,"5min":1.153,"15min":1.153},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90400000000000003,"5min":1.274,"15min":1.274},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":0.73899999999999999,"15min":0.67300000000000004},"min":{"1min":0.36499999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":16.628,"5min":16.628,"15min":16.628},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.76200000000000001,"15min":0.72399999999999998},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":11.332000000000001,"5min":11.332000000000001,"15min":11.332000000000001},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.46600000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.47599999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.83399999999999996,"15min":0.84199999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.82299999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.79100000000000004,"15min":0.79400000000000004},"min":{"1min":0.51100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.81100000000000005,"15min":0.83699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0880000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.44400000000000001}]},{"osd":7,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.879,"15min":0.89100000000000001},"min":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.90200000000000002,"15min":0.91600000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.71699999999999997}]},{"osd":8,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.86899999999999999,"15min":0.88700000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97499999999999998,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.504},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89700000000000002,"15min":0.90300000000000002},"min":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":0.73499999999999999}]},{"osd":10,"last update":"Thu Jul 6 20:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.76600000000000001}]}]},{"osd":8,"up_from":58,"seq":249108103203,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":443645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77200000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.64400000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.442,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81000000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44500000000000001,"15min":0.443},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.61099999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76900000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67300000000000004,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82199999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84599999999999997,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.754},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.46000000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.83099999999999996,"5min":0.87,"15min":0.87},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80500000000000005,"5min":0.878,"15min":0.878},"last":0.623},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59599999999999997,"15min":0.59199999999999997},"min":{"1min":0.316,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.877,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71199999999999997,"15min":0.71099999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92200000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.96299999999999997,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.88600000000000001,"15min":0.91200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.014,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.90800000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.63600000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.77200000000000002,"15min":0.75800000000000001},"min":{"1min":0.59799999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.82699999999999996}]},{"osd":10,"last update":"Thu Jul 6 20:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.86799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332136,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1536000,"data_stored":1036267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51300000000000001,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85299999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.38100000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.41799999999999998,"15min":0.40899999999999997},"min":{"1min":0.29499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77900000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.73999999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83799999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.57799999999999996,"15min":0.56399999999999995},"min":{"1min":0.54000000000000004,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68700000000000006,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85099999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.39900000000000002}]},{"osd":5,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84699999999999998,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.498,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.67800000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.53400000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.54900000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88300000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.72399999999999998}]},{"osd":8,"last update":"Thu Jul 6 20:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.81000000000000005}]},{"osd":9,"last update":"Thu Jul 6 20:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.56000000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96999999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.58199999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.84999999999999998}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.871},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.80000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":443513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.495,"15min":0.47499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jul 6 20:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.42999999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98199999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.65300000000000002},"min":{"1min":0.5,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95399999999999996,"5min":1.089,"15min":1.089},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jul 6 20:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68100000000000005,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.503}]},{"osd":3,"last update":"Thu Jul 6 20:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91300000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.55200000000000005,"5min":0.435,"15min":0.435},"max":{"1min":1.002,"5min":1.254,"15min":1.254},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71099999999999997,"15min":0.70599999999999996},"min":{"1min":0.51100000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0169999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.754,"15min":0.76800000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97999999999999998,"5min":1.393,"15min":1.393},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0569999999999999,"5min":1.131,"15min":1.131},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Jul 6 20:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.78500000000000003,"15min":0.77500000000000002},"min":{"1min":0.60699999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.78800000000000003},"min":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.748}]},{"osd":8,"last update":"Thu Jul 6 20:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.83599999999999997,"15min":0.85099999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.036,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96599999999999997,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.78200000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.56299999999999994,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.89000000000000001,"15min":0.90200000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.76500000000000001}]},{"osd":10,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.75800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":501939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.54000000000000004,"15min":0.48799999999999999},"min":{"1min":0.34799999999999998,"5min":0.248,"15min":0.248},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.52500000000000002},"min":{"1min":0.36199999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.82299999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jul 6 20:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.52900000000000003,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.95699999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.53700000000000003,"15min":0.501},"min":{"1min":0.43099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.871,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.753}]},{"osd":3,"last update":"Thu Jul 6 20:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.55100000000000005},"min":{"1min":0.38500000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.42899999999999999,"15min":0.39300000000000002},"min":{"1min":0.33800000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.46999999999999997,"15min":0.42599999999999999},"min":{"1min":0.33500000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Jul 6 20:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.52700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0169999999999999,"5min":1.042,"15min":1.042},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67500000000000004,"15min":0.65500000000000003},"min":{"1min":0.46300000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.55500000000000005}]},{"osd":7,"last update":"Thu Jul 6 20:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.624,"15min":0.59199999999999997},"min":{"1min":0.42399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.59599999999999997,"15min":0.55500000000000005},"min":{"1min":0.5,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.5}]},{"osd":8,"last update":"Thu Jul 6 20:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.66900000000000004,"15min":0.65500000000000003},"min":{"1min":0.49199999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.73899999999999999,"15min":0.72399999999999998},"min":{"1min":0.57899999999999996,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.80500000000000005}]},{"osd":9,"last update":"Thu Jul 6 20:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.75700000000000001,"15min":0.74299999999999999},"min":{"1min":0.57699999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.623},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77200000000000002,"15min":0.76600000000000001},"min":{"1min":0.626,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.73799999999999999}]},{"osd":10,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.83399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":444778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.317,"15min":0.317},"max":{"1min":4.1130000000000004,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.628,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.6400000000000001,"5min":4.1379999999999999,"15min":4.1379999999999999},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61399999999999999,"15min":0.58199999999999996},"min":{"1min":0.45100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85399999999999998,"5min":1.085,"15min":1.085},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.622,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":1.258,"15min":1.258},"last":0.84799999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60799999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.73799999999999999},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0049999999999999,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":0.752}]},{"osd":5,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65400000000000003,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0389999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0069999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.93200000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63300000000000001,"15min":0.64000000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":0.69399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81100000000000005,"5min":1.319,"15min":1.319},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.03,"5min":1.131,"15min":1.131},"last":0.505},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79700000000000004,"15min":0.79600000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.179,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.84399999999999997}]},{"osd":8,"last update":"Thu Jul 6 20:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76600000000000001,"15min":0.77800000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.004,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.80600000000000005,"15min":0.82199999999999995},"min":{"1min":0.51000000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94899999999999995,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.88200000000000001}]},{"osd":9,"last update":"Thu Jul 6 20:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.81599999999999995,"15min":0.82999999999999996},"min":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.014,"5min":1.161,"15min":1.161},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78100000000000003,"15min":0.79300000000000004},"min":{"1min":0.45100000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0409999999999999,"5min":1.121,"15min":1.121},"last":0.83299999999999996}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.81699999999999995}]},{"osd":11,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.004}]}]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":503407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55800000000000005,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.88700000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.56000000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69399999999999995,"5min":0.876,"15min":0.876},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60899999999999999,"15min":0.623},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83599999999999997,"5min":0.995,"15min":0.995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.627,"15min":0.627},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79800000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jul 6 20:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jul 6 20:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47199999999999998,"15min":0.45200000000000001},"min":{"1min":0.33500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.625},{"interface":"front","average":{"1min":0.503,"5min":0.47299999999999998,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.45500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86499999999999999,"5min":1.056,"15min":1.056},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67500000000000004,"15min":0.65000000000000002},"min":{"1min":0.53200000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86199999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.86399999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.624},"min":{"1min":0.435,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81899999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59799999999999998,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70999999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.75,"15min":0.754},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.93899999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.78700000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.93799999999999994,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.89800000000000002}]},{"osd":10,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.88500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":443830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.40699999999999997,"15min":0.38300000000000001},"min":{"1min":0.29099999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.435,"15min":0.40300000000000002},"min":{"1min":0.32900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65300000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.64200000000000002,"15min":0.57999999999999996},"min":{"1min":0.51000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.871,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.623},{"interface":"front","average":{"1min":0.626,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86199999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jul 6 20:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.621,"15min":0.58899999999999997},"min":{"1min":0.496,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81899999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.66700000000000004,"15min":0.622},"min":{"1min":0.48199999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.875,"5min":3.6720000000000002,"15min":3.6720000000000002},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.73199999999999998,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":6.9420000000000002,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.58699999999999997},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.65000000000000002,"15min":0.624},"min":{"1min":0.56899999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.46800000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92400000000000004,"5min":1.052,"15min":1.052},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59099999999999997,"15min":0.56499999999999995},"min":{"1min":0.496,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.47399999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.69199999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77800000000000002}]},{"osd":8,"last update":"Thu Jul 6 20:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.58299999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":0.74099999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.034,"5min":1.135,"15min":1.135},"last":0.50900000000000001}]},{"osd":10,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.76500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":444286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.55200000000000005,"15min":0.501},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":6.1539999999999999,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.51900000000000002,"15min":0.49299999999999999},"min":{"1min":0.45700000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.35799999999999998},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58399999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.376},{"interface":"front","average":{"1min":0.44,"5min":0.39600000000000002,"15min":0.35799999999999998},"min":{"1min":0.311,"5min":0.219,"15min":0.219},"max":{"1min":0.55900000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.47199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.44400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86699999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.53600000000000003},"min":{"1min":0.52200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.099,"5min":5.593,"15min":5.593},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.123,"5min":0.71699999999999997,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.378,"15min":0.378},"max":{"1min":9.7959999999999994,"5min":9.7959999999999994,"15min":9.7959999999999994},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jul 6 20:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.55600000000000005},"min":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89800000000000002,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.749}]},{"osd":6,"last update":"Thu Jul 6 20:17:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70299999999999996,"5min":1.109,"15min":1.109},"last":0.502},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.69099999999999995},"min":{"1min":0.54900000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89900000000000002,"5min":1.163,"15min":1.163},"last":0.65200000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.35499999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.97699999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.372,"5min":0.314,"15min":0.314},"max":{"1min":0.91200000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.70899999999999996}]},{"osd":8,"last update":"Thu Jul 6 20:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.47299999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.1890000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93700000000000006,"5min":1.194,"15min":1.194},"last":0.84099999999999997}]},{"osd":9,"last update":"Thu Jul 6 20:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.50700000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.94099999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.2470000000000001,"5min":0.92400000000000004,"15min":0.87},"min":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":8.3369999999999997,"5min":8.3369999999999997,"15min":8.3369999999999997},"last":0.82299999999999995}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.77400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1035278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80600000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69399999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.79200000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78100000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40300000000000002,"15min":0.39800000000000002},"min":{"1min":0.29299999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.628},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.436,"15min":0.42999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76200000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83299999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.753},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Jul 6 20:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85999999999999999,"5min":0.873,"15min":0.873},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64900000000000002,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.85199999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70099999999999996,"15min":0.70399999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.81999999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.38300000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.59899999999999998}]},{"osd":9,"last update":"Thu Jul 6 20:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76500000000000001,"15min":0.76300000000000001},"min":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.91200000000000003,"5min":1.012,"15min":1.012},"last":0.71299999999999997}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:18:21.487 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph pg dump --format=json 2023-07-06T20:18:22.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:21 smithi098 ceph-mon[128968]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:18:22.238 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:22.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:21 smithi069 ceph-mon[126399]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:18:22.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:21 smithi156 ceph-mon[129280]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:18:23.072 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:22 smithi069 ceph-mon[126399]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-06T20:18:23.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:22 smithi098 ceph-mon[128968]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-06T20:18:23.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:22 smithi156 ceph-mon[129280]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-06T20:18:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.746+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4254877672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3980729e0 msgr2=0x7fe398072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.746+0000 7fe39c9b3700 1 --2- 172.21.15.69:0/4254877672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3980729e0 0x7fe398072db0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe388009990 tx=0x7fe388009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.747+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4254877672 shutdown_connections 2023-07-06T20:18:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.747+0000 7fe39c9b3700 1 --2- 172.21.15.69:0/4254877672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe39810c510 0x7fe39810e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.747+0000 7fe39c9b3700 1 --2- 172.21.15.69:0/4254877672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3980729e0 0x7fe398072db0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.747+0000 7fe39c9b3700 1 --2- 172.21.15.69:0/4254877672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3980732f0 0x7fe39810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.747+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4254877672 >> 172.21.15.69:0/4254877672 conn(0x7fe3980fb790 msgr2=0x7fe3980fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.747+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4254877672 shutdown_connections 2023-07-06T20:18:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.747+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4254877672 wait complete. 2023-07-06T20:18:23.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.748+0000 7fe39c9b3700 1 Processor -- start 2023-07-06T20:18:23.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.748+0000 7fe39c9b3700 1 -- start start 2023-07-06T20:18:23.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe39c9b3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3980729e0 0x7fe398197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:23.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe39c9b3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3980732f0 0x7fe398198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:23.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe39c9b3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe39810c510 0x7fe39819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:23.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe39c9b3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39819d040 con 0x7fe39810c510 2023-07-06T20:18:23.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe39c9b3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39819d1b0 con 0x7fe3980729e0 2023-07-06T20:18:23.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe39c9b3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39819d370 con 0x7fe3980732f0 2023-07-06T20:18:23.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe39659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3980729e0 0x7fe398197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:23.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe396d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe39810c510 0x7fe39819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:23.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe395d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3980732f0 0x7fe398198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:23.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe39659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3980729e0 0x7fe398197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:42144/0 (socket says 172.21.15.69:42144) 2023-07-06T20:18:23.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe39659c700 1 -- 172.21.15.69:0/4247126229 learned_addr learned my addr 172.21.15.69:0/4247126229 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:23.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe395d9b700 1 -- 172.21.15.69:0/4247126229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3980729e0 msgr2=0x7fe398197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:23.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe395d9b700 1 --2- 172.21.15.69:0/4247126229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3980729e0 0x7fe398197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:23.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe395d9b700 1 -- 172.21.15.69:0/4247126229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe39810c510 msgr2=0x7fe39819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:23.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe395d9b700 1 --2- 172.21.15.69:0/4247126229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe39810c510 0x7fe39819c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:23.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.749+0000 7fe395d9b700 1 -- 172.21.15.69:0/4247126229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3880095f0 con 0x7fe3980732f0 2023-07-06T20:18:23.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.750+0000 7fe395d9b700 1 --2- 172.21.15.69:0/4247126229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3980732f0 0x7fe398198530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00e9f0 tx=0x7fe38c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:23.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.750+0000 7fe3877fe700 1 -- 172.21.15.69:0/4247126229 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c00c9c0 con 0x7fe3980732f0 2023-07-06T20:18:23.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.750+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4247126229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39819d650 con 0x7fe3980732f0 2023-07-06T20:18:23.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.750+0000 7fe3877fe700 1 -- 172.21.15.69:0/4247126229 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe38c010430 con 0x7fe3980732f0 2023-07-06T20:18:23.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.750+0000 7fe3877fe700 1 -- 172.21.15.69:0/4247126229 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c018680 con 0x7fe3980732f0 2023-07-06T20:18:23.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.750+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4247126229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3981a4e70 con 0x7fe3980732f0 2023-07-06T20:18:23.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.751+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4247126229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39804e340 con 0x7fe3980732f0 2023-07-06T20:18:23.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.753+0000 7fe3877fe700 1 -- 172.21.15.69:0/4247126229 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe38c010860 con 0x7fe3980732f0 2023-07-06T20:18:23.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.755+0000 7fe3877fe700 1 --2- 172.21.15.69:0/4247126229 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe3800a1260 0x7fe3800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.755+0000 7fe3877fe700 1 -- 172.21.15.69:0/4247126229 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe38c014070 con 0x7fe3980732f0 2023-07-06T20:18:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.755+0000 7fe39659c700 1 --2- 172.21.15.69:0/4247126229 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe3800a1260 0x7fe3800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.755+0000 7fe39659c700 1 --2- 172.21.15.69:0/4247126229 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe3800a1260 0x7fe3800a3710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe388006010 tx=0x7fe38800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:23.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.757+0000 7fe3877fe700 1 -- 172.21.15.69:0/4247126229 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe38c096300 con 0x7fe3980732f0 2023-07-06T20:18:23.953 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.951+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4247126229 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3980fd400 con 0x7fe3800a1260 2023-07-06T20:18:23.964 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.962+0000 7fe3877fe700 1 -- 172.21.15.69:0/4247126229 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339836 (secure 0 0 0) 0x7fe3980fd400 con 0x7fe3800a1260 2023-07-06T20:18:23.967 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:23.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.966+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4247126229 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe3800a1260 msgr2=0x7fe3800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:23.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.966+0000 7fe39c9b3700 1 --2- 172.21.15.69:0/4247126229 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe3800a1260 0x7fe3800a3710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe388006010 tx=0x7fe38800b260 comp rx=0 tx=0).stop 2023-07-06T20:18:23.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.966+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4247126229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3980732f0 msgr2=0x7fe398198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:23.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.966+0000 7fe39c9b3700 1 --2- 172.21.15.69:0/4247126229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3980732f0 0x7fe398198530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00e9f0 tx=0x7fe38c00ed00 comp rx=0 tx=0).stop 2023-07-06T20:18:23.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.967+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4247126229 shutdown_connections 2023-07-06T20:18:23.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.967+0000 7fe39c9b3700 1 --2- 172.21.15.69:0/4247126229 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe3800a1260 0x7fe3800a3710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:23.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.967+0000 7fe39c9b3700 1 --2- 172.21.15.69:0/4247126229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe39810c510 0x7fe39819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:23.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.967+0000 7fe39c9b3700 1 --2- 172.21.15.69:0/4247126229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3980729e0 0x7fe398197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:23.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.967+0000 7fe39c9b3700 1 --2- 172.21.15.69:0/4247126229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3980732f0 0x7fe398198530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:23.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.967+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4247126229 >> 172.21.15.69:0/4247126229 conn(0x7fe3980fb790 msgr2=0x7fe39810e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:23.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.967+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4247126229 shutdown_connections 2023-07-06T20:18:23.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:23.967+0000 7fe39c9b3700 1 -- 172.21.15.69:0/4247126229 wait complete. 2023-07-06T20:18:23.971 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-07-06T20:18:24.413 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2023-07-06T20:18:23.533316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653435,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":442,"num_read_kb":330,"num_write":333,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336388,"kb_used_data":12964,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605436,"statfs":{"total":1151940427776,"available":1151595966464,"internally_reserved":0,"allocated":13275136,"data_stored":7279293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003156"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744044+0000","last_change":"2023-07-06T20:17:28.575520+0000","last_active":"2023-07-06T20:17:58.744044+0000","last_peered":"2023-07-06T20:17:58.744044+0000","last_clean":"2023-07-06T20:17:58.744044+0000","last_became_active":"2023-07-06T20:17:28.575209+0000","last_became_peered":"2023-07-06T20:17:28.575209+0000","last_unstale":"2023-07-06T20:17:58.744044+0000","last_undegraded":"2023-07-06T20:17:58.744044+0000","last_fullsized":"2023-07-06T20:17:58.744044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:52:25.496969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744067+0000","last_change":"2023-07-06T20:17:28.572547+0000","last_active":"2023-07-06T20:17:58.744067+0000","last_peered":"2023-07-06T20:17:58.744067+0000","last_clean":"2023-07-06T20:17:58.744067+0000","last_became_active":"2023-07-06T20:17:28.572150+0000","last_became_peered":"2023-07-06T20:17:28.572150+0000","last_unstale":"2023-07-06T20:17:58.744067+0000","last_undegraded":"2023-07-06T20:17:58.744067+0000","last_fullsized":"2023-07-06T20:17:58.744067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:12:53.896783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744407+0000","last_change":"2023-07-06T20:17:28.574926+0000","last_active":"2023-07-06T20:17:58.744407+0000","last_peered":"2023-07-06T20:17:58.744407+0000","last_clean":"2023-07-06T20:17:58.744407+0000","last_became_active":"2023-07-06T20:17:28.574602+0000","last_became_peered":"2023-07-06T20:17:28.574602+0000","last_unstale":"2023-07-06T20:17:58.744407+0000","last_undegraded":"2023-07-06T20:17:58.744407+0000","last_fullsized":"2023-07-06T20:17:58.744407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:36:52.873371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862515+0000","last_change":"2023-07-06T20:17:28.571533+0000","last_active":"2023-07-06T20:17:58.862515+0000","last_peered":"2023-07-06T20:17:58.862515+0000","last_clean":"2023-07-06T20:17:58.862515+0000","last_became_active":"2023-07-06T20:17:28.571099+0000","last_became_peered":"2023-07-06T20:17:28.571099+0000","last_unstale":"2023-07-06T20:17:58.862515+0000","last_undegraded":"2023-07-06T20:17:58.862515+0000","last_fullsized":"2023-07-06T20:17:58.862515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:34:07.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746458+0000","last_change":"2023-07-06T20:17:28.572100+0000","last_active":"2023-07-06T20:17:58.746458+0000","last_peered":"2023-07-06T20:17:58.746458+0000","last_clean":"2023-07-06T20:17:58.746458+0000","last_became_active":"2023-07-06T20:17:28.571772+0000","last_became_peered":"2023-07-06T20:17:28.571772+0000","last_unstale":"2023-07-06T20:17:58.746458+0000","last_undegraded":"2023-07-06T20:17:58.746458+0000","last_fullsized":"2023-07-06T20:17:58.746458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:50.574562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744230+0000","last_change":"2023-07-06T20:17:28.577816+0000","last_active":"2023-07-06T20:17:58.744230+0000","last_peered":"2023-07-06T20:17:58.744230+0000","last_clean":"2023-07-06T20:17:58.744230+0000","last_became_active":"2023-07-06T20:17:28.577382+0000","last_became_peered":"2023-07-06T20:17:28.577382+0000","last_unstale":"2023-07-06T20:17:58.744230+0000","last_undegraded":"2023-07-06T20:17:58.744230+0000","last_fullsized":"2023-07-06T20:17:58.744230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:46:12.848233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743783+0000","last_change":"2023-07-06T20:17:28.570415+0000","last_active":"2023-07-06T20:17:58.743783+0000","last_peered":"2023-07-06T20:17:58.743783+0000","last_clean":"2023-07-06T20:17:58.743783+0000","last_became_active":"2023-07-06T20:17:28.570091+0000","last_became_peered":"2023-07-06T20:17:28.570091+0000","last_unstale":"2023-07-06T20:17:58.743783+0000","last_undegraded":"2023-07-06T20:17:58.743783+0000","last_fullsized":"2023-07-06T20:17:58.743783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:19:38.692740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861426+0000","last_change":"2023-07-06T20:17:28.568370+0000","last_active":"2023-07-06T20:17:58.861426+0000","last_peered":"2023-07-06T20:17:58.861426+0000","last_clean":"2023-07-06T20:17:58.861426+0000","last_became_active":"2023-07-06T20:17:28.568032+0000","last_became_peered":"2023-07-06T20:17:28.568032+0000","last_unstale":"2023-07-06T20:17:58.861426+0000","last_undegraded":"2023-07-06T20:17:58.861426+0000","last_fullsized":"2023-07-06T20:17:58.861426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:56:58.507364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747148+0000","last_change":"2023-07-06T20:17:28.563386+0000","last_active":"2023-07-06T20:17:58.747148+0000","last_peered":"2023-07-06T20:17:58.747148+0000","last_clean":"2023-07-06T20:17:58.747148+0000","last_became_active":"2023-07-06T20:17:28.562864+0000","last_became_peered":"2023-07-06T20:17:28.562864+0000","last_unstale":"2023-07-06T20:17:58.747148+0000","last_undegraded":"2023-07-06T20:17:58.747148+0000","last_fullsized":"2023-07-06T20:17:58.747148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:38:18.267804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743951+0000","last_change":"2023-07-06T20:17:28.578016+0000","last_active":"2023-07-06T20:17:58.743951+0000","last_peered":"2023-07-06T20:17:58.743951+0000","last_clean":"2023-07-06T20:17:58.743951+0000","last_became_active":"2023-07-06T20:17:28.577305+0000","last_became_peered":"2023-07-06T20:17:28.577305+0000","last_unstale":"2023-07-06T20:17:58.743951+0000","last_undegraded":"2023-07-06T20:17:58.743951+0000","last_fullsized":"2023-07-06T20:17:58.743951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:27:34.666203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744374+0000","last_change":"2023-07-06T20:17:28.569683+0000","last_active":"2023-07-06T20:17:58.744374+0000","last_peered":"2023-07-06T20:17:58.744374+0000","last_clean":"2023-07-06T20:17:58.744374+0000","last_became_active":"2023-07-06T20:17:28.569070+0000","last_became_peered":"2023-07-06T20:17:28.569070+0000","last_unstale":"2023-07-06T20:17:58.744374+0000","last_undegraded":"2023-07-06T20:17:58.744374+0000","last_fullsized":"2023-07-06T20:17:58.744374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:38:56.624764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862482+0000","last_change":"2023-07-06T20:17:28.579463+0000","last_active":"2023-07-06T20:17:58.862482+0000","last_peered":"2023-07-06T20:17:58.862482+0000","last_clean":"2023-07-06T20:17:58.862482+0000","last_became_active":"2023-07-06T20:17:28.579141+0000","last_became_peered":"2023-07-06T20:17:28.579141+0000","last_unstale":"2023-07-06T20:17:58.862482+0000","last_undegraded":"2023-07-06T20:17:58.862482+0000","last_fullsized":"2023-07-06T20:17:58.862482+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:11.840039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.864016+0000","last_change":"2023-07-06T20:17:58.864016+0000","last_active":"2023-07-06T20:17:58.864016+0000","last_peered":"2023-07-06T20:17:58.864016+0000","last_clean":"2023-07-06T20:17:58.864016+0000","last_became_active":"2023-07-06T20:17:58.863583+0000","last_became_peered":"2023-07-06T20:17:58.863583+0000","last_unstale":"2023-07-06T20:17:58.864016+0000","last_undegraded":"2023-07-06T20:17:58.864016+0000","last_fullsized":"2023-07-06T20:17:58.864016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:18:13.895396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743986+0000","last_change":"2023-07-06T20:17:28.572197+0000","last_active":"2023-07-06T20:17:58.743986+0000","last_peered":"2023-07-06T20:17:58.743986+0000","last_clean":"2023-07-06T20:17:58.743986+0000","last_became_active":"2023-07-06T20:17:28.571816+0000","last_became_peered":"2023-07-06T20:17:28.571816+0000","last_unstale":"2023-07-06T20:17:58.743986+0000","last_undegraded":"2023-07-06T20:17:58.743986+0000","last_fullsized":"2023-07-06T20:17:58.743986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:03:41.788066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746966+0000","last_change":"2023-07-06T20:17:58.746966+0000","last_active":"2023-07-06T20:17:58.746966+0000","last_peered":"2023-07-06T20:17:58.746966+0000","last_clean":"2023-07-06T20:17:58.746966+0000","last_became_active":"2023-07-06T20:17:58.746598+0000","last_became_peered":"2023-07-06T20:17:58.746598+0000","last_unstale":"2023-07-06T20:17:58.746966+0000","last_undegraded":"2023-07-06T20:17:58.746966+0000","last_fullsized":"2023-07-06T20:17:58.746966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:46:09.676858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747319+0000","last_change":"2023-07-06T20:17:28.573552+0000","last_active":"2023-07-06T20:17:58.747319+0000","last_peered":"2023-07-06T20:17:58.747319+0000","last_clean":"2023-07-06T20:17:58.747319+0000","last_became_active":"2023-07-06T20:17:28.573253+0000","last_became_peered":"2023-07-06T20:17:28.573253+0000","last_unstale":"2023-07-06T20:17:58.747319+0000","last_undegraded":"2023-07-06T20:17:58.747319+0000","last_fullsized":"2023-07-06T20:17:58.747319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:41:53.685379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744159+0000","last_change":"2023-07-06T20:17:28.572425+0000","last_active":"2023-07-06T20:17:58.744159+0000","last_peered":"2023-07-06T20:17:58.744159+0000","last_clean":"2023-07-06T20:17:58.744159+0000","last_became_active":"2023-07-06T20:17:28.572104+0000","last_became_peered":"2023-07-06T20:17:28.572104+0000","last_unstale":"2023-07-06T20:17:58.744159+0000","last_undegraded":"2023-07-06T20:17:58.744159+0000","last_fullsized":"2023-07-06T20:17:58.744159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:54:48.624955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862859+0000","last_change":"2023-07-06T20:17:28.573814+0000","last_active":"2023-07-06T20:17:58.862859+0000","last_peered":"2023-07-06T20:17:58.862859+0000","last_clean":"2023-07-06T20:17:58.862859+0000","last_became_active":"2023-07-06T20:17:28.573470+0000","last_became_peered":"2023-07-06T20:17:28.573470+0000","last_unstale":"2023-07-06T20:17:58.862859+0000","last_undegraded":"2023-07-06T20:17:58.862859+0000","last_fullsized":"2023-07-06T20:17:58.862859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:03:31.057892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.865802+0000","last_change":"2023-07-06T20:17:58.865802+0000","last_active":"2023-07-06T20:17:58.865802+0000","last_peered":"2023-07-06T20:17:58.865802+0000","last_clean":"2023-07-06T20:17:58.865802+0000","last_became_active":"2023-07-06T20:17:58.865290+0000","last_became_peered":"2023-07-06T20:17:58.865290+0000","last_unstale":"2023-07-06T20:17:58.865802+0000","last_undegraded":"2023-07-06T20:17:58.865802+0000","last_fullsized":"2023-07-06T20:17:58.865802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:40:17.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.863042+0000","last_change":"2023-07-06T20:17:27.603771+0000","last_active":"2023-07-06T20:17:58.863042+0000","last_peered":"2023-07-06T20:17:58.863042+0000","last_clean":"2023-07-06T20:17:58.863042+0000","last_became_active":"2023-07-06T20:17:27.603404+0000","last_became_peered":"2023-07-06T20:17:27.603404+0000","last_unstale":"2023-07-06T20:17:58.863042+0000","last_undegraded":"2023-07-06T20:17:58.863042+0000","last_fullsized":"2023-07-06T20:17:58.863042+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:17:53.507946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862034+0000","last_change":"2023-07-06T20:17:28.568371+0000","last_active":"2023-07-06T20:17:58.862034+0000","last_peered":"2023-07-06T20:17:58.862034+0000","last_clean":"2023-07-06T20:17:58.862034+0000","last_became_active":"2023-07-06T20:17:28.568034+0000","last_became_peered":"2023-07-06T20:17:28.568034+0000","last_unstale":"2023-07-06T20:17:58.862034+0000","last_undegraded":"2023-07-06T20:17:58.862034+0000","last_fullsized":"2023-07-06T20:17:58.862034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:37:29.119572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862283+0000","last_change":"2023-07-06T20:17:27.609833+0000","last_active":"2023-07-06T20:17:58.862283+0000","last_peered":"2023-07-06T20:17:58.862283+0000","last_clean":"2023-07-06T20:17:58.862283+0000","last_became_active":"2023-07-06T20:17:27.609520+0000","last_became_peered":"2023-07-06T20:17:27.609520+0000","last_unstale":"2023-07-06T20:17:58.862283+0000","last_undegraded":"2023-07-06T20:17:58.862283+0000","last_fullsized":"2023-07-06T20:17:58.862283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:05:16.374974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743868+0000","last_change":"2023-07-06T20:17:28.574520+0000","last_active":"2023-07-06T20:17:58.743868+0000","last_peered":"2023-07-06T20:17:58.743868+0000","last_clean":"2023-07-06T20:17:58.743868+0000","last_became_active":"2023-07-06T20:17:28.574158+0000","last_became_peered":"2023-07-06T20:17:28.574158+0000","last_unstale":"2023-07-06T20:17:58.743868+0000","last_undegraded":"2023-07-06T20:17:58.743868+0000","last_fullsized":"2023-07-06T20:17:58.743868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:10:19.258285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747197+0000","last_change":"2023-07-06T20:17:27.601612+0000","last_active":"2023-07-06T20:17:58.747197+0000","last_peered":"2023-07-06T20:17:58.747197+0000","last_clean":"2023-07-06T20:17:58.747197+0000","last_became_active":"2023-07-06T20:17:27.601313+0000","last_became_peered":"2023-07-06T20:17:27.601313+0000","last_unstale":"2023-07-06T20:17:58.747197+0000","last_undegraded":"2023-07-06T20:17:58.747197+0000","last_fullsized":"2023-07-06T20:17:58.747197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:40:08.926500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746884+0000","last_change":"2023-07-06T20:17:28.566389+0000","last_active":"2023-07-06T20:17:58.746884+0000","last_peered":"2023-07-06T20:17:58.746884+0000","last_clean":"2023-07-06T20:17:58.746884+0000","last_became_active":"2023-07-06T20:17:28.566094+0000","last_became_peered":"2023-07-06T20:17:28.566094+0000","last_unstale":"2023-07-06T20:17:58.746884+0000","last_undegraded":"2023-07-06T20:17:58.746884+0000","last_fullsized":"2023-07-06T20:17:58.746884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:15:29.583885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746810+0000","last_change":"2023-07-06T20:17:27.599893+0000","last_active":"2023-07-06T20:17:58.746810+0000","last_peered":"2023-07-06T20:17:58.746810+0000","last_clean":"2023-07-06T20:17:58.746810+0000","last_became_active":"2023-07-06T20:17:27.599544+0000","last_became_peered":"2023-07-06T20:17:27.599544+0000","last_unstale":"2023-07-06T20:17:58.746810+0000","last_undegraded":"2023-07-06T20:17:58.746810+0000","last_fullsized":"2023-07-06T20:17:58.746810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:52:55.097220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861850+0000","last_change":"2023-07-06T20:17:28.575794+0000","last_active":"2023-07-06T20:17:58.861850+0000","last_peered":"2023-07-06T20:17:58.861850+0000","last_clean":"2023-07-06T20:17:58.861850+0000","last_became_active":"2023-07-06T20:17:28.575456+0000","last_became_peered":"2023-07-06T20:17:28.575456+0000","last_unstale":"2023-07-06T20:17:58.861850+0000","last_undegraded":"2023-07-06T20:17:58.861850+0000","last_fullsized":"2023-07-06T20:17:58.861850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:39:54.919014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":77,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:18:20.635461+0000","last_change":"2023-07-06T20:17:27.596351+0000","last_active":"2023-07-06T20:18:20.635461+0000","last_peered":"2023-07-06T20:18:20.635461+0000","last_clean":"2023-07-06T20:18:20.635461+0000","last_became_active":"2023-07-06T20:17:27.595982+0000","last_became_peered":"2023-07-06T20:17:27.595982+0000","last_unstale":"2023-07-06T20:18:20.635461+0000","last_undegraded":"2023-07-06T20:18:20.635461+0000","last_fullsized":"2023-07-06T20:18:20.635461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:15:37.772241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746328+0000","last_change":"2023-07-06T20:17:28.570018+0000","last_active":"2023-07-06T20:17:58.746328+0000","last_peered":"2023-07-06T20:17:58.746328+0000","last_clean":"2023-07-06T20:17:58.746328+0000","last_became_active":"2023-07-06T20:17:28.569674+0000","last_became_peered":"2023-07-06T20:17:28.569674+0000","last_unstale":"2023-07-06T20:17:58.746328+0000","last_undegraded":"2023-07-06T20:17:58.746328+0000","last_fullsized":"2023-07-06T20:17:58.746328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:24:04.467030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.866940+0000","last_change":"2023-07-06T20:17:58.866940+0000","last_active":"2023-07-06T20:17:58.866940+0000","last_peered":"2023-07-06T20:17:58.866940+0000","last_clean":"2023-07-06T20:17:58.866940+0000","last_became_active":"2023-07-06T20:17:58.863435+0000","last_became_peered":"2023-07-06T20:17:58.863435+0000","last_unstale":"2023-07-06T20:17:58.866940+0000","last_undegraded":"2023-07-06T20:17:58.866940+0000","last_fullsized":"2023-07-06T20:17:58.866940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:32:08.233817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861923+0000","last_change":"2023-07-06T20:17:28.575249+0000","last_active":"2023-07-06T20:17:58.861923+0000","last_peered":"2023-07-06T20:17:58.861923+0000","last_clean":"2023-07-06T20:17:58.861923+0000","last_became_active":"2023-07-06T20:17:28.574954+0000","last_became_peered":"2023-07-06T20:17:28.574954+0000","last_unstale":"2023-07-06T20:17:58.861923+0000","last_undegraded":"2023-07-06T20:17:58.861923+0000","last_fullsized":"2023-07-06T20:17:58.861923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:27:13.436799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.740609+0000","last_change":"2023-07-06T20:17:27.600552+0000","last_active":"2023-07-06T20:17:58.740609+0000","last_peered":"2023-07-06T20:17:58.740609+0000","last_clean":"2023-07-06T20:17:58.740609+0000","last_became_active":"2023-07-06T20:17:27.600016+0000","last_became_peered":"2023-07-06T20:17:27.600016+0000","last_unstale":"2023-07-06T20:17:58.740609+0000","last_undegraded":"2023-07-06T20:17:58.740609+0000","last_fullsized":"2023-07-06T20:17:58.740609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:11:39.398698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861595+0000","last_change":"2023-07-06T20:17:28.566364+0000","last_active":"2023-07-06T20:17:58.861595+0000","last_peered":"2023-07-06T20:17:58.861595+0000","last_clean":"2023-07-06T20:17:58.861595+0000","last_became_active":"2023-07-06T20:17:28.565474+0000","last_became_peered":"2023-07-06T20:17:28.565474+0000","last_unstale":"2023-07-06T20:17:58.861595+0000","last_undegraded":"2023-07-06T20:17:58.861595+0000","last_fullsized":"2023-07-06T20:17:58.861595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:49:42.093269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861332+0000","last_change":"2023-07-06T20:17:27.598606+0000","last_active":"2023-07-06T20:17:58.861332+0000","last_peered":"2023-07-06T20:17:58.861332+0000","last_clean":"2023-07-06T20:17:58.861332+0000","last_became_active":"2023-07-06T20:17:27.598292+0000","last_became_peered":"2023-07-06T20:17:27.598292+0000","last_unstale":"2023-07-06T20:17:58.861332+0000","last_undegraded":"2023-07-06T20:17:58.861332+0000","last_fullsized":"2023-07-06T20:17:58.861332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:42:01.805838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862700+0000","last_change":"2023-07-06T20:17:28.571246+0000","last_active":"2023-07-06T20:17:58.862700+0000","last_peered":"2023-07-06T20:17:58.862700+0000","last_clean":"2023-07-06T20:17:58.862700+0000","last_became_active":"2023-07-06T20:17:28.570846+0000","last_became_peered":"2023-07-06T20:17:28.570846+0000","last_unstale":"2023-07-06T20:17:58.862700+0000","last_undegraded":"2023-07-06T20:17:58.862700+0000","last_fullsized":"2023-07-06T20:17:58.862700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:45:47.944924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":69,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:18:18.631906+0000","last_change":"2023-07-06T20:17:27.595879+0000","last_active":"2023-07-06T20:18:18.631906+0000","last_peered":"2023-07-06T20:18:18.631906+0000","last_clean":"2023-07-06T20:18:18.631906+0000","last_became_active":"2023-07-06T20:17:27.595553+0000","last_became_peered":"2023-07-06T20:17:27.595553+0000","last_unstale":"2023-07-06T20:18:18.631906+0000","last_undegraded":"2023-07-06T20:18:18.631906+0000","last_fullsized":"2023-07-06T20:18:18.631906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:12:02.846365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747175+0000","last_change":"2023-07-06T20:17:28.566816+0000","last_active":"2023-07-06T20:17:58.747175+0000","last_peered":"2023-07-06T20:17:58.747175+0000","last_clean":"2023-07-06T20:17:58.747175+0000","last_became_active":"2023-07-06T20:17:28.566565+0000","last_became_peered":"2023-07-06T20:17:28.566565+0000","last_unstale":"2023-07-06T20:17:58.747175+0000","last_undegraded":"2023-07-06T20:17:58.747175+0000","last_fullsized":"2023-07-06T20:17:58.747175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:50:37.443167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.868853+0000","last_change":"2023-07-06T20:17:58.868853+0000","last_active":"2023-07-06T20:17:58.868853+0000","last_peered":"2023-07-06T20:17:58.868853+0000","last_clean":"2023-07-06T20:17:58.868853+0000","last_became_active":"2023-07-06T20:17:58.863304+0000","last_became_peered":"2023-07-06T20:17:58.863304+0000","last_unstale":"2023-07-06T20:17:58.868853+0000","last_undegraded":"2023-07-06T20:17:58.868853+0000","last_fullsized":"2023-07-06T20:17:58.868853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:16:21.561050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743734+0000","last_change":"2023-07-06T20:17:28.572400+0000","last_active":"2023-07-06T20:17:58.743734+0000","last_peered":"2023-07-06T20:17:58.743734+0000","last_clean":"2023-07-06T20:17:58.743734+0000","last_became_active":"2023-07-06T20:17:28.572047+0000","last_became_peered":"2023-07-06T20:17:28.572047+0000","last_unstale":"2023-07-06T20:17:58.743734+0000","last_undegraded":"2023-07-06T20:17:58.743734+0000","last_fullsized":"2023-07-06T20:17:58.743734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:46:01.868129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862013+0000","last_change":"2023-07-06T20:17:27.603224+0000","last_active":"2023-07-06T20:17:58.862013+0000","last_peered":"2023-07-06T20:17:58.862013+0000","last_clean":"2023-07-06T20:17:58.862013+0000","last_became_active":"2023-07-06T20:17:27.602864+0000","last_became_peered":"2023-07-06T20:17:27.602864+0000","last_unstale":"2023-07-06T20:17:58.862013+0000","last_undegraded":"2023-07-06T20:17:58.862013+0000","last_fullsized":"2023-07-06T20:17:58.862013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:19:13.048990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747083+0000","last_change":"2023-07-06T20:17:28.572683+0000","last_active":"2023-07-06T20:17:58.747083+0000","last_peered":"2023-07-06T20:17:58.747083+0000","last_clean":"2023-07-06T20:17:58.747083+0000","last_became_active":"2023-07-06T20:17:28.572352+0000","last_became_peered":"2023-07-06T20:17:28.572352+0000","last_unstale":"2023-07-06T20:17:58.747083+0000","last_undegraded":"2023-07-06T20:17:58.747083+0000","last_fullsized":"2023-07-06T20:17:58.747083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:24:33.562887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'27","reported_seq":149,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:18:19.637149+0000","last_change":"2023-07-06T20:17:27.597285+0000","last_active":"2023-07-06T20:18:19.637149+0000","last_peered":"2023-07-06T20:18:19.637149+0000","last_clean":"2023-07-06T20:18:19.637149+0000","last_became_active":"2023-07-06T20:17:27.596850+0000","last_became_peered":"2023-07-06T20:17:27.596850+0000","last_unstale":"2023-07-06T20:18:19.637149+0000","last_undegraded":"2023-07-06T20:18:19.637149+0000","last_fullsized":"2023-07-06T20:18:19.637149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:41:53.734853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58527,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":104,"num_read_kb":108,"num_write":28,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746860+0000","last_change":"2023-07-06T20:17:28.569855+0000","last_active":"2023-07-06T20:17:58.746860+0000","last_peered":"2023-07-06T20:17:58.746860+0000","last_clean":"2023-07-06T20:17:58.746860+0000","last_became_active":"2023-07-06T20:17:28.569495+0000","last_became_peered":"2023-07-06T20:17:28.569495+0000","last_unstale":"2023-07-06T20:17:58.746860+0000","last_undegraded":"2023-07-06T20:17:58.746860+0000","last_fullsized":"2023-07-06T20:17:58.746860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:53:51.114104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.741227+0000","last_change":"2023-07-06T20:17:27.605898+0000","last_active":"2023-07-06T20:17:58.741227+0000","last_peered":"2023-07-06T20:17:58.741227+0000","last_clean":"2023-07-06T20:17:58.741227+0000","last_became_active":"2023-07-06T20:17:27.605605+0000","last_became_peered":"2023-07-06T20:17:27.605605+0000","last_unstale":"2023-07-06T20:17:58.741227+0000","last_undegraded":"2023-07-06T20:17:58.741227+0000","last_fullsized":"2023-07-06T20:17:58.741227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:27:10.201428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.740770+0000","last_change":"2023-07-06T20:17:28.579236+0000","last_active":"2023-07-06T20:17:58.740770+0000","last_peered":"2023-07-06T20:17:58.740770+0000","last_clean":"2023-07-06T20:17:58.740770+0000","last_became_active":"2023-07-06T20:17:28.578854+0000","last_became_peered":"2023-07-06T20:17:28.578854+0000","last_unstale":"2023-07-06T20:17:58.740770+0000","last_undegraded":"2023-07-06T20:17:58.740770+0000","last_fullsized":"2023-07-06T20:17:58.740770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:27:54.674535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861554+0000","last_change":"2023-07-06T20:17:27.595724+0000","last_active":"2023-07-06T20:17:58.861554+0000","last_peered":"2023-07-06T20:17:58.861554+0000","last_clean":"2023-07-06T20:17:58.861554+0000","last_became_active":"2023-07-06T20:17:27.595360+0000","last_became_peered":"2023-07-06T20:17:27.595360+0000","last_unstale":"2023-07-06T20:17:58.861554+0000","last_undegraded":"2023-07-06T20:17:58.861554+0000","last_fullsized":"2023-07-06T20:17:58.861554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:26:42.090999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744193+0000","last_change":"2023-07-06T20:17:28.573447+0000","last_active":"2023-07-06T20:17:58.744193+0000","last_peered":"2023-07-06T20:17:58.744193+0000","last_clean":"2023-07-06T20:17:58.744193+0000","last_became_active":"2023-07-06T20:17:28.573161+0000","last_became_peered":"2023-07-06T20:17:28.573161+0000","last_unstale":"2023-07-06T20:17:58.744193+0000","last_undegraded":"2023-07-06T20:17:58.744193+0000","last_fullsized":"2023-07-06T20:17:58.744193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:19:39.016518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":74,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:18:20.645425+0000","last_change":"2023-07-06T20:17:27.596254+0000","last_active":"2023-07-06T20:18:20.645425+0000","last_peered":"2023-07-06T20:18:20.645425+0000","last_clean":"2023-07-06T20:18:20.645425+0000","last_became_active":"2023-07-06T20:17:27.595178+0000","last_became_peered":"2023-07-06T20:17:27.595178+0000","last_unstale":"2023-07-06T20:18:20.645425+0000","last_undegraded":"2023-07-06T20:18:20.645425+0000","last_fullsized":"2023-07-06T20:18:20.645425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:19:24.488674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743517+0000","last_change":"2023-07-06T20:17:28.569745+0000","last_active":"2023-07-06T20:17:58.743517+0000","last_peered":"2023-07-06T20:17:58.743517+0000","last_clean":"2023-07-06T20:17:58.743517+0000","last_became_active":"2023-07-06T20:17:28.569468+0000","last_became_peered":"2023-07-06T20:17:28.569468+0000","last_unstale":"2023-07-06T20:17:58.743517+0000","last_undegraded":"2023-07-06T20:17:58.743517+0000","last_fullsized":"2023-07-06T20:17:58.743517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:43:29.072902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.772326+0000","last_change":"2023-07-06T20:17:58.771936+0000","last_active":"2023-07-06T20:17:58.772326+0000","last_peered":"2023-07-06T20:17:58.772326+0000","last_clean":"2023-07-06T20:17:58.772326+0000","last_became_active":"2023-07-06T20:17:58.748196+0000","last_became_peered":"2023-07-06T20:17:58.748196+0000","last_unstale":"2023-07-06T20:17:58.772326+0000","last_undegraded":"2023-07-06T20:17:58.772326+0000","last_fullsized":"2023-07-06T20:17:58.772326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:51:51.798767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861791+0000","last_change":"2023-07-06T20:17:28.575390+0000","last_active":"2023-07-06T20:17:58.861791+0000","last_peered":"2023-07-06T20:17:58.861791+0000","last_clean":"2023-07-06T20:17:58.861791+0000","last_became_active":"2023-07-06T20:17:28.575083+0000","last_became_peered":"2023-07-06T20:17:28.575083+0000","last_unstale":"2023-07-06T20:17:58.861791+0000","last_undegraded":"2023-07-06T20:17:58.861791+0000","last_fullsized":"2023-07-06T20:17:58.861791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:39:36.857081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747087+0000","last_change":"2023-07-06T20:17:27.600891+0000","last_active":"2023-07-06T20:17:58.747087+0000","last_peered":"2023-07-06T20:17:58.747087+0000","last_clean":"2023-07-06T20:17:58.747087+0000","last_became_active":"2023-07-06T20:17:27.600533+0000","last_became_peered":"2023-07-06T20:17:27.600533+0000","last_unstale":"2023-07-06T20:17:58.747087+0000","last_undegraded":"2023-07-06T20:17:58.747087+0000","last_fullsized":"2023-07-06T20:17:58.747087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:32:22.858270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.863073+0000","last_change":"2023-07-06T20:17:58.863073+0000","last_active":"2023-07-06T20:17:58.863073+0000","last_peered":"2023-07-06T20:17:58.863073+0000","last_clean":"2023-07-06T20:17:58.863073+0000","last_became_active":"2023-07-06T20:17:58.862613+0000","last_became_peered":"2023-07-06T20:17:58.862613+0000","last_unstale":"2023-07-06T20:17:58.863073+0000","last_undegraded":"2023-07-06T20:17:58.863073+0000","last_fullsized":"2023-07-06T20:17:58.863073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:51:25.715070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862771+0000","last_change":"2023-07-06T20:17:27.604464+0000","last_active":"2023-07-06T20:17:58.862771+0000","last_peered":"2023-07-06T20:17:58.862771+0000","last_clean":"2023-07-06T20:17:58.862771+0000","last_became_active":"2023-07-06T20:17:27.604036+0000","last_became_peered":"2023-07-06T20:17:27.604036+0000","last_unstale":"2023-07-06T20:17:58.862771+0000","last_undegraded":"2023-07-06T20:17:58.862771+0000","last_fullsized":"2023-07-06T20:17:58.862771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:14:15.333486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743848+0000","last_change":"2023-07-06T20:17:28.577302+0000","last_active":"2023-07-06T20:17:58.743848+0000","last_peered":"2023-07-06T20:17:58.743848+0000","last_clean":"2023-07-06T20:17:58.743848+0000","last_became_active":"2023-07-06T20:17:28.576998+0000","last_became_peered":"2023-07-06T20:17:28.576998+0000","last_unstale":"2023-07-06T20:17:58.743848+0000","last_undegraded":"2023-07-06T20:17:58.743848+0000","last_fullsized":"2023-07-06T20:17:58.743848+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:52:54.469918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744736+0000","last_change":"2023-07-06T20:17:27.608391+0000","last_active":"2023-07-06T20:17:58.744736+0000","last_peered":"2023-07-06T20:17:58.744736+0000","last_clean":"2023-07-06T20:17:58.744736+0000","last_became_active":"2023-07-06T20:17:27.608073+0000","last_became_peered":"2023-07-06T20:17:27.608073+0000","last_unstale":"2023-07-06T20:17:58.744736+0000","last_undegraded":"2023-07-06T20:17:58.744736+0000","last_fullsized":"2023-07-06T20:17:58.744736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:46:04.264772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744267+0000","last_change":"2023-07-06T20:17:28.577554+0000","last_active":"2023-07-06T20:17:58.744267+0000","last_peered":"2023-07-06T20:17:58.744267+0000","last_clean":"2023-07-06T20:17:58.744267+0000","last_became_active":"2023-07-06T20:17:28.577208+0000","last_became_peered":"2023-07-06T20:17:28.577208+0000","last_unstale":"2023-07-06T20:17:58.744267+0000","last_undegraded":"2023-07-06T20:17:58.744267+0000","last_fullsized":"2023-07-06T20:17:58.744267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:26:43.899763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861568+0000","last_change":"2023-07-06T20:17:28.571678+0000","last_active":"2023-07-06T20:17:58.861568+0000","last_peered":"2023-07-06T20:17:58.861568+0000","last_clean":"2023-07-06T20:17:58.861568+0000","last_became_active":"2023-07-06T20:17:28.570974+0000","last_became_peered":"2023-07-06T20:17:28.570974+0000","last_unstale":"2023-07-06T20:17:58.861568+0000","last_undegraded":"2023-07-06T20:17:58.861568+0000","last_fullsized":"2023-07-06T20:17:58.861568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:56:31.725747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:18:17.639991+0000","last_change":"2023-07-06T20:17:27.607172+0000","last_active":"2023-07-06T20:18:17.639991+0000","last_peered":"2023-07-06T20:18:17.639991+0000","last_clean":"2023-07-06T20:18:17.639991+0000","last_became_active":"2023-07-06T20:17:27.606841+0000","last_became_peered":"2023-07-06T20:17:27.606841+0000","last_unstale":"2023-07-06T20:18:17.639991+0000","last_undegraded":"2023-07-06T20:18:17.639991+0000","last_fullsized":"2023-07-06T20:18:17.639991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:41:58.034920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861784+0000","last_change":"2023-07-06T20:17:28.567707+0000","last_active":"2023-07-06T20:17:58.861784+0000","last_peered":"2023-07-06T20:17:58.861784+0000","last_clean":"2023-07-06T20:17:58.861784+0000","last_became_active":"2023-07-06T20:17:28.567394+0000","last_became_peered":"2023-07-06T20:17:28.567394+0000","last_unstale":"2023-07-06T20:17:58.861784+0000","last_undegraded":"2023-07-06T20:17:58.861784+0000","last_fullsized":"2023-07-06T20:17:58.861784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:22:52.559395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862245+0000","last_change":"2023-07-06T20:17:27.595746+0000","last_active":"2023-07-06T20:17:58.862245+0000","last_peered":"2023-07-06T20:17:58.862245+0000","last_clean":"2023-07-06T20:17:58.862245+0000","last_became_active":"2023-07-06T20:17:27.595039+0000","last_became_peered":"2023-07-06T20:17:27.595039+0000","last_unstale":"2023-07-06T20:17:58.862245+0000","last_undegraded":"2023-07-06T20:17:58.862245+0000","last_fullsized":"2023-07-06T20:17:58.862245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:51:26.692902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":337,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743684+0000","last_change":"2023-07-06T20:16:23.542324+0000","last_active":"2023-07-06T20:17:58.743684+0000","last_peered":"2023-07-06T20:17:58.743684+0000","last_clean":"2023-07-06T20:17:58.743684+0000","last_became_active":"2023-07-06T20:16:23.541832+0000","last_became_peered":"2023-07-06T20:16:23.541832+0000","last_unstale":"2023-07-06T20:17:58.743684+0000","last_undegraded":"2023-07-06T20:17:58.743684+0000","last_fullsized":"2023-07-06T20:17:58.743684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_clean_scrub_stamp":"2023-07-06T20:13:14.072924+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:43:32.297556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.740438+0000","last_change":"2023-07-06T20:17:28.577309+0000","last_active":"2023-07-06T20:17:58.740438+0000","last_peered":"2023-07-06T20:17:58.740438+0000","last_clean":"2023-07-06T20:17:58.740438+0000","last_became_active":"2023-07-06T20:17:28.576962+0000","last_became_peered":"2023-07-06T20:17:28.576962+0000","last_unstale":"2023-07-06T20:17:58.740438+0000","last_undegraded":"2023-07-06T20:17:58.740438+0000","last_fullsized":"2023-07-06T20:17:58.740438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:27:20.743560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743813+0000","last_change":"2023-07-06T20:17:27.601487+0000","last_active":"2023-07-06T20:17:58.743813+0000","last_peered":"2023-07-06T20:17:58.743813+0000","last_clean":"2023-07-06T20:17:58.743813+0000","last_became_active":"2023-07-06T20:17:27.601195+0000","last_became_peered":"2023-07-06T20:17:27.601195+0000","last_unstale":"2023-07-06T20:17:58.743813+0000","last_undegraded":"2023-07-06T20:17:58.743813+0000","last_fullsized":"2023-07-06T20:17:58.743813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:22:55.958720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747176+0000","last_change":"2023-07-06T20:17:27.610374+0000","last_active":"2023-07-06T20:17:58.747176+0000","last_peered":"2023-07-06T20:17:58.747176+0000","last_clean":"2023-07-06T20:17:58.747176+0000","last_became_active":"2023-07-06T20:17:27.610104+0000","last_became_peered":"2023-07-06T20:17:27.610104+0000","last_unstale":"2023-07-06T20:17:58.747176+0000","last_undegraded":"2023-07-06T20:17:58.747176+0000","last_fullsized":"2023-07-06T20:17:58.747176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:25:52.553271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.740620+0000","last_change":"2023-07-06T20:17:28.570146+0000","last_active":"2023-07-06T20:17:58.740620+0000","last_peered":"2023-07-06T20:17:58.740620+0000","last_clean":"2023-07-06T20:17:58.740620+0000","last_became_active":"2023-07-06T20:17:28.569341+0000","last_became_peered":"2023-07-06T20:17:28.569341+0000","last_unstale":"2023-07-06T20:17:58.740620+0000","last_undegraded":"2023-07-06T20:17:58.740620+0000","last_fullsized":"2023-07-06T20:17:58.740620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:47:51.639184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862987+0000","last_change":"2023-07-06T20:17:27.604630+0000","last_active":"2023-07-06T20:17:58.862987+0000","last_peered":"2023-07-06T20:17:58.862987+0000","last_clean":"2023-07-06T20:17:58.862987+0000","last_became_active":"2023-07-06T20:17:27.604118+0000","last_became_peered":"2023-07-06T20:17:27.604118+0000","last_unstale":"2023-07-06T20:17:58.862987+0000","last_undegraded":"2023-07-06T20:17:58.862987+0000","last_fullsized":"2023-07-06T20:17:58.862987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:02:10.943054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862071+0000","last_change":"2023-07-06T20:17:28.571753+0000","last_active":"2023-07-06T20:17:58.862071+0000","last_peered":"2023-07-06T20:17:58.862071+0000","last_clean":"2023-07-06T20:17:58.862071+0000","last_became_active":"2023-07-06T20:17:28.571073+0000","last_became_peered":"2023-07-06T20:17:28.571073+0000","last_unstale":"2023-07-06T20:17:58.862071+0000","last_undegraded":"2023-07-06T20:17:58.862071+0000","last_fullsized":"2023-07-06T20:17:58.862071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:43:01.162549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862660+0000","last_change":"2023-07-06T20:17:27.594354+0000","last_active":"2023-07-06T20:17:58.862660+0000","last_peered":"2023-07-06T20:17:58.862660+0000","last_clean":"2023-07-06T20:17:58.862660+0000","last_became_active":"2023-07-06T20:17:27.594042+0000","last_became_peered":"2023-07-06T20:17:27.594042+0000","last_unstale":"2023-07-06T20:17:58.862660+0000","last_undegraded":"2023-07-06T20:17:58.862660+0000","last_fullsized":"2023-07-06T20:17:58.862660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:16:57.362089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744208+0000","last_change":"2023-07-06T20:17:28.576170+0000","last_active":"2023-07-06T20:17:58.744208+0000","last_peered":"2023-07-06T20:17:58.744208+0000","last_clean":"2023-07-06T20:17:58.744208+0000","last_became_active":"2023-07-06T20:17:28.575493+0000","last_became_peered":"2023-07-06T20:17:28.575493+0000","last_unstale":"2023-07-06T20:17:58.744208+0000","last_undegraded":"2023-07-06T20:17:58.744208+0000","last_fullsized":"2023-07-06T20:17:58.744208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:03:49.657987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.747401+0000","last_change":"2023-07-06T20:17:27.602984+0000","last_active":"2023-07-06T20:17:58.747401+0000","last_peered":"2023-07-06T20:17:58.747401+0000","last_clean":"2023-07-06T20:17:58.747401+0000","last_became_active":"2023-07-06T20:17:27.602720+0000","last_became_peered":"2023-07-06T20:17:27.602720+0000","last_unstale":"2023-07-06T20:17:58.747401+0000","last_undegraded":"2023-07-06T20:17:58.747401+0000","last_fullsized":"2023-07-06T20:17:58.747401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:38:05.396361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.745930+0000","last_change":"2023-07-06T20:17:28.565954+0000","last_active":"2023-07-06T20:17:58.745930+0000","last_peered":"2023-07-06T20:17:58.745930+0000","last_clean":"2023-07-06T20:17:58.745930+0000","last_became_active":"2023-07-06T20:17:28.565300+0000","last_became_peered":"2023-07-06T20:17:28.565300+0000","last_unstale":"2023-07-06T20:17:58.745930+0000","last_undegraded":"2023-07-06T20:17:58.745930+0000","last_fullsized":"2023-07-06T20:17:58.745930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:18:37.459355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861635+0000","last_change":"2023-07-06T20:17:27.597859+0000","last_active":"2023-07-06T20:17:58.861635+0000","last_peered":"2023-07-06T20:17:58.861635+0000","last_clean":"2023-07-06T20:17:58.861635+0000","last_became_active":"2023-07-06T20:17:27.597479+0000","last_became_peered":"2023-07-06T20:17:27.597479+0000","last_unstale":"2023-07-06T20:17:58.861635+0000","last_undegraded":"2023-07-06T20:17:58.861635+0000","last_fullsized":"2023-07-06T20:17:58.861635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:12:44.598603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744672+0000","last_change":"2023-07-06T20:17:28.573069+0000","last_active":"2023-07-06T20:17:58.744672+0000","last_peered":"2023-07-06T20:17:58.744672+0000","last_clean":"2023-07-06T20:17:58.744672+0000","last_became_active":"2023-07-06T20:17:28.572742+0000","last_became_peered":"2023-07-06T20:17:28.572742+0000","last_unstale":"2023-07-06T20:17:58.744672+0000","last_undegraded":"2023-07-06T20:17:58.744672+0000","last_fullsized":"2023-07-06T20:17:58.744672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:52:06.992269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.741766+0000","last_change":"2023-07-06T20:17:27.599259+0000","last_active":"2023-07-06T20:17:58.741766+0000","last_peered":"2023-07-06T20:17:58.741766+0000","last_clean":"2023-07-06T20:17:58.741766+0000","last_became_active":"2023-07-06T20:17:27.598883+0000","last_became_peered":"2023-07-06T20:17:27.598883+0000","last_unstale":"2023-07-06T20:17:58.741766+0000","last_undegraded":"2023-07-06T20:17:58.741766+0000","last_fullsized":"2023-07-06T20:17:58.741766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:05:19.031900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861912+0000","last_change":"2023-07-06T20:17:28.573874+0000","last_active":"2023-07-06T20:17:58.861912+0000","last_peered":"2023-07-06T20:17:58.861912+0000","last_clean":"2023-07-06T20:17:58.861912+0000","last_became_active":"2023-07-06T20:17:28.573379+0000","last_became_peered":"2023-07-06T20:17:28.573379+0000","last_unstale":"2023-07-06T20:17:58.861912+0000","last_undegraded":"2023-07-06T20:17:58.861912+0000","last_fullsized":"2023-07-06T20:17:58.861912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:45:45.647998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746518+0000","last_change":"2023-07-06T20:17:27.601474+0000","last_active":"2023-07-06T20:17:58.746518+0000","last_peered":"2023-07-06T20:17:58.746518+0000","last_clean":"2023-07-06T20:17:58.746518+0000","last_became_active":"2023-07-06T20:17:27.601099+0000","last_became_peered":"2023-07-06T20:17:27.601099+0000","last_unstale":"2023-07-06T20:17:58.746518+0000","last_undegraded":"2023-07-06T20:17:58.746518+0000","last_fullsized":"2023-07-06T20:17:58.746518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:59:28.147471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744872+0000","last_change":"2023-07-06T20:17:28.571170+0000","last_active":"2023-07-06T20:17:58.744872+0000","last_peered":"2023-07-06T20:17:58.744872+0000","last_clean":"2023-07-06T20:17:58.744872+0000","last_became_active":"2023-07-06T20:17:28.570874+0000","last_became_peered":"2023-07-06T20:17:28.570874+0000","last_unstale":"2023-07-06T20:17:58.744872+0000","last_undegraded":"2023-07-06T20:17:58.744872+0000","last_fullsized":"2023-07-06T20:17:58.744872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:52:55.391441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.741459+0000","last_change":"2023-07-06T20:17:27.603604+0000","last_active":"2023-07-06T20:17:58.741459+0000","last_peered":"2023-07-06T20:17:58.741459+0000","last_clean":"2023-07-06T20:17:58.741459+0000","last_became_active":"2023-07-06T20:17:27.603015+0000","last_became_peered":"2023-07-06T20:17:27.603015+0000","last_unstale":"2023-07-06T20:17:58.741459+0000","last_undegraded":"2023-07-06T20:17:58.741459+0000","last_fullsized":"2023-07-06T20:17:58.741459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:09:30.856672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744507+0000","last_change":"2023-07-06T20:17:28.567906+0000","last_active":"2023-07-06T20:17:58.744507+0000","last_peered":"2023-07-06T20:17:58.744507+0000","last_clean":"2023-07-06T20:17:58.744507+0000","last_became_active":"2023-07-06T20:17:28.567509+0000","last_became_peered":"2023-07-06T20:17:28.567509+0000","last_unstale":"2023-07-06T20:17:58.744507+0000","last_undegraded":"2023-07-06T20:17:58.744507+0000","last_fullsized":"2023-07-06T20:17:58.744507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:36:49.402373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861781+0000","last_change":"2023-07-06T20:17:27.598323+0000","last_active":"2023-07-06T20:17:58.861781+0000","last_peered":"2023-07-06T20:17:58.861781+0000","last_clean":"2023-07-06T20:17:58.861781+0000","last_became_active":"2023-07-06T20:17:27.598032+0000","last_became_peered":"2023-07-06T20:17:27.598032+0000","last_unstale":"2023-07-06T20:17:58.861781+0000","last_undegraded":"2023-07-06T20:17:58.861781+0000","last_fullsized":"2023-07-06T20:17:58.861781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:54:23.185321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743964+0000","last_change":"2023-07-06T20:17:28.571756+0000","last_active":"2023-07-06T20:17:58.743964+0000","last_peered":"2023-07-06T20:17:58.743964+0000","last_clean":"2023-07-06T20:17:58.743964+0000","last_became_active":"2023-07-06T20:17:28.571440+0000","last_became_peered":"2023-07-06T20:17:28.571440+0000","last_unstale":"2023-07-06T20:17:58.743964+0000","last_undegraded":"2023-07-06T20:17:58.743964+0000","last_fullsized":"2023-07-06T20:17:58.743964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:13:22.304976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.741065+0000","last_change":"2023-07-06T20:17:27.602620+0000","last_active":"2023-07-06T20:17:58.741065+0000","last_peered":"2023-07-06T20:17:58.741065+0000","last_clean":"2023-07-06T20:17:58.741065+0000","last_became_active":"2023-07-06T20:17:27.602224+0000","last_became_peered":"2023-07-06T20:17:27.602224+0000","last_unstale":"2023-07-06T20:17:58.741065+0000","last_undegraded":"2023-07-06T20:17:58.741065+0000","last_fullsized":"2023-07-06T20:17:58.741065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:43:07.550009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744425+0000","last_change":"2023-07-06T20:17:28.569203+0000","last_active":"2023-07-06T20:17:58.744425+0000","last_peered":"2023-07-06T20:17:58.744425+0000","last_clean":"2023-07-06T20:17:58.744425+0000","last_became_active":"2023-07-06T20:17:28.568948+0000","last_became_peered":"2023-07-06T20:17:28.568948+0000","last_unstale":"2023-07-06T20:17:58.744425+0000","last_undegraded":"2023-07-06T20:17:58.744425+0000","last_fullsized":"2023-07-06T20:17:58.744425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:47:14.103672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744725+0000","last_change":"2023-07-06T20:17:28.568829+0000","last_active":"2023-07-06T20:17:58.744725+0000","last_peered":"2023-07-06T20:17:58.744725+0000","last_clean":"2023-07-06T20:17:58.744725+0000","last_became_active":"2023-07-06T20:17:28.568511+0000","last_became_peered":"2023-07-06T20:17:28.568511+0000","last_unstale":"2023-07-06T20:17:58.744725+0000","last_undegraded":"2023-07-06T20:17:58.744725+0000","last_fullsized":"2023-07-06T20:17:58.744725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:13:19.422184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.863188+0000","last_change":"2023-07-06T20:17:58.863188+0000","last_active":"2023-07-06T20:17:58.863188+0000","last_peered":"2023-07-06T20:17:58.863188+0000","last_clean":"2023-07-06T20:17:58.863188+0000","last_became_active":"2023-07-06T20:17:58.862743+0000","last_became_peered":"2023-07-06T20:17:58.862743+0000","last_unstale":"2023-07-06T20:17:58.863188+0000","last_undegraded":"2023-07-06T20:17:58.863188+0000","last_fullsized":"2023-07-06T20:17:58.863188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:08:47.107647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.741447+0000","last_change":"2023-07-06T20:17:28.572264+0000","last_active":"2023-07-06T20:17:58.741447+0000","last_peered":"2023-07-06T20:17:58.741447+0000","last_clean":"2023-07-06T20:17:58.741447+0000","last_became_active":"2023-07-06T20:17:28.571914+0000","last_became_peered":"2023-07-06T20:17:28.571914+0000","last_unstale":"2023-07-06T20:17:58.741447+0000","last_undegraded":"2023-07-06T20:17:58.741447+0000","last_fullsized":"2023-07-06T20:17:58.741447+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:54:59.445149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.744193+0000","last_change":"2023-07-06T20:17:28.581374+0000","last_active":"2023-07-06T20:17:58.744193+0000","last_peered":"2023-07-06T20:17:58.744193+0000","last_clean":"2023-07-06T20:17:58.744193+0000","last_became_active":"2023-07-06T20:17:28.580995+0000","last_became_peered":"2023-07-06T20:17:28.580995+0000","last_unstale":"2023-07-06T20:17:58.744193+0000","last_undegraded":"2023-07-06T20:17:58.744193+0000","last_fullsized":"2023-07-06T20:17:58.744193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:55:00.813142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.740889+0000","last_change":"2023-07-06T20:17:28.578926+0000","last_active":"2023-07-06T20:17:58.740889+0000","last_peered":"2023-07-06T20:17:58.740889+0000","last_clean":"2023-07-06T20:17:58.740889+0000","last_became_active":"2023-07-06T20:17:28.578620+0000","last_became_peered":"2023-07-06T20:17:28.578620+0000","last_unstale":"2023-07-06T20:17:58.740889+0000","last_undegraded":"2023-07-06T20:17:58.740889+0000","last_fullsized":"2023-07-06T20:17:58.740889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:02:08.450840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.864937+0000","last_change":"2023-07-06T20:17:58.864937+0000","last_active":"2023-07-06T20:17:58.864937+0000","last_peered":"2023-07-06T20:17:58.864937+0000","last_clean":"2023-07-06T20:17:58.864937+0000","last_became_active":"2023-07-06T20:17:58.864457+0000","last_became_peered":"2023-07-06T20:17:58.864457+0000","last_unstale":"2023-07-06T20:17:58.864937+0000","last_undegraded":"2023-07-06T20:17:58.864937+0000","last_fullsized":"2023-07-06T20:17:58.864937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:13:23.476623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.741781+0000","last_change":"2023-07-06T20:17:28.567965+0000","last_active":"2023-07-06T20:17:58.741781+0000","last_peered":"2023-07-06T20:17:58.741781+0000","last_clean":"2023-07-06T20:17:58.741781+0000","last_became_active":"2023-07-06T20:17:28.567618+0000","last_became_peered":"2023-07-06T20:17:28.567618+0000","last_unstale":"2023-07-06T20:17:58.741781+0000","last_undegraded":"2023-07-06T20:17:58.741781+0000","last_fullsized":"2023-07-06T20:17:58.741781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:30:26.630851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.862035+0000","last_change":"2023-07-06T20:17:28.571934+0000","last_active":"2023-07-06T20:17:58.862035+0000","last_peered":"2023-07-06T20:17:58.862035+0000","last_clean":"2023-07-06T20:17:58.862035+0000","last_became_active":"2023-07-06T20:17:28.571472+0000","last_became_peered":"2023-07-06T20:17:28.571472+0000","last_unstale":"2023-07-06T20:17:58.862035+0000","last_undegraded":"2023-07-06T20:17:58.862035+0000","last_fullsized":"2023-07-06T20:17:58.862035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:29.285045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861417+0000","last_change":"2023-07-06T20:17:28.571904+0000","last_active":"2023-07-06T20:17:58.861417+0000","last_peered":"2023-07-06T20:17:58.861417+0000","last_clean":"2023-07-06T20:17:58.861417+0000","last_became_active":"2023-07-06T20:17:28.571375+0000","last_became_peered":"2023-07-06T20:17:28.571375+0000","last_unstale":"2023-07-06T20:17:58.861417+0000","last_undegraded":"2023-07-06T20:17:58.861417+0000","last_fullsized":"2023-07-06T20:17:58.861417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:33:16.020700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.861467+0000","last_change":"2023-07-06T20:17:28.569139+0000","last_active":"2023-07-06T20:17:58.861467+0000","last_peered":"2023-07-06T20:17:58.861467+0000","last_clean":"2023-07-06T20:17:58.861467+0000","last_became_active":"2023-07-06T20:17:28.568828+0000","last_became_peered":"2023-07-06T20:17:28.568828+0000","last_unstale":"2023-07-06T20:17:58.861467+0000","last_undegraded":"2023-07-06T20:17:58.861467+0000","last_fullsized":"2023-07-06T20:17:58.861467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:41:29.170094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.746782+0000","last_change":"2023-07-06T20:17:28.562352+0000","last_active":"2023-07-06T20:17:58.746782+0000","last_peered":"2023-07-06T20:17:58.746782+0000","last_clean":"2023-07-06T20:17:58.746782+0000","last_became_active":"2023-07-06T20:17:28.562128+0000","last_became_peered":"2023-07-06T20:17:28.562128+0000","last_unstale":"2023-07-06T20:17:58.746782+0000","last_undegraded":"2023-07-06T20:17:58.746782+0000","last_fullsized":"2023-07-06T20:17:58.746782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:47.370429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.740766+0000","last_change":"2023-07-06T20:17:28.578940+0000","last_active":"2023-07-06T20:17:58.740766+0000","last_peered":"2023-07-06T20:17:58.740766+0000","last_clean":"2023-07-06T20:17:58.740766+0000","last_became_active":"2023-07-06T20:17:28.578660+0000","last_became_peered":"2023-07-06T20:17:28.578660+0000","last_unstale":"2023-07-06T20:17:58.740766+0000","last_undegraded":"2023-07-06T20:17:58.740766+0000","last_fullsized":"2023-07-06T20:17:58.740766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:52:29.675150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T20:17:58.743663+0000","last_change":"2023-07-06T20:17:28.573964+0000","last_active":"2023-07-06T20:17:58.743663+0000","last_peered":"2023-07-06T20:17:58.743663+0000","last_clean":"2023-07-06T20:17:58.743663+0000","last_became_active":"2023-07-06T20:17:28.573618+0000","last_became_peered":"2023-07-06T20:17:28.573618+0000","last_unstale":"2023-07-06T20:17:58.743663+0000","last_undegraded":"2023-07-06T20:17:58.743663+0000","last_fullsized":"2023-07-06T20:17:58.743663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:55:42.131166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62821,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":316,"num_read_kb":221,"num_write":150,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":188463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449112,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.621}]},{"osd":6,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68300000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.81699999999999995}]},{"osd":10,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.748}]}]},{"osd":10,"up_from":71,"seq":304942678043,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1622016,"data_stored":1094238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.436}]},{"osd":5,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.45500000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.72799999999999998}]},{"osd":9,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":5.4219999999999997,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.66000000000000003}]},{"osd":11,"last update":"Thu Jul 6 20:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.70299999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841567,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":444118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.505,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":5.7919999999999998,"5min":5.7919999999999998,"15min":5.7919999999999998},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71199999999999997,"15min":0.71699999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.93799999999999994,"5min":1.153,"15min":1.153},"last":0.44800000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90400000000000003,"5min":1.274,"15min":1.274},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":0.73899999999999999,"15min":0.67300000000000004},"min":{"1min":0.36499999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":16.628,"5min":16.628,"15min":16.628},"last":0.628},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.76200000000000001,"15min":0.72399999999999998},"min":{"1min":0.40100000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":11.332000000000001,"5min":11.332000000000001,"15min":11.332000000000001},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.46600000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.47599999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.83399999999999996,"15min":0.84199999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.82299999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.75}]},{"osd":6,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.79100000000000004,"15min":0.79400000000000004},"min":{"1min":0.51100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.81100000000000005,"15min":0.83699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0880000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.76800000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.879,"15min":0.89100000000000001},"min":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.90200000000000002,"15min":0.91600000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.81399999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.86899999999999999,"15min":0.88700000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.97499999999999998,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89700000000000002,"15min":0.90300000000000002},"min":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":0.73799999999999999}]},{"osd":10,"last update":"Thu Jul 6 20:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.78200000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103204,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":443645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77200000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.64400000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.442,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.63},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81000000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44500000000000001,"15min":0.443},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.61099999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.437},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76900000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.497},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67300000000000004,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82199999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84599999999999997,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.46000000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.83099999999999996,"5min":0.87,"15min":0.87},"last":0.66700000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80500000000000005,"5min":0.878,"15min":0.878},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59599999999999997,"15min":0.59199999999999997},"min":{"1min":0.316,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.877,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71199999999999997,"15min":0.71099999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92200000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.96299999999999997,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.88600000000000001,"15min":0.91200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.014,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.81000000000000005}]},{"osd":9,"last update":"Thu Jul 6 20:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.63600000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.77200000000000002,"15min":0.75800000000000001},"min":{"1min":0.59799999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.79500000000000004}]},{"osd":10,"last update":"Thu Jul 6 20:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.76100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332137,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1536000,"data_stored":1036267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51300000000000001,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85299999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.39400000000000002,"15min":0.38400000000000001},"min":{"1min":0.34699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.41299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.625,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.40400000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80400000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.68000000000000005,"15min":0.622},"min":{"1min":0.40100000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.034,"5min":0.67300000000000004,"15min":0.59599999999999997},"min":{"1min":0.47899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.048,"5min":6.048,"15min":6.048},"last":0.75900000000000001}]},{"osd":4,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.32300000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.499,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59299999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.44700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.82999999999999996}]},{"osd":6,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.755,"15min":0.68100000000000005},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":8.1579999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.77000000000000002,"15min":0.70299999999999996},"min":{"1min":0.48799999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":8.1980000000000004,"5min":8.1980000000000004,"15min":8.1980000000000004},"last":0.88400000000000001}]},{"osd":8,"last update":"Thu Jul 6 20:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.90500000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.56000000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96999999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.58199999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.877}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.91900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":443513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.495,"15min":0.47499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jul 6 20:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.42999999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98199999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.65300000000000002},"min":{"1min":0.5,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95399999999999996,"5min":1.089,"15min":1.089},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jul 6 20:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68100000000000005,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.503}]},{"osd":3,"last update":"Thu Jul 6 20:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91300000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.55200000000000005,"5min":0.435,"15min":0.435},"max":{"1min":1.002,"5min":1.254,"15min":1.254},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71099999999999997,"15min":0.70599999999999996},"min":{"1min":0.51100000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0169999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jul 6 20:17:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.754,"15min":0.76800000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97999999999999998,"5min":1.393,"15min":1.393},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0569999999999999,"5min":1.131,"15min":1.131},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Jul 6 20:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.78500000000000003,"15min":0.77500000000000002},"min":{"1min":0.60699999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79500000000000004,"15min":0.78800000000000003},"min":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.748}]},{"osd":8,"last update":"Thu Jul 6 20:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.83599999999999997,"15min":0.85099999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.036,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96599999999999997,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.78200000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.56299999999999994,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.89000000000000001,"15min":0.90200000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.76500000000000001}]},{"osd":10,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.75800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":501939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.54000000000000004,"15min":0.48799999999999999},"min":{"1min":0.34799999999999998,"5min":0.248,"15min":0.248},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.52500000000000002},"min":{"1min":0.36199999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.82299999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.67100000000000004}]},{"osd":2,"last update":"Thu Jul 6 20:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56899999999999995,"15min":0.51300000000000001},"min":{"1min":0.41599999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.93600000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.59399999999999997,"15min":0.52000000000000002},"min":{"1min":0.60699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94999999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.77500000000000002}]},{"osd":3,"last update":"Thu Jul 6 20:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.55100000000000005},"min":{"1min":0.38500000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.42899999999999999,"15min":0.39300000000000002},"min":{"1min":0.33800000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.497},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.46999999999999997,"15min":0.42599999999999999},"min":{"1min":0.33500000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jul 6 20:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.52700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0169999999999999,"5min":1.042,"15min":1.042},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67500000000000004,"15min":0.65500000000000003},"min":{"1min":0.46300000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jul 6 20:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.624,"15min":0.59199999999999997},"min":{"1min":0.42399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.59599999999999997,"15min":0.55500000000000005},"min":{"1min":0.5,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.75800000000000001}]},{"osd":8,"last update":"Thu Jul 6 20:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.66900000000000004,"15min":0.65500000000000003},"min":{"1min":0.49199999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.746},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.73899999999999999,"15min":0.72399999999999998},"min":{"1min":0.57899999999999996,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.90100000000000002}]},{"osd":9,"last update":"Thu Jul 6 20:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.75700000000000001,"15min":0.74299999999999999},"min":{"1min":0.57699999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.77200000000000002,"15min":0.76600000000000001},"min":{"1min":0.626,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.78700000000000003}]},{"osd":10,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.92800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":444778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.317,"15min":0.317},"max":{"1min":4.1130000000000004,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.628,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.6400000000000001,"5min":4.1379999999999999,"15min":4.1379999999999999},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61399999999999999,"15min":0.58199999999999996},"min":{"1min":0.45100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85399999999999998,"5min":1.085,"15min":1.085},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.622,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89000000000000001,"5min":1.258,"15min":1.258},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jul 6 20:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jul 6 20:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.73799999999999999},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0049999999999999,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65400000000000003,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0389999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0069999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jul 6 20:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63300000000000001,"15min":0.64000000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":0.69399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81100000000000005,"5min":1.319,"15min":1.319},"last":0.68100000000000005}]},{"osd":7,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.03,"5min":1.131,"15min":1.131},"last":0.505},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79700000000000004,"15min":0.79600000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.179,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.84399999999999997}]},{"osd":8,"last update":"Thu Jul 6 20:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76600000000000001,"15min":0.77800000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.004,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.80600000000000005,"15min":0.82199999999999995},"min":{"1min":0.51000000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94899999999999995,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70099999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.81599999999999995,"15min":0.82999999999999996},"min":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.014,"5min":1.161,"15min":1.161},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78100000000000003,"15min":0.79300000000000004},"min":{"1min":0.45100000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0409999999999999,"5min":1.121,"15min":1.121},"last":0.65400000000000003}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.626},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.76400000000000001}]},{"osd":11,"last update":"Thu Jul 6 20:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.90900000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":503407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55800000000000005,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.88700000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.56000000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69399999999999995,"5min":0.876,"15min":0.876},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60899999999999999,"15min":0.623},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83599999999999997,"5min":0.995,"15min":0.995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.627,"15min":0.627},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79800000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jul 6 20:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jul 6 20:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47199999999999998,"15min":0.45200000000000001},"min":{"1min":0.33500000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.625},{"interface":"front","average":{"1min":0.503,"5min":0.47299999999999998,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.45500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.86499999999999999,"5min":1.056,"15min":1.056},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67500000000000004,"15min":0.65000000000000002},"min":{"1min":0.53200000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86199999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.86399999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.624},"min":{"1min":0.435,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81899999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59799999999999998,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70999999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.75,"15min":0.754},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.93899999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.78700000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.93799999999999994,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.89800000000000002}]},{"osd":10,"last update":"Thu Jul 6 20:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.88500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":443830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.40699999999999997,"15min":0.38300000000000001},"min":{"1min":0.29099999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.435,"15min":0.40300000000000002},"min":{"1min":0.32900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65300000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jul 6 20:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.64200000000000002,"15min":0.57999999999999996},"min":{"1min":0.51000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.871,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86199999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jul 6 20:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.621,"15min":0.58899999999999997},"min":{"1min":0.496,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81899999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.66700000000000004,"15min":0.622},"min":{"1min":0.48199999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.875,"5min":3.6720000000000002,"15min":3.6720000000000002},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jul 6 20:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.73199999999999998,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":6.9420000000000002,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.58699999999999997},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66900000000000004}]},{"osd":5,"last update":"Thu Jul 6 20:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.65000000000000002,"15min":0.624},"min":{"1min":0.56899999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.46800000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92400000000000004,"5min":1.052,"15min":1.052},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59099999999999997,"15min":0.56499999999999995},"min":{"1min":0.496,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.40799999999999997}]},{"osd":7,"last update":"Thu Jul 6 20:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.625},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.79900000000000004,"15min":0.76500000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.1859999999999999,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":0.72599999999999998}]},{"osd":8,"last update":"Thu Jul 6 20:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.58299999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":0.74099999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.034,"5min":1.135,"15min":1.135},"last":0.55200000000000005}]},{"osd":10,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":444286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.55200000000000005,"15min":0.501},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":6.1539999999999999,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.51900000000000002,"15min":0.49299999999999999},"min":{"1min":0.45700000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.35799999999999998},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58399999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.436},{"interface":"front","average":{"1min":0.44,"5min":0.39600000000000002,"15min":0.35799999999999998},"min":{"1min":0.311,"5min":0.219,"15min":0.219},"max":{"1min":0.55900000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.47199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.44400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86699999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.53600000000000003},"min":{"1min":0.52200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.099,"5min":5.593,"15min":5.593},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.123,"5min":0.71699999999999997,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.378,"15min":0.378},"max":{"1min":9.7959999999999994,"5min":9.7959999999999994,"15min":9.7959999999999994},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jul 6 20:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.55600000000000005},"min":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89800000000000002,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.625},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:17:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70299999999999996,"5min":1.109,"15min":1.109},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.69099999999999995},"min":{"1min":0.54900000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89900000000000002,"5min":1.163,"15min":1.163},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jul 6 20:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.35499999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.97699999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.372,"5min":0.314,"15min":0.314},"max":{"1min":0.91200000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.60999999999999999}]},{"osd":8,"last update":"Thu Jul 6 20:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.47299999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.1890000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93700000000000006,"5min":1.194,"15min":1.194},"last":0.66600000000000004}]},{"osd":9,"last update":"Thu Jul 6 20:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.50700000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.94099999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.2470000000000001,"5min":0.92400000000000004,"15min":0.87},"min":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":8.3369999999999997,"5min":8.3369999999999997,"15min":8.3369999999999997},"last":0.81699999999999995}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":5,"up_from":38,"seq":163208757299,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1035278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80600000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69399999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78100000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.69699999999999995}]},{"osd":2,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40300000000000002,"15min":0.39800000000000002},"min":{"1min":0.29299999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.436,"15min":0.42999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76200000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83299999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.622}]},{"osd":4,"last update":"Thu Jul 6 20:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.755},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.85399999999999998}]},{"osd":6,"last update":"Thu Jul 6 20:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85999999999999999,"5min":0.873,"15min":0.873},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64900000000000002,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.78800000000000003}]},{"osd":8,"last update":"Thu Jul 6 20:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.38300000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66300000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76500000000000001,"15min":0.76300000000000001},"min":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.91200000000000003,"5min":1.012,"15min":1.012},"last":0.79600000000000004}]},{"osd":10,"last update":"Thu Jul 6 20:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.81299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:18:24.421 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-06T20:18:24.421 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-06T20:18:24.422 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-06T20:18:24.422 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph health --format=json 2023-07-06T20:18:25.176 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:25.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:24 smithi069 ceph-mon[126399]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:25.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:24 smithi069 ceph-mon[126399]: from='client.24874 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:18:25.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:24 smithi156 ceph-mon[129280]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:25.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:24 smithi156 ceph-mon[129280]: from='client.24874 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:18:25.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:24 smithi098 ceph-mon[128968]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:25.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:24 smithi098 ceph-mon[128968]: from='client.24874 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:18:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.669+0000 7f83bdc8e700 1 -- 172.21.15.69:0/1223857377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b81008d0 msgr2=0x7f83b81048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.669+0000 7f83bdc8e700 1 --2- 172.21.15.69:0/1223857377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b81008d0 0x7f83b81048a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0099c0 tx=0x7f83ac009cd0 comp rx=0 tx=0).stop 2023-07-06T20:18:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.670+0000 7f83bdc8e700 1 -- 172.21.15.69:0/1223857377 shutdown_connections 2023-07-06T20:18:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.670+0000 7f83bdc8e700 1 --2- 172.21.15.69:0/1223857377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83b8104fd0 0x7f83b810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.670+0000 7f83bdc8e700 1 --2- 172.21.15.69:0/1223857377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83b80fff30 0x7f83b8100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.670+0000 7f83bdc8e700 1 --2- 172.21.15.69:0/1223857377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b81008d0 0x7f83b81048a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.670+0000 7f83bdc8e700 1 -- 172.21.15.69:0/1223857377 >> 172.21.15.69:0/1223857377 conn(0x7f83b80fb790 msgr2=0x7f83b80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.670+0000 7f83bdc8e700 1 -- 172.21.15.69:0/1223857377 shutdown_connections 2023-07-06T20:18:26.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.670+0000 7f83bdc8e700 1 -- 172.21.15.69:0/1223857377 wait complete. 2023-07-06T20:18:26.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.671+0000 7f83bdc8e700 1 Processor -- start 2023-07-06T20:18:26.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.671+0000 7f83bdc8e700 1 -- start start 2023-07-06T20:18:26.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.671+0000 7f83bdc8e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83b80fff30 0x7f83b8072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:26.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.671+0000 7f83bdc8e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83b81008d0 0x7f83b8072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.671+0000 7f83bdc8e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8104fd0 0x7f83b80733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.671+0000 7f83bdc8e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b81a41f0 con 0x7f83b8104fd0 2023-07-06T20:18:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.671+0000 7f83bdc8e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b81a4360 con 0x7f83b80fff30 2023-07-06T20:18:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.671+0000 7f83bdc8e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b81a4520 con 0x7f83b81008d0 2023-07-06T20:18:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.671+0000 7f83b6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83b81008d0 0x7f83b8072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.671+0000 7f83b77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83b80fff30 0x7f83b8072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.671+0000 7f83b6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83b81008d0 0x7f83b8072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:53058/0 (socket says 172.21.15.69:53058) 2023-07-06T20:18:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.671+0000 7f83b6ffd700 1 -- 172.21.15.69:0/194086873 learned_addr learned my addr 172.21.15.69:0/194086873 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83b7fff700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8104fd0 0x7f83b80733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83b6ffd700 1 -- 172.21.15.69:0/194086873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83b80fff30 msgr2=0x7f83b8072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83b6ffd700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83b80fff30 0x7f83b8072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83b6ffd700 1 -- 172.21.15.69:0/194086873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8104fd0 msgr2=0x7f83b80733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83b6ffd700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8104fd0 0x7f83b80733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83b6ffd700 1 -- 172.21.15.69:0/194086873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac0095f0 con 0x7f83b81008d0 2023-07-06T20:18:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83b77fe700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83b80fff30 0x7f83b8072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83b7fff700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8104fd0 0x7f83b80733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83b6ffd700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83b81008d0 0x7f83b8072e80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009f50 tx=0x7f83ac005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83b4ff9700 1 -- 172.21.15.69:0/194086873 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac01d070 con 0x7f83b81008d0 2023-07-06T20:18:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83bdc8e700 1 -- 172.21.15.69:0/194086873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b81a47a0 con 0x7f83b81008d0 2023-07-06T20:18:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83b4ff9700 1 -- 172.21.15.69:0/194086873 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83ac003f90 con 0x7f83b81008d0 2023-07-06T20:18:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.672+0000 7f83b4ff9700 1 -- 172.21.15.69:0/194086873 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac022bf0 con 0x7f83b81008d0 2023-07-06T20:18:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.673+0000 7f83bdc8e700 1 -- 172.21.15.69:0/194086873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b81a4c10 con 0x7f83b81008d0 2023-07-06T20:18:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.673+0000 7f83bdc8e700 1 -- 172.21.15.69:0/194086873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b804e340 con 0x7f83b81008d0 2023-07-06T20:18:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.676+0000 7f83b4ff9700 1 -- 172.21.15.69:0/194086873 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f83ac022d50 con 0x7f83b81008d0 2023-07-06T20:18:26.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.677+0000 7f83b4ff9700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f83a00a11e0 0x7f83a00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:26.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.677+0000 7f83b4ff9700 1 -- 172.21.15.69:0/194086873 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f83ac0ce330 con 0x7f83b81008d0 2023-07-06T20:18:26.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.677+0000 7f83b77fe700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f83a00a11e0 0x7f83a00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:26.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.678+0000 7f83b77fe700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f83a00a11e0 0x7f83a00a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f83a8005950 tx=0x7f83a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:26.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.679+0000 7f83b4ff9700 1 -- 172.21.15.69:0/194086873 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f83ac097940 con 0x7f83b81008d0 2023-07-06T20:18:26.918 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.917+0000 7f83bdc8e700 1 -- 172.21.15.69:0/194086873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f83b81a4e30 con 0x7f83b81008d0 2023-07-06T20:18:26.919 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.918+0000 7f83b4ff9700 1 -- 172.21.15.69:0/194086873 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f83ac09af60 con 0x7f83b81008d0 2023-07-06T20:18:26.919 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:26.919 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T20:18:26.921 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.920+0000 7f83bdc8e700 1 -- 172.21.15.69:0/194086873 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f83a00a11e0 msgr2=0x7f83a00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:26.922 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.920+0000 7f83bdc8e700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f83a00a11e0 0x7f83a00a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f83a8005950 tx=0x7f83a800b410 comp rx=0 tx=0).stop 2023-07-06T20:18:26.922 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.920+0000 7f83bdc8e700 1 -- 172.21.15.69:0/194086873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83b81008d0 msgr2=0x7f83b8072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:26.922 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.920+0000 7f83bdc8e700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83b81008d0 0x7f83b8072e80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009f50 tx=0x7f83ac005740 comp rx=0 tx=0).stop 2023-07-06T20:18:26.922 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.921+0000 7f83bdc8e700 1 -- 172.21.15.69:0/194086873 shutdown_connections 2023-07-06T20:18:26.922 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.921+0000 7f83bdc8e700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f83a00a11e0 0x7f83a00a3690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:26.922 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.921+0000 7f83bdc8e700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8104fd0 0x7f83b80733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.921+0000 7f83bdc8e700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83b80fff30 0x7f83b8072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.921+0000 7f83bdc8e700 1 --2- 172.21.15.69:0/194086873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83b81008d0 0x7f83b8072e80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.921+0000 7f83bdc8e700 1 -- 172.21.15.69:0/194086873 >> 172.21.15.69:0/194086873 conn(0x7f83b80fb790 msgr2=0x7f83b80fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.921+0000 7f83bdc8e700 1 -- 172.21.15.69:0/194086873 shutdown_connections 2023-07-06T20:18:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:26.921+0000 7f83bdc8e700 1 -- 172.21.15.69:0/194086873 wait complete. 2023-07-06T20:18:27.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:26 smithi069 ceph-mon[126399]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-06T20:18:27.350 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-06T20:18:27.351 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:18:27.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:26 smithi156 ceph-mon[129280]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-06T20:18:27.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:26 smithi098 ceph-mon[128968]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-06T20:18:27.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.549+0000 7f82a71ff700 1 -- 172.21.15.69:0/3560849194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82a0100e20 msgr2=0x7f82a0101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:27.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.549+0000 7f82a71ff700 1 --2- 172.21.15.69:0/3560849194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82a0100e20 0x7f82a0101270 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f82900098f0 tx=0x7f8290009c00 comp rx=0 tx=0).stop 2023-07-06T20:18:27.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.550+0000 7f82a71ff700 1 -- 172.21.15.69:0/3560849194 shutdown_connections 2023-07-06T20:18:27.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.550+0000 7f82a71ff700 1 --2- 172.21.15.69:0/3560849194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82a01017b0 0x7f82a0108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.550+0000 7f82a71ff700 1 --2- 172.21.15.69:0/3560849194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a00ffc30 0x7f82a0100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.550+0000 7f82a71ff700 1 --2- 172.21.15.69:0/3560849194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82a0100e20 0x7f82a0101270 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.550+0000 7f82a71ff700 1 -- 172.21.15.69:0/3560849194 >> 172.21.15.69:0/3560849194 conn(0x7f82a00fb210 msgr2=0x7f82a00fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:27.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.550+0000 7f82a71ff700 1 -- 172.21.15.69:0/3560849194 shutdown_connections 2023-07-06T20:18:27.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.550+0000 7f82a71ff700 1 -- 172.21.15.69:0/3560849194 wait complete. 2023-07-06T20:18:27.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.551+0000 7f82a71ff700 1 Processor -- start 2023-07-06T20:18:27.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.551+0000 7f82a71ff700 1 -- start start 2023-07-06T20:18:27.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.551+0000 7f82a71ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a00ffc30 0x7f82a0197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:27.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.551+0000 7f82a71ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82a0100e20 0x7f82a0198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:27.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.551+0000 7f82a71ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82a01017b0 0x7f82a019f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:27.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.551+0000 7f82a71ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a019f790 con 0x7f82a01017b0 2023-07-06T20:18:27.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.551+0000 7f82a71ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a019f900 con 0x7f82a0100e20 2023-07-06T20:18:27.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.551+0000 7f82a71ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a019fac0 con 0x7f82a00ffc30 2023-07-06T20:18:27.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.551+0000 7f82a579c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82a01017b0 0x7f82a019f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:27.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.551+0000 7f8297fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82a0100e20 0x7f82a0198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:27.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f82a579c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82a01017b0 0x7f82a019f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59968/0 (socket says 172.21.15.69:59968) 2023-07-06T20:18:27.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f82a579c700 1 -- 172.21.15.69:0/3860446456 learned_addr learned my addr 172.21.15.69:0/3860446456 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:27.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f82a4f9b700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a00ffc30 0x7f82a0197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:27.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f82a579c700 1 -- 172.21.15.69:0/3860446456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a00ffc30 msgr2=0x7f82a0197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:27.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f82a579c700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a00ffc30 0x7f82a0197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f82a579c700 1 -- 172.21.15.69:0/3860446456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82a0100e20 msgr2=0x7f82a0198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:27.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f82a579c700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82a0100e20 0x7f82a0198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f82a579c700 1 -- 172.21.15.69:0/3860446456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8298009d00 con 0x7f82a01017b0 2023-07-06T20:18:27.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f82a579c700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82a01017b0 0x7f82a019f1c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f829800e3c0 tx=0x7f829800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:27.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f8295ffb700 1 -- 172.21.15.69:0/3860446456 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82980090d0 con 0x7f82a01017b0 2023-07-06T20:18:27.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f8295ffb700 1 -- 172.21.15.69:0/3860446456 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f829800f040 con 0x7f82a01017b0 2023-07-06T20:18:27.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f8295ffb700 1 -- 172.21.15.69:0/3860446456 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8298007c20 con 0x7f82a01017b0 2023-07-06T20:18:27.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f82a71ff700 1 -- 172.21.15.69:0/3860446456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8290009520 con 0x7f82a01017b0 2023-07-06T20:18:27.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.552+0000 7f82a71ff700 1 -- 172.21.15.69:0/3860446456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a01a0120 con 0x7f82a01017b0 2023-07-06T20:18:27.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.553+0000 7f82a71ff700 1 -- 172.21.15.69:0/3860446456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a0192310 con 0x7f82a01017b0 2023-07-06T20:18:27.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.555+0000 7f8295ffb700 1 -- 172.21.15.69:0/3860446456 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8298003a00 con 0x7f82a01017b0 2023-07-06T20:18:27.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.557+0000 7f8295ffb700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f82880a12a0 0x7f82880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:27.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.557+0000 7f8295ffb700 1 -- 172.21.15.69:0/3860446456 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f82980cc960 con 0x7f82a01017b0 2023-07-06T20:18:27.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.557+0000 7f82a4f9b700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f82880a12a0 0x7f82880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:27.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.557+0000 7f82a4f9b700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f82880a12a0 0x7f82880a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f82a0199120 tx=0x7f829c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:27.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.561+0000 7f8295ffb700 1 -- 172.21.15.69:0/3860446456 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8298095f70 con 0x7f82a01017b0 2023-07-06T20:18:27.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.768+0000 7f82a71ff700 1 -- 172.21.15.69:0/3860446456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82a002caf0 con 0x7f82a01017b0 2023-07-06T20:18:27.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.769+0000 7f8295ffb700 1 -- 172.21.15.69:0/3860446456 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22564 (secure 0 0 0) 0x7f8298099590 con 0x7f82a01017b0 2023-07-06T20:18:27.771 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:27.771 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":85,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:17:58.736720+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:18:27.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.772+0000 7f82a71ff700 1 -- 172.21.15.69:0/3860446456 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f82880a12a0 msgr2=0x7f82880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:27.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.772+0000 7f82a71ff700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f82880a12a0 0x7f82880a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f82a0199120 tx=0x7f829c007400 comp rx=0 tx=0).stop 2023-07-06T20:18:27.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.772+0000 7f82a71ff700 1 -- 172.21.15.69:0/3860446456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82a01017b0 msgr2=0x7f82a019f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:27.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.772+0000 7f82a71ff700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82a01017b0 0x7f82a019f1c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f829800e3c0 tx=0x7f829800e6d0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.772+0000 7f82a71ff700 1 -- 172.21.15.69:0/3860446456 shutdown_connections 2023-07-06T20:18:27.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.772+0000 7f82a71ff700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f82880a12a0 0x7f82880a3750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.772+0000 7f82a71ff700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82a01017b0 0x7f82a019f1c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.772+0000 7f82a71ff700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a00ffc30 0x7f82a0197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.772+0000 7f82a71ff700 1 --2- 172.21.15.69:0/3860446456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82a0100e20 0x7f82a0198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.772+0000 7f82a71ff700 1 -- 172.21.15.69:0/3860446456 >> 172.21.15.69:0/3860446456 conn(0x7f82a00fb210 msgr2=0x7f82a01066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:27.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.772+0000 7f82a71ff700 1 -- 172.21.15.69:0/3860446456 shutdown_connections 2023-07-06T20:18:27.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.772+0000 7f82a71ff700 1 -- 172.21.15.69:0/3860446456 wait complete. 2023-07-06T20:18:27.792 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:14.022799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:17:26.523183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:17:27.543386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:18:27.792 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:18:27.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.991+0000 7f75e6fff700 1 -- 172.21.15.69:0/3049302064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75e01017a0 msgr2=0x7f75e0108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:27.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.991+0000 7f75e6fff700 1 --2- 172.21.15.69:0/3049302064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75e01017a0 0x7f75e0108060 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f75d800b0a0 tx=0x7f75d800b3b0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.991+0000 7f75e6fff700 1 -- 172.21.15.69:0/3049302064 shutdown_connections 2023-07-06T20:18:27.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.991+0000 7f75e6fff700 1 --2- 172.21.15.69:0/3049302064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75e01017a0 0x7f75e0108060 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.991+0000 7f75e6fff700 1 --2- 172.21.15.69:0/3049302064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e00ffc20 0x7f75e0100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.994 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.991+0000 7f75e6fff700 1 --2- 172.21.15.69:0/3049302064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e0100e10 0x7f75e0101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.994 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.991+0000 7f75e6fff700 1 -- 172.21.15.69:0/3049302064 >> 172.21.15.69:0/3049302064 conn(0x7f75e00fb260 msgr2=0x7f75e00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:27.994 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.991+0000 7f75e6fff700 1 -- 172.21.15.69:0/3049302064 shutdown_connections 2023-07-06T20:18:27.994 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.991+0000 7f75e6fff700 1 -- 172.21.15.69:0/3049302064 wait complete. 2023-07-06T20:18:27.995 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.992+0000 7f75e6fff700 1 Processor -- start 2023-07-06T20:18:27.996 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.993+0000 7f75e6fff700 1 -- start start 2023-07-06T20:18:27.996 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.993+0000 7f75e6fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75e00ffc20 0x7f75e0195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:27.996 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.993+0000 7f75e6fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e0100e10 0x7f75e0195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:27.996 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.993+0000 7f75e6fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e01017a0 0x7f75e019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:27.996 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.993+0000 7f75e6fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e019d610 con 0x7f75e00ffc20 2023-07-06T20:18:27.996 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.993+0000 7f75e6fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e019d750 con 0x7f75e0100e10 2023-07-06T20:18:27.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.993+0000 7f75e6fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e019d8e0 con 0x7f75e01017a0 2023-07-06T20:18:27.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.993+0000 7f75e4d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75e00ffc20 0x7f75e0195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:27.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.993+0000 7f75e4d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75e00ffc20 0x7f75e0195a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59972/0 (socket says 172.21.15.69:59972) 2023-07-06T20:18:27.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.993+0000 7f75e4d9b700 1 -- 172.21.15.69:0/491411218 learned_addr learned my addr 172.21.15.69:0/491411218 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:27.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.993+0000 7f75e559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e01017a0 0x7f75e019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:27.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.993+0000 7f75dffff700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e0100e10 0x7f75e0195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:27.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.994+0000 7f75e4d9b700 1 -- 172.21.15.69:0/491411218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e01017a0 msgr2=0x7f75e019d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:27.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.994+0000 7f75e4d9b700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e01017a0 0x7f75e019d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.994+0000 7f75e4d9b700 1 -- 172.21.15.69:0/491411218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e0100e10 msgr2=0x7f75e0195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:27.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.994+0000 7f75e4d9b700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e0100e10 0x7f75e0195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:27.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.994+0000 7f75e4d9b700 1 -- 172.21.15.69:0/491411218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d8009d00 con 0x7f75e00ffc20 2023-07-06T20:18:27.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.994+0000 7f75e4d9b700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75e00ffc20 0x7f75e0195a90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f75d400d980 tx=0x7f75d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:27.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.994+0000 7f75ddffb700 1 -- 172.21.15.69:0/491411218 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4004020 con 0x7f75e00ffc20 2023-07-06T20:18:27.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.994+0000 7f75ddffb700 1 -- 172.21.15.69:0/491411218 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75d400f690 con 0x7f75e00ffc20 2023-07-06T20:18:27.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.994+0000 7f75ddffb700 1 -- 172.21.15.69:0/491411218 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4003a60 con 0x7f75e00ffc20 2023-07-06T20:18:27.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.994+0000 7f75e6fff700 1 -- 172.21.15.69:0/491411218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e019dbc0 con 0x7f75e00ffc20 2023-07-06T20:18:27.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.994+0000 7f75e6fff700 1 -- 172.21.15.69:0/491411218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e019e080 con 0x7f75e00ffc20 2023-07-06T20:18:27.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.995+0000 7f75e6fff700 1 -- 172.21.15.69:0/491411218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e0106ab0 con 0x7f75e00ffc20 2023-07-06T20:18:28.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.997+0000 7f75ddffb700 1 -- 172.21.15.69:0/491411218 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f75d400fb00 con 0x7f75e00ffc20 2023-07-06T20:18:28.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.998+0000 7f75ddffb700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f75d00a11e0 0x7f75d00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:28.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.999+0000 7f75ddffb700 1 -- 172.21.15.69:0/491411218 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f75d40cc700 con 0x7f75e00ffc20 2023-07-06T20:18:28.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.999+0000 7f75dffff700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f75d00a11e0 0x7f75d00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:28.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:27.999+0000 7f75dffff700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f75d00a11e0 0x7f75d00a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009c60 tx=0x7f75cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:28.003 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.003+0000 7f75ddffb700 1 -- 172.21.15.69:0/491411218 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f75d4095c80 con 0x7f75e00ffc20 2023-07-06T20:18:28.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.202+0000 7f75e6fff700 1 -- 172.21.15.69:0/491411218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75e002cb50 con 0x7f75e00ffc20 2023-07-06T20:18:28.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.203+0000 7f75ddffb700 1 -- 172.21.15.69:0/491411218 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75d40992a0 con 0x7f75e00ffc20 2023-07-06T20:18:28.204 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2023-07-06T20:18:28.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.205+0000 7f75e6fff700 1 -- 172.21.15.69:0/491411218 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f75d00a11e0 msgr2=0x7f75d00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:28.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.205+0000 7f75e6fff700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f75d00a11e0 0x7f75d00a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009c60 tx=0x7f75cc009260 comp rx=0 tx=0).stop 2023-07-06T20:18:28.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.205+0000 7f75e6fff700 1 -- 172.21.15.69:0/491411218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75e00ffc20 msgr2=0x7f75e0195a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:28.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.205+0000 7f75e6fff700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75e00ffc20 0x7f75e0195a90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f75d400d980 tx=0x7f75d400dc90 comp rx=0 tx=0).stop 2023-07-06T20:18:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.205+0000 7f75e6fff700 1 -- 172.21.15.69:0/491411218 shutdown_connections 2023-07-06T20:18:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.205+0000 7f75e6fff700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f75d00a11e0 0x7f75d00a3690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.205+0000 7f75e6fff700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e01017a0 0x7f75e019d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.205+0000 7f75e6fff700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75e00ffc20 0x7f75e0195a90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.205+0000 7f75e6fff700 1 --2- 172.21.15.69:0/491411218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e0100e10 0x7f75e0195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.205+0000 7f75e6fff700 1 -- 172.21.15.69:0/491411218 >> 172.21.15.69:0/491411218 conn(0x7f75e00fb260 msgr2=0x7f75e00fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:28.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.205+0000 7f75e6fff700 1 -- 172.21.15.69:0/491411218 shutdown_connections 2023-07-06T20:18:28.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.205+0000 7f75e6fff700 1 -- 172.21.15.69:0/491411218 wait complete. 2023-07-06T20:18:28.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:27 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/194086873' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-06T20:18:28.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:27 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3860446456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:28.225 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:18:28.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:27 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/194086873' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-06T20:18:28.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:27 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3860446456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:28.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:27 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/194086873' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-06T20:18:28.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:27 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3860446456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:28.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.422+0000 7fa49dbff700 1 -- 172.21.15.69:0/248179528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa498100e00 msgr2=0x7fa498101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:28.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.422+0000 7fa49dbff700 1 --2- 172.21.15.69:0/248179528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa498100e00 0x7fa498101250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009990 tx=0x7fa48c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.422+0000 7fa49dbff700 1 -- 172.21.15.69:0/248179528 shutdown_connections 2023-07-06T20:18:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.422+0000 7fa49dbff700 1 --2- 172.21.15.69:0/248179528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa498101790 0x7fa498108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.422+0000 7fa49dbff700 1 --2- 172.21.15.69:0/248179528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4980ffc10 0x7fa498100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.422+0000 7fa49dbff700 1 --2- 172.21.15.69:0/248179528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa498100e00 0x7fa498101250 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.422+0000 7fa49dbff700 1 -- 172.21.15.69:0/248179528 >> 172.21.15.69:0/248179528 conn(0x7fa4980fb210 msgr2=0x7fa4980fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.422+0000 7fa49dbff700 1 -- 172.21.15.69:0/248179528 shutdown_connections 2023-07-06T20:18:28.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.422+0000 7fa49dbff700 1 -- 172.21.15.69:0/248179528 wait complete. 2023-07-06T20:18:28.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.423+0000 7fa49dbff700 1 Processor -- start 2023-07-06T20:18:28.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.423+0000 7fa49dbff700 1 -- start start 2023-07-06T20:18:28.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.423+0000 7fa49dbff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4980ffc10 0x7fa498197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:28.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.423+0000 7fa49dbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa498100e00 0x7fa498198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:28.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.423+0000 7fa49dbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa498101790 0x7fa49819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:28.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.423+0000 7fa49dbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49819f740 con 0x7fa498101790 2023-07-06T20:18:28.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.423+0000 7fa49dbff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49819f880 con 0x7fa4980ffc10 2023-07-06T20:18:28.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.423+0000 7fa49dbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49819fa10 con 0x7fa498100e00 2023-07-06T20:18:28.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.424+0000 7fa497fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa498101790 0x7fa49819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:28.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.424+0000 7fa497fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa498101790 0x7fa49819f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59990/0 (socket says 172.21.15.69:59990) 2023-07-06T20:18:28.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.424+0000 7fa497fff700 1 -- 172.21.15.69:0/845640588 learned_addr learned my addr 172.21.15.69:0/845640588 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:28.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.424+0000 7fa4977fe700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4980ffc10 0x7fa498197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:28.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.424+0000 7fa496ffd700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa498100e00 0x7fa498198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:28.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.424+0000 7fa497fff700 1 -- 172.21.15.69:0/845640588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa498100e00 msgr2=0x7fa498198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:28.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.424+0000 7fa497fff700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa498100e00 0x7fa498198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.424+0000 7fa497fff700 1 -- 172.21.15.69:0/845640588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4980ffc10 msgr2=0x7fa498197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:28.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.424+0000 7fa497fff700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4980ffc10 0x7fa498197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.424+0000 7fa497fff700 1 -- 172.21.15.69:0/845640588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c0095f0 con 0x7fa498101790 2023-07-06T20:18:28.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.424+0000 7fa497fff700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa498101790 0x7fa49819f170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa49000ef30 tx=0x7fa49000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:28.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.424+0000 7fa494ff9700 1 -- 172.21.15.69:0/845640588 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490018070 con 0x7fa498101790 2023-07-06T20:18:28.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.425+0000 7fa494ff9700 1 -- 172.21.15.69:0/845640588 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa49000f040 con 0x7fa498101790 2023-07-06T20:18:28.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.425+0000 7fa49dbff700 1 -- 172.21.15.69:0/845640588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49819fc90 con 0x7fa498101790 2023-07-06T20:18:28.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.425+0000 7fa494ff9700 1 -- 172.21.15.69:0/845640588 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49000cd00 con 0x7fa498101790 2023-07-06T20:18:28.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.425+0000 7fa49dbff700 1 -- 172.21.15.69:0/845640588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4981a01a0 con 0x7fa498101790 2023-07-06T20:18:28.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.426+0000 7fa49dbff700 1 -- 172.21.15.69:0/845640588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49810b430 con 0x7fa498101790 2023-07-06T20:18:28.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.428+0000 7fa494ff9700 1 -- 172.21.15.69:0/845640588 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa490003a00 con 0x7fa498101790 2023-07-06T20:18:28.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.429+0000 7fa494ff9700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa4800a1260 0x7fa4800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:28.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.429+0000 7fa494ff9700 1 -- 172.21.15.69:0/845640588 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fa4900cc570 con 0x7fa498101790 2023-07-06T20:18:28.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.429+0000 7fa4977fe700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa4800a1260 0x7fa4800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:28.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.430+0000 7fa4977fe700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa4800a1260 0x7fa4800a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa498100c60 tx=0x7fa488009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:28.433 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.433+0000 7fa494ff9700 1 -- 172.21.15.69:0/845640588 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa490095b80 con 0x7fa498101790 2023-07-06T20:18:28.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.632+0000 7fa49dbff700 1 -- 172.21.15.69:0/845640588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4981a07b0 con 0x7fa498101790 2023-07-06T20:18:28.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.632+0000 7fa494ff9700 1 -- 172.21.15.69:0/845640588 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa4900991a0 con 0x7fa498101790 2023-07-06T20:18:28.633 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2023-07-06T20:18:28.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.634+0000 7fa49dbff700 1 -- 172.21.15.69:0/845640588 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa4800a1260 msgr2=0x7fa4800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:28.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.634+0000 7fa49dbff700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa4800a1260 0x7fa4800a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa498100c60 tx=0x7fa488009260 comp rx=0 tx=0).stop 2023-07-06T20:18:28.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.634+0000 7fa49dbff700 1 -- 172.21.15.69:0/845640588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa498101790 msgr2=0x7fa49819f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:28.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.634+0000 7fa49dbff700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa498101790 0x7fa49819f170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa49000ef30 tx=0x7fa49000c3a0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.634+0000 7fa49dbff700 1 -- 172.21.15.69:0/845640588 shutdown_connections 2023-07-06T20:18:28.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.634+0000 7fa49dbff700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa4800a1260 0x7fa4800a3710 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.634+0000 7fa49dbff700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa498101790 0x7fa49819f170 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.636 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.634+0000 7fa49dbff700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4980ffc10 0x7fa498197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.637 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.634+0000 7fa49dbff700 1 --2- 172.21.15.69:0/845640588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa498100e00 0x7fa498198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.637 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.634+0000 7fa49dbff700 1 -- 172.21.15.69:0/845640588 >> 172.21.15.69:0/845640588 conn(0x7fa4980fb210 msgr2=0x7fa4980fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:28.637 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.635+0000 7fa49dbff700 1 -- 172.21.15.69:0/845640588 shutdown_connections 2023-07-06T20:18:28.637 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.635+0000 7fa49dbff700 1 -- 172.21.15.69:0/845640588 wait complete. 2023-07-06T20:18:28.654 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:18:28.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.851+0000 7fbffd9ff700 1 -- 172.21.15.69:0/358294600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff80ffc20 msgr2=0x7fbff8100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:28.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.851+0000 7fbffd9ff700 1 --2- 172.21.15.69:0/358294600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff80ffc20 0x7fbff8100030 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009990 tx=0x7fbfe8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.851+0000 7fbffd9ff700 1 -- 172.21.15.69:0/358294600 shutdown_connections 2023-07-06T20:18:28.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.851+0000 7fbffd9ff700 1 --2- 172.21.15.69:0/358294600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff81017a0 0x7fbff8108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.851+0000 7fbffd9ff700 1 --2- 172.21.15.69:0/358294600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff80ffc20 0x7fbff8100030 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.851+0000 7fbffd9ff700 1 --2- 172.21.15.69:0/358294600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbff8100e10 0x7fbff8101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.851+0000 7fbffd9ff700 1 -- 172.21.15.69:0/358294600 >> 172.21.15.69:0/358294600 conn(0x7fbff80fb260 msgr2=0x7fbff80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:28.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.852+0000 7fbffd9ff700 1 -- 172.21.15.69:0/358294600 shutdown_connections 2023-07-06T20:18:28.854 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.852+0000 7fbffd9ff700 1 -- 172.21.15.69:0/358294600 wait complete. 2023-07-06T20:18:28.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbffd9ff700 1 Processor -- start 2023-07-06T20:18:28.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbffd9ff700 1 -- start start 2023-07-06T20:18:28.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbffd9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff80ffc20 0x7fbff80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:28.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbffd9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff8100e10 0x7fbff8072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:28.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbffd9ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbff81017a0 0x7fbff8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:28.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbffd9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff8073710 con 0x7fbff80ffc20 2023-07-06T20:18:28.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbffd9ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff8073880 con 0x7fbff81017a0 2023-07-06T20:18:28.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbffd9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff819fa60 con 0x7fbff8100e10 2023-07-06T20:18:28.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbff6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff80ffc20 0x7fbff80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:28.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbff6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff80ffc20 0x7fbff80743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60008/0 (socket says 172.21.15.69:60008) 2023-07-06T20:18:28.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbff6ffd700 1 -- 172.21.15.69:0/2496513973 learned_addr learned my addr 172.21.15.69:0/2496513973 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:28.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbff77fe700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbff81017a0 0x7fbff8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:28.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.853+0000 7fbff67fc700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff8100e10 0x7fbff8072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:28.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.854+0000 7fbff6ffd700 1 -- 172.21.15.69:0/2496513973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff8100e10 msgr2=0x7fbff8072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:28.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.854+0000 7fbff6ffd700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff8100e10 0x7fbff8072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.854+0000 7fbff6ffd700 1 -- 172.21.15.69:0/2496513973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbff81017a0 msgr2=0x7fbff8072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:28.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.854+0000 7fbff6ffd700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbff81017a0 0x7fbff8072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:28.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.854+0000 7fbff6ffd700 1 -- 172.21.15.69:0/2496513973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80095f0 con 0x7fbff80ffc20 2023-07-06T20:18:28.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.854+0000 7fbff6ffd700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff80ffc20 0x7fbff80743f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8004730 tx=0x7fbfe8004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:28.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.854+0000 7fbffc9fd700 1 -- 172.21.15.69:0/2496513973 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe801d070 con 0x7fbff80ffc20 2023-07-06T20:18:28.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.854+0000 7fbffc9fd700 1 -- 172.21.15.69:0/2496513973 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfe8021410 con 0x7fbff80ffc20 2023-07-06T20:18:28.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.854+0000 7fbffd9ff700 1 -- 172.21.15.69:0/2496513973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff819fce0 con 0x7fbff80ffc20 2023-07-06T20:18:28.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.854+0000 7fbffc9fd700 1 -- 172.21.15.69:0/2496513973 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8017650 con 0x7fbff80ffc20 2023-07-06T20:18:28.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.854+0000 7fbffd9ff700 1 -- 172.21.15.69:0/2496513973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff81a0160 con 0x7fbff80ffc20 2023-07-06T20:18:28.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.855+0000 7fbffd9ff700 1 -- 172.21.15.69:0/2496513973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff81071b0 con 0x7fbff80ffc20 2023-07-06T20:18:28.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.857+0000 7fbffc9fd700 1 -- 172.21.15.69:0/2496513973 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbfe8021810 con 0x7fbff80ffc20 2023-07-06T20:18:28.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.858+0000 7fbffc9fd700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbfe00a12e0 0x7fbfe00a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:28.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.859+0000 7fbffc9fd700 1 -- 172.21.15.69:0/2496513973 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fbfe80ce590 con 0x7fbff80ffc20 2023-07-06T20:18:28.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.859+0000 7fbff67fc700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbfe00a12e0 0x7fbfe00a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:28.860 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.859+0000 7fbff67fc700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbfe00a12e0 0x7fbfe00a3790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009bf0 tx=0x7fbfec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:28.861 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:28.861+0000 7fbffc9fd700 1 -- 172.21.15.69:0/2496513973 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbfe8097ba0 con 0x7fbff80ffc20 2023-07-06T20:18:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:28 smithi069 ceph-mon[126399]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:28 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/491411218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:18:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:28 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/845640588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:18:29.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.059+0000 7fbffd9ff700 1 -- 172.21.15.69:0/2496513973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbff802cd60 con 0x7fbff80ffc20 2023-07-06T20:18:29.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.060+0000 7fbffc9fd700 1 -- 172.21.15.69:0/2496513973 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbff802cd60 con 0x7fbff80ffc20 2023-07-06T20:18:29.061 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2023-07-06T20:18:29.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.062+0000 7fbffd9ff700 1 -- 172.21.15.69:0/2496513973 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbfe00a12e0 msgr2=0x7fbfe00a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:29.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.062+0000 7fbffd9ff700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbfe00a12e0 0x7fbfe00a3790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009bf0 tx=0x7fbfec009260 comp rx=0 tx=0).stop 2023-07-06T20:18:29.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.062+0000 7fbffd9ff700 1 -- 172.21.15.69:0/2496513973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff80ffc20 msgr2=0x7fbff80743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:29.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.062+0000 7fbffd9ff700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff80ffc20 0x7fbff80743f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8004730 tx=0x7fbfe8004760 comp rx=0 tx=0).stop 2023-07-06T20:18:29.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.062+0000 7fbffd9ff700 1 -- 172.21.15.69:0/2496513973 shutdown_connections 2023-07-06T20:18:29.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.062+0000 7fbffd9ff700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbfe00a12e0 0x7fbfe00a3790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.062+0000 7fbffd9ff700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbff81017a0 0x7fbff8072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.062+0000 7fbffd9ff700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff80ffc20 0x7fbff80743f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.062+0000 7fbffd9ff700 1 --2- 172.21.15.69:0/2496513973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff8100e10 0x7fbff8072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.062+0000 7fbffd9ff700 1 -- 172.21.15.69:0/2496513973 >> 172.21.15.69:0/2496513973 conn(0x7fbff80fb260 msgr2=0x7fbff81042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:29.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.063+0000 7fbffd9ff700 1 -- 172.21.15.69:0/2496513973 shutdown_connections 2023-07-06T20:18:29.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.063+0000 7fbffd9ff700 1 -- 172.21.15.69:0/2496513973 wait complete. 2023-07-06T20:18:29.082 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:18:29.277 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.276+0000 7ff3d23ff700 1 -- 172.21.15.69:0/4026733669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3cc0ffc30 msgr2=0x7ff3cc100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:29.277 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.276+0000 7ff3d23ff700 1 --2- 172.21.15.69:0/4026733669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3cc0ffc30 0x7ff3cc100040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009990 tx=0x7ff3bc009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.276+0000 7ff3d23ff700 1 -- 172.21.15.69:0/4026733669 shutdown_connections 2023-07-06T20:18:29.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.276+0000 7ff3d23ff700 1 --2- 172.21.15.69:0/4026733669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3cc1017b0 0x7ff3cc108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.276+0000 7ff3d23ff700 1 --2- 172.21.15.69:0/4026733669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3cc0ffc30 0x7ff3cc100040 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.276+0000 7ff3d23ff700 1 --2- 172.21.15.69:0/4026733669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3cc100e20 0x7ff3cc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.276+0000 7ff3d23ff700 1 -- 172.21.15.69:0/4026733669 >> 172.21.15.69:0/4026733669 conn(0x7ff3cc0fb210 msgr2=0x7ff3cc0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:29.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.276+0000 7ff3d23ff700 1 -- 172.21.15.69:0/4026733669 shutdown_connections 2023-07-06T20:18:29.278 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.276+0000 7ff3d23ff700 1 -- 172.21.15.69:0/4026733669 wait complete. 2023-07-06T20:18:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.277+0000 7ff3d23ff700 1 Processor -- start 2023-07-06T20:18:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.278+0000 7ff3d23ff700 1 -- start start 2023-07-06T20:18:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.278+0000 7ff3d23ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3cc0ffc30 0x7ff3cc195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.278+0000 7ff3d23ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3cc100e20 0x7ff3cc195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.278+0000 7ff3d23ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3cc1017b0 0x7ff3cc19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:29.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.278+0000 7ff3d23ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc19d4f0 con 0x7ff3cc100e20 2023-07-06T20:18:29.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.278+0000 7ff3d23ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc19d630 con 0x7ff3cc0ffc30 2023-07-06T20:18:29.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.278+0000 7ff3d23ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc19d7c0 con 0x7ff3cc1017b0 2023-07-06T20:18:29.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.278+0000 7ff3cb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3cc100e20 0x7ff3cc195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:29.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.278+0000 7ff3cb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3cc100e20 0x7ff3cc195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60026/0 (socket says 172.21.15.69:60026) 2023-07-06T20:18:29.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.278+0000 7ff3cb7fe700 1 -- 172.21.15.69:0/3213986918 learned_addr learned my addr 172.21.15.69:0/3213986918 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:29.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3d099c700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3cc1017b0 0x7ff3cc19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:29.281 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3cbfff700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3cc0ffc30 0x7ff3cc195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:29.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3cb7fe700 1 -- 172.21.15.69:0/3213986918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3cc1017b0 msgr2=0x7ff3cc19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:29.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3cb7fe700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3cc1017b0 0x7ff3cc19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3cb7fe700 1 -- 172.21.15.69:0/3213986918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3cc0ffc30 msgr2=0x7ff3cc195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:29.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3cb7fe700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3cc0ffc30 0x7ff3cc195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3cb7fe700 1 -- 172.21.15.69:0/3213986918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc0095f0 con 0x7ff3cc100e20 2023-07-06T20:18:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3cb7fe700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3cc100e20 0x7ff3cc195eb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000e9a0 tx=0x7ff3c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3c97fa700 1 -- 172.21.15.69:0/3213986918 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0009790 con 0x7ff3cc100e20 2023-07-06T20:18:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3c97fa700 1 -- 172.21.15.69:0/3213986918 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3c0010430 con 0x7ff3cc100e20 2023-07-06T20:18:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3d23ff700 1 -- 172.21.15.69:0/3213986918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3cc19daa0 con 0x7ff3cc100e20 2023-07-06T20:18:29.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3c97fa700 1 -- 172.21.15.69:0/3213986918 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0018680 con 0x7ff3cc100e20 2023-07-06T20:18:29.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.279+0000 7ff3d23ff700 1 -- 172.21.15.69:0/3213986918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3cc19dfb0 con 0x7ff3cc100e20 2023-07-06T20:18:29.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.280+0000 7ff3d23ff700 1 -- 172.21.15.69:0/3213986918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3cc10b450 con 0x7ff3cc100e20 2023-07-06T20:18:29.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.283+0000 7ff3c97fa700 1 -- 172.21.15.69:0/3213986918 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff3c00105a0 con 0x7ff3cc100e20 2023-07-06T20:18:29.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.284+0000 7ff3c97fa700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff3b40a1220 0x7ff3b40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:29.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.284+0000 7ff3c97fa700 1 -- 172.21.15.69:0/3213986918 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff3c0014070 con 0x7ff3cc100e20 2023-07-06T20:18:29.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.284+0000 7ff3cbfff700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff3b40a1220 0x7ff3b40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:29.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.285+0000 7ff3cbfff700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff3b40a1220 0x7ff3b40a36d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00b2e0 tx=0x7ff3bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:29.288 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.287+0000 7ff3c97fa700 1 -- 172.21.15.69:0/3213986918 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff3c0051770 con 0x7ff3cc100e20 2023-07-06T20:18:29.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:28 smithi156 ceph-mon[129280]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:29.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:28 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/491411218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:18:29.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:28 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/845640588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:18:29.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:28 smithi098 ceph-mon[128968]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:29.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:28 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/491411218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:18:29.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:28 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/845640588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:18:29.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.526+0000 7ff3d23ff700 1 -- 172.21.15.69:0/3213986918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3cc10b610 con 0x7ff3cc100e20 2023-07-06T20:18:29.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.527+0000 7ff3c97fa700 1 -- 172.21.15.69:0/3213986918 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10795 (secure 0 0 0) 0x7ff3c0096250 con 0x7ff3cc100e20 2023-07-06T20:18:29.528 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:29.528 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-06T20:17:28.144582+0000","modified":"2023-07-06T20:17:33.557511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24539,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/3235525224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3235525224},{"type":"v1","addr":"172.21.15.98:6839","nonce":3235525224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:18:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.529+0000 7ff3d23ff700 1 -- 172.21.15.69:0/3213986918 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff3b40a1220 msgr2=0x7ff3b40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.529+0000 7ff3d23ff700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff3b40a1220 0x7ff3b40a36d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00b2e0 tx=0x7ff3bc005fb0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.529+0000 7ff3d23ff700 1 -- 172.21.15.69:0/3213986918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3cc100e20 msgr2=0x7ff3cc195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.529+0000 7ff3d23ff700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3cc100e20 0x7ff3cc195eb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000e9a0 tx=0x7ff3c000ecb0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.529+0000 7ff3d23ff700 1 -- 172.21.15.69:0/3213986918 shutdown_connections 2023-07-06T20:18:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.529+0000 7ff3d23ff700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff3b40a1220 0x7ff3b40a36d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.529+0000 7ff3d23ff700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3cc1017b0 0x7ff3cc19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.529+0000 7ff3d23ff700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3cc0ffc30 0x7ff3cc195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.529+0000 7ff3d23ff700 1 --2- 172.21.15.69:0/3213986918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3cc100e20 0x7ff3cc195eb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.529+0000 7ff3d23ff700 1 -- 172.21.15.69:0/3213986918 >> 172.21.15.69:0/3213986918 conn(0x7ff3cc0fb210 msgr2=0x7ff3cc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:29.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.529+0000 7ff3d23ff700 1 -- 172.21.15.69:0/3213986918 shutdown_connections 2023-07-06T20:18:29.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.529+0000 7ff3d23ff700 1 -- 172.21.15.69:0/3213986918 wait complete. 2023-07-06T20:18:29.532 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2023-07-06T20:18:29.549 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:18:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.745+0000 7f3fff9ff700 1 -- 172.21.15.69:0/3967785695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff80ffc30 msgr2=0x7f3ff8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.745+0000 7f3fff9ff700 1 --2- 172.21.15.69:0/3967785695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff80ffc30 0x7f3ff8100040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009990 tx=0x7f3ff4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.745+0000 7f3fff9ff700 1 -- 172.21.15.69:0/3967785695 shutdown_connections 2023-07-06T20:18:29.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.745+0000 7f3fff9ff700 1 --2- 172.21.15.69:0/3967785695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ff81017b0 0x7f3ff8108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.745+0000 7f3fff9ff700 1 --2- 172.21.15.69:0/3967785695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff80ffc30 0x7f3ff8100040 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.745+0000 7f3fff9ff700 1 --2- 172.21.15.69:0/3967785695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff8100e20 0x7f3ff8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.745+0000 7f3fff9ff700 1 -- 172.21.15.69:0/3967785695 >> 172.21.15.69:0/3967785695 conn(0x7f3ff80fb210 msgr2=0x7f3ff80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.745+0000 7f3fff9ff700 1 -- 172.21.15.69:0/3967785695 shutdown_connections 2023-07-06T20:18:29.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.745+0000 7f3fff9ff700 1 -- 172.21.15.69:0/3967785695 wait complete. 2023-07-06T20:18:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.746+0000 7f3fff9ff700 1 Processor -- start 2023-07-06T20:18:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.746+0000 7f3fff9ff700 1 -- start start 2023-07-06T20:18:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.746+0000 7f3fff9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ff80ffc30 0x7f3ff8195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.746+0000 7f3fff9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff8100e20 0x7f3ff8195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.746+0000 7f3fff9ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff81017b0 0x7f3ff819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.746+0000 7f3fff9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff819d4e0 con 0x7f3ff8100e20 2023-07-06T20:18:29.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.746+0000 7f3fff9ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff819d620 con 0x7f3ff81017b0 2023-07-06T20:18:29.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.746+0000 7f3fff9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff819d7b0 con 0x7f3ff80ffc30 2023-07-06T20:18:29.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3ffcf9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff8100e20 0x7f3ff8195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:29.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3ffcf9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff8100e20 0x7f3ff8195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60050/0 (socket says 172.21.15.69:60050) 2023-07-06T20:18:29.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3ffcf9a700 1 -- 172.21.15.69:0/1917368597 learned_addr learned my addr 172.21.15.69:0/1917368597 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:29.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3ffdf9c700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff81017b0 0x7f3ff819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:29.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3ffcf9a700 1 -- 172.21.15.69:0/1917368597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ff80ffc30 msgr2=0x7f3ff8195960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T20:18:29.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3ffcf9a700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ff80ffc30 0x7f3ff8195960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3ffcf9a700 1 -- 172.21.15.69:0/1917368597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff81017b0 msgr2=0x7f3ff819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3ffcf9a700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff81017b0 0x7f3ff819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3ffcf9a700 1 -- 172.21.15.69:0/1917368597 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff40095f0 con 0x7f3ff8100e20 2023-07-06T20:18:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3ffcf9a700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff8100e20 0x7f3ff8195ea0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800e9a0 tx=0x7f3fe800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3fee7fc700 1 -- 172.21.15.69:0/1917368597 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8009790 con 0x7f3ff8100e20 2023-07-06T20:18:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3fee7fc700 1 -- 172.21.15.69:0/1917368597 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fe800cbd0 con 0x7f3ff8100e20 2023-07-06T20:18:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3fff9ff700 1 -- 172.21.15.69:0/1917368597 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff819da90 con 0x7f3ff8100e20 2023-07-06T20:18:29.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.747+0000 7f3fee7fc700 1 -- 172.21.15.69:0/1917368597 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8018680 con 0x7f3ff8100e20 2023-07-06T20:18:29.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.748+0000 7f3fff9ff700 1 -- 172.21.15.69:0/1917368597 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff819dfa0 con 0x7f3ff8100e20 2023-07-06T20:18:29.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.749+0000 7f3fff9ff700 1 -- 172.21.15.69:0/1917368597 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff810b450 con 0x7f3ff8100e20 2023-07-06T20:18:29.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.751+0000 7f3fee7fc700 1 -- 172.21.15.69:0/1917368597 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3fe800cd40 con 0x7f3ff8100e20 2023-07-06T20:18:29.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.752+0000 7f3fee7fc700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fe40a1220 0x7f3fe40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:29.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.752+0000 7f3fee7fc700 1 -- 172.21.15.69:0/1917368597 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f3fe8014070 con 0x7f3ff8100e20 2023-07-06T20:18:29.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.752+0000 7f3ffd79b700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fe40a1220 0x7f3fe40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:29.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.753+0000 7f3ffd79b700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fe40a1220 0x7f3fe40a36d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400b620 tx=0x7f3ff4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:29.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.756+0000 7f3fee7fc700 1 -- 172.21.15.69:0/1917368597 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3fe8051770 con 0x7f3ff8100e20 2023-07-06T20:18:29.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.956+0000 7f3fff9ff700 1 -- 172.21.15.69:0/1917368597 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ff8106ac0 con 0x7f3ff8100e20 2023-07-06T20:18:29.959 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.958+0000 7f3fee7fc700 1 -- 172.21.15.69:0/1917368597 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22564 (secure 0 0 0) 0x7f3fe8096330 con 0x7f3ff8100e20 2023-07-06T20:18:29.960 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:29.960 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":85,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:17:58.736720+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:18:29.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.960+0000 7f3fff9ff700 1 -- 172.21.15.69:0/1917368597 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fe40a1220 msgr2=0x7f3fe40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:29.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.960+0000 7f3fff9ff700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fe40a1220 0x7f3fe40a36d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400b620 tx=0x7f3ff4000bc0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.960+0000 7f3fff9ff700 1 -- 172.21.15.69:0/1917368597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff8100e20 msgr2=0x7f3ff8195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.960+0000 7f3fff9ff700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff8100e20 0x7f3ff8195ea0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800e9a0 tx=0x7f3fe800ecb0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.960+0000 7f3fff9ff700 1 -- 172.21.15.69:0/1917368597 shutdown_connections 2023-07-06T20:18:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.960+0000 7f3fff9ff700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fe40a1220 0x7f3fe40a36d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.960+0000 7f3fff9ff700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff81017b0 0x7f3ff819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.960+0000 7f3fff9ff700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ff80ffc30 0x7f3ff8195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.960+0000 7f3fff9ff700 1 --2- 172.21.15.69:0/1917368597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ff8100e20 0x7f3ff8195ea0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:29.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.960+0000 7f3fff9ff700 1 -- 172.21.15.69:0/1917368597 >> 172.21.15.69:0/1917368597 conn(0x7f3ff80fb210 msgr2=0x7f3ff80fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:29.964 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.960+0000 7f3fff9ff700 1 -- 172.21.15.69:0/1917368597 shutdown_connections 2023-07-06T20:18:29.964 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:29.960+0000 7f3fff9ff700 1 -- 172.21.15.69:0/1917368597 wait complete. 2023-07-06T20:18:29.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:14.022799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:17:26.523183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:17:27.543386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:18:29.982 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:18:30.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.177+0000 7f55a83ff700 1 -- 172.21.15.69:0/4099163687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a01017a0 msgr2=0x7f55a0108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:30.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.177+0000 7f55a83ff700 1 --2- 172.21.15.69:0/4099163687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a01017a0 0x7f55a0108060 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f559800b0a0 tx=0x7f559800b3b0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.177+0000 7f55a83ff700 1 -- 172.21.15.69:0/4099163687 shutdown_connections 2023-07-06T20:18:30.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.178+0000 7f55a83ff700 1 --2- 172.21.15.69:0/4099163687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a01017a0 0x7f55a0108060 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.178+0000 7f55a83ff700 1 --2- 172.21.15.69:0/4099163687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55a00ffc20 0x7f55a0100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.181 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.178+0000 7f55a83ff700 1 --2- 172.21.15.69:0/4099163687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55a0100e10 0x7f55a0101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.181 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.178+0000 7f55a83ff700 1 -- 172.21.15.69:0/4099163687 >> 172.21.15.69:0/4099163687 conn(0x7f55a00fb260 msgr2=0x7f55a00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:30.181 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.178+0000 7f55a83ff700 1 -- 172.21.15.69:0/4099163687 shutdown_connections 2023-07-06T20:18:30.181 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.178+0000 7f55a83ff700 1 -- 172.21.15.69:0/4099163687 wait complete. 2023-07-06T20:18:30.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.179+0000 7f55a83ff700 1 Processor -- start 2023-07-06T20:18:30.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.179+0000 7f55a83ff700 1 -- start start 2023-07-06T20:18:30.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a83ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55a00ffc20 0x7f55a00743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:30.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a83ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55a0100e10 0x7f55a0072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:30.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a83ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a01017a0 0x7f55a0072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:30.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a83ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a0073710 con 0x7f55a01017a0 2023-07-06T20:18:30.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a83ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a0073880 con 0x7f55a0100e10 2023-07-06T20:18:30.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a83ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a019fb50 con 0x7f55a00ffc20 2023-07-06T20:18:30.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a699c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a01017a0 0x7f55a0072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:30.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a699c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a01017a0 0x7f55a0072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60068/0 (socket says 172.21.15.69:60068) 2023-07-06T20:18:30.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a699c700 1 -- 172.21.15.69:0/4058233249 learned_addr learned my addr 172.21.15.69:0/4058233249 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:30.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a619b700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55a00ffc20 0x7f55a00743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:30.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a599a700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55a0100e10 0x7f55a0072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:30.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a699c700 1 -- 172.21.15.69:0/4058233249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55a00ffc20 msgr2=0x7f55a00743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:30.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a699c700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55a00ffc20 0x7f55a00743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a699c700 1 -- 172.21.15.69:0/4058233249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55a0100e10 msgr2=0x7f55a0072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:30.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a699c700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55a0100e10 0x7f55a0072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.180+0000 7f55a699c700 1 -- 172.21.15.69:0/4058233249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5598009d00 con 0x7f55a01017a0 2023-07-06T20:18:30.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.181+0000 7f55a699c700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a01017a0 0x7f55a0072f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f559800d010 tx=0x7f559800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:30.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.181+0000 7f55977fe700 1 -- 172.21.15.69:0/4058233249 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559800e040 con 0x7f55a01017a0 2023-07-06T20:18:30.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.181+0000 7f55a83ff700 1 -- 172.21.15.69:0/4058233249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a019fdd0 con 0x7f55a01017a0 2023-07-06T20:18:30.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.181+0000 7f55977fe700 1 -- 172.21.15.69:0/4058233249 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f559800bdb0 con 0x7f55a01017a0 2023-07-06T20:18:30.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.181+0000 7f55977fe700 1 -- 172.21.15.69:0/4058233249 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598005260 con 0x7f55a01017a0 2023-07-06T20:18:30.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.181+0000 7f55a83ff700 1 -- 172.21.15.69:0/4058233249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a01a01a0 con 0x7f55a01017a0 2023-07-06T20:18:30.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.182+0000 7f55a83ff700 1 -- 172.21.15.69:0/4058233249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a0106ab0 con 0x7f55a01017a0 2023-07-06T20:18:30.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.184+0000 7f55977fe700 1 -- 172.21.15.69:0/4058233249 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f559801d480 con 0x7f55a01017a0 2023-07-06T20:18:30.187 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.185+0000 7f55977fe700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f558c0a11e0 0x7f558c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:30.187 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.185+0000 7f55977fe700 1 -- 172.21.15.69:0/4058233249 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f55980cdf20 con 0x7f55a01017a0 2023-07-06T20:18:30.187 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.185+0000 7f55a619b700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f558c0a11e0 0x7f558c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:30.188 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.186+0000 7f55a619b700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f558c0a11e0 0x7f558c0a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f55a0100c70 tx=0x7f559c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.189+0000 7f55977fe700 1 -- 172.21.15.69:0/4058233249 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5598097530 con 0x7f55a01017a0 2023-07-06T20:18:30.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:29 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2496513973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:18:30.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:29 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3213986918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:30.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:29 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2496513973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:18:30.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:29 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3213986918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:30.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:29 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2496513973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:18:30.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:29 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3213986918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:30.387 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.386+0000 7f55a83ff700 1 -- 172.21.15.69:0/4058233249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55a010b4e0 con 0x7f55a01017a0 2023-07-06T20:18:30.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.386+0000 7f55977fe700 1 -- 172.21.15.69:0/4058233249 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f559809ab50 con 0x7f55a01017a0 2023-07-06T20:18:30.388 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2023-07-06T20:18:30.390 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.388+0000 7f55a83ff700 1 -- 172.21.15.69:0/4058233249 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f558c0a11e0 msgr2=0x7f558c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:30.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.388+0000 7f55a83ff700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f558c0a11e0 0x7f558c0a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f55a0100c70 tx=0x7f559c009190 comp rx=0 tx=0).stop 2023-07-06T20:18:30.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.388+0000 7f55a83ff700 1 -- 172.21.15.69:0/4058233249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a01017a0 msgr2=0x7f55a0072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:30.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.388+0000 7f55a83ff700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a01017a0 0x7f55a0072f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f559800d010 tx=0x7f559800bc80 comp rx=0 tx=0).stop 2023-07-06T20:18:30.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.389+0000 7f55a83ff700 1 -- 172.21.15.69:0/4058233249 shutdown_connections 2023-07-06T20:18:30.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.389+0000 7f55a83ff700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f558c0a11e0 0x7f558c0a3690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.389+0000 7f55a83ff700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55a01017a0 0x7f55a0072f70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.389+0000 7f55a83ff700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55a00ffc20 0x7f55a00743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.389+0000 7f55a83ff700 1 --2- 172.21.15.69:0/4058233249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55a0100e10 0x7f55a0072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.389+0000 7f55a83ff700 1 -- 172.21.15.69:0/4058233249 >> 172.21.15.69:0/4058233249 conn(0x7f55a00fb260 msgr2=0x7f55a00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:30.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.389+0000 7f55a83ff700 1 -- 172.21.15.69:0/4058233249 shutdown_connections 2023-07-06T20:18:30.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.389+0000 7f55a83ff700 1 -- 172.21.15.69:0/4058233249 wait complete. 2023-07-06T20:18:30.409 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:18:30.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.606+0000 7f16f1fff700 1 -- 172.21.15.69:0/3330679900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ec100e10 msgr2=0x7f16ec101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:30.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.606+0000 7f16f1fff700 1 --2- 172.21.15.69:0/3330679900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ec100e10 0x7f16ec101260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009990 tx=0x7f16e0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.606+0000 7f16f1fff700 1 -- 172.21.15.69:0/3330679900 shutdown_connections 2023-07-06T20:18:30.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.606+0000 7f16f1fff700 1 --2- 172.21.15.69:0/3330679900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16ec1017a0 0x7f16ec108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.606+0000 7f16f1fff700 1 --2- 172.21.15.69:0/3330679900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16ec0ffc20 0x7f16ec100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.606+0000 7f16f1fff700 1 --2- 172.21.15.69:0/3330679900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ec100e10 0x7f16ec101260 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.606+0000 7f16f1fff700 1 -- 172.21.15.69:0/3330679900 >> 172.21.15.69:0/3330679900 conn(0x7f16ec0fb260 msgr2=0x7f16ec0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:30.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.606+0000 7f16f1fff700 1 -- 172.21.15.69:0/3330679900 shutdown_connections 2023-07-06T20:18:30.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.606+0000 7f16f1fff700 1 -- 172.21.15.69:0/3330679900 wait complete. 2023-07-06T20:18:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.607+0000 7f16f1fff700 1 Processor -- start 2023-07-06T20:18:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.607+0000 7f16f1fff700 1 -- start start 2023-07-06T20:18:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.607+0000 7f16f1fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16ec0ffc20 0x7f16ec197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.607+0000 7f16f1fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ec100e10 0x7f16ec198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:30.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.607+0000 7f16f1fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16ec1017a0 0x7f16ec19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:30.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.607+0000 7f16f1fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec19f790 con 0x7f16ec100e10 2023-07-06T20:18:30.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.607+0000 7f16f1fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec19f900 con 0x7f16ec1017a0 2023-07-06T20:18:30.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.607+0000 7f16f1fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec19fac0 con 0x7f16ec0ffc20 2023-07-06T20:18:30.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.608+0000 7f16eaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ec100e10 0x7f16ec198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:30.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.608+0000 7f16eaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ec100e10 0x7f16ec198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60080/0 (socket says 172.21.15.69:60080) 2023-07-06T20:18:30.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.608+0000 7f16eaffd700 1 -- 172.21.15.69:0/1982510674 learned_addr learned my addr 172.21.15.69:0/1982510674 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:30.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.608+0000 7f16eb7fe700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16ec0ffc20 0x7f16ec197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:30.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.608+0000 7f16ebfff700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16ec1017a0 0x7f16ec19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:30.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.608+0000 7f16eaffd700 1 -- 172.21.15.69:0/1982510674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16ec0ffc20 msgr2=0x7f16ec197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:30.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.608+0000 7f16eaffd700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16ec0ffc20 0x7f16ec197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.608+0000 7f16eaffd700 1 -- 172.21.15.69:0/1982510674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16ec1017a0 msgr2=0x7f16ec19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.608+0000 7f16eaffd700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16ec1017a0 0x7f16ec19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.608+0000 7f16eaffd700 1 -- 172.21.15.69:0/1982510674 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e4009c40 con 0x7f16ec100e10 2023-07-06T20:18:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.608+0000 7f16ebfff700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16ec1017a0 0x7f16ec19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.608+0000 7f16eaffd700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ec100e10 0x7f16ec198190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f16e0004760 tx=0x7f16e0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.609+0000 7f16e8ff9700 1 -- 172.21.15.69:0/1982510674 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e001d070 con 0x7f16ec100e10 2023-07-06T20:18:30.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.609+0000 7f16f1fff700 1 -- 172.21.15.69:0/1982510674 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e00095f0 con 0x7f16ec100e10 2023-07-06T20:18:30.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.609+0000 7f16e8ff9700 1 -- 172.21.15.69:0/1982510674 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16e0004830 con 0x7f16ec100e10 2023-07-06T20:18:30.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.609+0000 7f16e8ff9700 1 -- 172.21.15.69:0/1982510674 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e0005480 con 0x7f16ec100e10 2023-07-06T20:18:30.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.609+0000 7f16f1fff700 1 -- 172.21.15.69:0/1982510674 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec19ffd0 con 0x7f16ec100e10 2023-07-06T20:18:30.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.610+0000 7f16f1fff700 1 -- 172.21.15.69:0/1982510674 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ec10b440 con 0x7f16ec100e10 2023-07-06T20:18:30.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.612+0000 7f16e8ff9700 1 -- 172.21.15.69:0/1982510674 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f16e000fad0 con 0x7f16ec100e10 2023-07-06T20:18:30.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.614+0000 7f16e8ff9700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f16d40a11e0 0x7f16d40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:30.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.614+0000 7f16e8ff9700 1 -- 172.21.15.69:0/1982510674 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f16e00cdca0 con 0x7f16ec100e10 2023-07-06T20:18:30.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.614+0000 7f16eb7fe700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f16d40a11e0 0x7f16d40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:30.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.615+0000 7f16eb7fe700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f16d40a11e0 0x7f16d40a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f16ec199120 tx=0x7f16dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:30.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.618+0000 7f16e8ff9700 1 -- 172.21.15.69:0/1982510674 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f16e00972b0 con 0x7f16ec100e10 2023-07-06T20:18:30.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.836+0000 7f16f1fff700 1 -- 172.21.15.69:0/1982510674 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f16ec1a05e0 con 0x7f16ec100e10 2023-07-06T20:18:30.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.836+0000 7f16e8ff9700 1 -- 172.21.15.69:0/1982510674 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f16e009a8d0 con 0x7f16ec100e10 2023-07-06T20:18:30.838 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2023-07-06T20:18:30.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.839+0000 7f16f1fff700 1 -- 172.21.15.69:0/1982510674 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f16d40a11e0 msgr2=0x7f16d40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:30.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.839+0000 7f16f1fff700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f16d40a11e0 0x7f16d40a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f16ec199120 tx=0x7f16dc009260 comp rx=0 tx=0).stop 2023-07-06T20:18:30.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.839+0000 7f16f1fff700 1 -- 172.21.15.69:0/1982510674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ec100e10 msgr2=0x7f16ec198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:30.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.839+0000 7f16f1fff700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ec100e10 0x7f16ec198190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f16e0004760 tx=0x7f16e0004790 comp rx=0 tx=0).stop 2023-07-06T20:18:30.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.839+0000 7f16f1fff700 1 -- 172.21.15.69:0/1982510674 shutdown_connections 2023-07-06T20:18:30.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.839+0000 7f16f1fff700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f16d40a11e0 0x7f16d40a3690 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.839+0000 7f16f1fff700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16ec1017a0 0x7f16ec19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.839+0000 7f16f1fff700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16ec0ffc20 0x7f16ec197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.839+0000 7f16f1fff700 1 --2- 172.21.15.69:0/1982510674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16ec100e10 0x7f16ec198190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:30.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.839+0000 7f16f1fff700 1 -- 172.21.15.69:0/1982510674 >> 172.21.15.69:0/1982510674 conn(0x7f16ec0fb260 msgr2=0x7f16ec0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:30.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.839+0000 7f16f1fff700 1 -- 172.21.15.69:0/1982510674 shutdown_connections 2023-07-06T20:18:30.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:30.839+0000 7f16f1fff700 1 -- 172.21.15.69:0/1982510674 wait complete. 2023-07-06T20:18:30.860 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:18:31.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.070+0000 7f87d47ff700 1 -- 172.21.15.69:0/2756623600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87cc1017b0 msgr2=0x7f87cc108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:31.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.070+0000 7f87d47ff700 1 --2- 172.21.15.69:0/2756623600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87cc1017b0 0x7f87cc108070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009f90 tx=0x7f87c400b330 comp rx=0 tx=0).stop 2023-07-06T20:18:31.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.070+0000 7f87d47ff700 1 -- 172.21.15.69:0/2756623600 shutdown_connections 2023-07-06T20:18:31.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.070+0000 7f87d47ff700 1 --2- 172.21.15.69:0/2756623600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87cc1017b0 0x7f87cc108070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.070+0000 7f87d47ff700 1 --2- 172.21.15.69:0/2756623600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc0ffc30 0x7f87cc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.070+0000 7f87d47ff700 1 --2- 172.21.15.69:0/2756623600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc100e20 0x7f87cc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.070+0000 7f87d47ff700 1 -- 172.21.15.69:0/2756623600 >> 172.21.15.69:0/2756623600 conn(0x7f87cc0fb250 msgr2=0x7f87cc0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:31.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.070+0000 7f87d47ff700 1 -- 172.21.15.69:0/2756623600 shutdown_connections 2023-07-06T20:18:31.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.070+0000 7f87d47ff700 1 -- 172.21.15.69:0/2756623600 wait complete. 2023-07-06T20:18:31.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.072+0000 7f87d47ff700 1 Processor -- start 2023-07-06T20:18:31.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.072+0000 7f87d47ff700 1 -- start start 2023-07-06T20:18:31.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.072+0000 7f87d47ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87cc0ffc30 0x7f87cc195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:31.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.072+0000 7f87d47ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc100e20 0x7f87cc195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:31.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.072+0000 7f87d47ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc1017b0 0x7f87cc19d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:31.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d47ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc19d620 con 0x7f87cc1017b0 2023-07-06T20:18:31.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d47ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc19d760 con 0x7f87cc0ffc30 2023-07-06T20:18:31.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d47ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc19d8f0 con 0x7f87cc100e20 2023-07-06T20:18:31.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d2d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc1017b0 0x7f87cc19d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:31.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d2d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc1017b0 0x7f87cc19d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60110/0 (socket says 172.21.15.69:60110) 2023-07-06T20:18:31.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d2d9c700 1 -- 172.21.15.69:0/1070407383 learned_addr learned my addr 172.21.15.69:0/1070407383 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:31.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d259b700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87cc0ffc30 0x7f87cc195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:31.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d1d9a700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc100e20 0x7f87cc195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:31.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d2d9c700 1 -- 172.21.15.69:0/1070407383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc100e20 msgr2=0x7f87cc195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:31.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d2d9c700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc100e20 0x7f87cc195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d2d9c700 1 -- 172.21.15.69:0/1070407383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87cc0ffc30 msgr2=0x7f87cc195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:31.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d2d9c700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87cc0ffc30 0x7f87cc195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d2d9c700 1 -- 172.21.15.69:0/1070407383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c80095f0 con 0x7f87cc1017b0 2023-07-06T20:18:31.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d1d9a700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc100e20 0x7f87cc195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:31.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.073+0000 7f87d2d9c700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc1017b0 0x7f87cc19d050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f87c4007930 tx=0x7f87c4004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:31.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.074+0000 7f87c37fe700 1 -- 172.21.15.69:0/1070407383 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c400e040 con 0x7f87cc1017b0 2023-07-06T20:18:31.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.074+0000 7f87c37fe700 1 -- 172.21.15.69:0/1070407383 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87c400bda0 con 0x7f87cc1017b0 2023-07-06T20:18:31.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.074+0000 7f87c37fe700 1 -- 172.21.15.69:0/1070407383 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c40052a0 con 0x7f87cc1017b0 2023-07-06T20:18:31.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.074+0000 7f87d47ff700 1 -- 172.21.15.69:0/1070407383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c4009c40 con 0x7f87cc1017b0 2023-07-06T20:18:31.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.074+0000 7f87d47ff700 1 -- 172.21.15.69:0/1070407383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc19de10 con 0x7f87cc1017b0 2023-07-06T20:18:31.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.075+0000 7f87d47ff700 1 -- 172.21.15.69:0/1070407383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87cc107150 con 0x7f87cc1017b0 2023-07-06T20:18:31.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.077+0000 7f87c37fe700 1 -- 172.21.15.69:0/1070407383 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f87c401d470 con 0x7f87cc1017b0 2023-07-06T20:18:31.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.078+0000 7f87c37fe700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f87b80a1260 0x7f87b80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:31.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.078+0000 7f87c37fe700 1 -- 172.21.15.69:0/1070407383 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f87c40ce210 con 0x7f87cc1017b0 2023-07-06T20:18:31.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.078+0000 7f87d259b700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f87b80a1260 0x7f87b80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:31.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.079+0000 7f87d259b700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f87b80a1260 0x7f87b80a3710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f87cc100c80 tx=0x7f87c8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:31.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.085+0000 7f87c37fe700 1 -- 172.21.15.69:0/1070407383 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f87c4097820 con 0x7f87cc1017b0 2023-07-06T20:18:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:30 smithi069 ceph-mon[126399]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-06T20:18:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:30 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1917368597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:30 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/4058233249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:18:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:30 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:18:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:30 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1982510674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:18:31.324 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.322+0000 7f87d47ff700 1 -- 172.21.15.69:0/1070407383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87cc02cd40 con 0x7f87cc1017b0 2023-07-06T20:18:31.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.323+0000 7f87c37fe700 1 -- 172.21.15.69:0/1070407383 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87c4008f30 con 0x7f87cc1017b0 2023-07-06T20:18:31.325 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2023-07-06T20:18:31.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.325+0000 7f87d47ff700 1 -- 172.21.15.69:0/1070407383 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f87b80a1260 msgr2=0x7f87b80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:31.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.325+0000 7f87d47ff700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f87b80a1260 0x7f87b80a3710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f87cc100c80 tx=0x7f87c8009310 comp rx=0 tx=0).stop 2023-07-06T20:18:31.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.325+0000 7f87d47ff700 1 -- 172.21.15.69:0/1070407383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc1017b0 msgr2=0x7f87cc19d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.325+0000 7f87d47ff700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc1017b0 0x7f87cc19d050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f87c4007930 tx=0x7f87c4004c60 comp rx=0 tx=0).stop 2023-07-06T20:18:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.325+0000 7f87d47ff700 1 -- 172.21.15.69:0/1070407383 shutdown_connections 2023-07-06T20:18:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.325+0000 7f87d47ff700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f87b80a1260 0x7f87b80a3710 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.325+0000 7f87d47ff700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87cc1017b0 0x7f87cc19d050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.325+0000 7f87d47ff700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87cc0ffc30 0x7f87cc195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.325+0000 7f87d47ff700 1 --2- 172.21.15.69:0/1070407383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87cc100e20 0x7f87cc195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.325+0000 7f87d47ff700 1 -- 172.21.15.69:0/1070407383 >> 172.21.15.69:0/1070407383 conn(0x7f87cc0fb250 msgr2=0x7f87cc104290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.325+0000 7f87d47ff700 1 -- 172.21.15.69:0/1070407383 shutdown_connections 2023-07-06T20:18:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.325+0000 7f87d47ff700 1 -- 172.21.15.69:0/1070407383 wait complete. 2023-07-06T20:18:31.346 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:18:31.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:30 smithi156 ceph-mon[129280]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-06T20:18:31.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:30 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1917368597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:31.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:30 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/4058233249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:18:31.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:30 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:18:31.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:30 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1982510674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:18:31.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:30 smithi098 ceph-mon[128968]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-06T20:18:31.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:30 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1917368597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:31.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:30 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/4058233249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:18:31.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:30 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:18:31.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:30 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1982510674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:18:31.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.549+0000 7f4df25ff700 1 -- 172.21.15.69:0/3888828496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dec073020 msgr2=0x7f4dec073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:31.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.549+0000 7f4df25ff700 1 --2- 172.21.15.69:0/3888828496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dec073020 0x7f4dec073490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4de00098f0 tx=0x7f4de0009c00 comp rx=0 tx=0).stop 2023-07-06T20:18:31.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.549+0000 7f4df25ff700 1 -- 172.21.15.69:0/3888828496 shutdown_connections 2023-07-06T20:18:31.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.549+0000 7f4df25ff700 1 --2- 172.21.15.69:0/3888828496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec107ae0 0x7f4dec109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.549+0000 7f4df25ff700 1 --2- 172.21.15.69:0/3888828496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4dec074600 0x7f4dec072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.549+0000 7f4df25ff700 1 --2- 172.21.15.69:0/3888828496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dec073020 0x7f4dec073490 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.549+0000 7f4df25ff700 1 -- 172.21.15.69:0/3888828496 >> 172.21.15.69:0/3888828496 conn(0x7f4dec0fb210 msgr2=0x7f4dec0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:31.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.549+0000 7f4df25ff700 1 -- 172.21.15.69:0/3888828496 shutdown_connections 2023-07-06T20:18:31.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.549+0000 7f4df25ff700 1 -- 172.21.15.69:0/3888828496 wait complete. 2023-07-06T20:18:31.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.550+0000 7f4df25ff700 1 Processor -- start 2023-07-06T20:18:31.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.550+0000 7f4df25ff700 1 -- start start 2023-07-06T20:18:31.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.550+0000 7f4df25ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec073020 0x7f4dec197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:31.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.550+0000 7f4df25ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4dec074600 0x7f4dec198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:31.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.550+0000 7f4df25ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dec107ae0 0x7f4dec19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:31.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.550+0000 7f4df25ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec19f760 con 0x7f4dec107ae0 2023-07-06T20:18:31.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.550+0000 7f4df25ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec19f8a0 con 0x7f4dec074600 2023-07-06T20:18:31.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.550+0000 7f4df25ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec19fa30 con 0x7f4dec073020 2023-07-06T20:18:31.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.551+0000 7f4df0b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dec107ae0 0x7f4dec19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:31.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.551+0000 7f4debfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec073020 0x7f4dec197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:31.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.551+0000 7f4df0b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dec107ae0 0x7f4dec19f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60132/0 (socket says 172.21.15.69:60132) 2023-07-06T20:18:31.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.551+0000 7f4df0b9c700 1 -- 172.21.15.69:0/745663563 learned_addr learned my addr 172.21.15.69:0/745663563 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:31.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.551+0000 7f4deb7fe700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4dec074600 0x7f4dec198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:31.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.551+0000 7f4debfff700 1 -- 172.21.15.69:0/745663563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4dec074600 msgr2=0x7f4dec198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:31.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.551+0000 7f4debfff700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4dec074600 0x7f4dec198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.551+0000 7f4debfff700 1 -- 172.21.15.69:0/745663563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dec107ae0 msgr2=0x7f4dec19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:31.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.551+0000 7f4debfff700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dec107ae0 0x7f4dec19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.551+0000 7f4debfff700 1 -- 172.21.15.69:0/745663563 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de0009520 con 0x7f4dec073020 2023-07-06T20:18:31.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.551+0000 7f4df0b9c700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dec107ae0 0x7f4dec19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:31.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.552+0000 7f4debfff700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec073020 0x7f4dec197be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00e9c0 tx=0x7f4ddc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:31.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.552+0000 7f4de97fa700 1 -- 172.21.15.69:0/745663563 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc00c9c0 con 0x7f4dec073020 2023-07-06T20:18:31.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.552+0000 7f4df25ff700 1 -- 172.21.15.69:0/745663563 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec19fd10 con 0x7f4dec073020 2023-07-06T20:18:31.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.552+0000 7f4de97fa700 1 -- 172.21.15.69:0/745663563 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ddc010430 con 0x7f4dec073020 2023-07-06T20:18:31.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.552+0000 7f4de97fa700 1 -- 172.21.15.69:0/745663563 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc018680 con 0x7f4dec073020 2023-07-06T20:18:31.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.553+0000 7f4df25ff700 1 -- 172.21.15.69:0/745663563 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dec1a0240 con 0x7f4dec073020 2023-07-06T20:18:31.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.553+0000 7f4df25ff700 1 -- 172.21.15.69:0/745663563 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dec106220 con 0x7f4dec073020 2023-07-06T20:18:31.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.555+0000 7f4de97fa700 1 -- 172.21.15.69:0/745663563 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4ddc010860 con 0x7f4dec073020 2023-07-06T20:18:31.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.557+0000 7f4de97fa700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4dd40a12a0 0x7f4dd40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:31.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.557+0000 7f4de97fa700 1 -- 172.21.15.69:0/745663563 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f4ddc014070 con 0x7f4dec073020 2023-07-06T20:18:31.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.557+0000 7f4deb7fe700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4dd40a12a0 0x7f4dd40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:31.560 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.558+0000 7f4deb7fe700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4dd40a12a0 0x7f4dd40a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4de000fa70 tx=0x7f4de0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:31.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.560+0000 7f4de97fa700 1 -- 172.21.15.69:0/745663563 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4ddc096190 con 0x7f4dec073020 2023-07-06T20:18:31.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.827+0000 7f4df25ff700 1 -- 172.21.15.69:0/745663563 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dec02c8e0 con 0x7f4dec073020 2023-07-06T20:18:31.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.828+0000 7f4de97fa700 1 -- 172.21.15.69:0/745663563 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10795 (secure 0 0 0) 0x7f4ddc0997b0 con 0x7f4dec073020 2023-07-06T20:18:31.831 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:31.831 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-06T20:17:28.144582+0000","modified":"2023-07-06T20:17:33.557511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24539,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/3235525224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3235525224},{"type":"v1","addr":"172.21.15.98:6839","nonce":3235525224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:18:31.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.831+0000 7f4df25ff700 1 -- 172.21.15.69:0/745663563 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4dd40a12a0 msgr2=0x7f4dd40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:31.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.831+0000 7f4df25ff700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4dd40a12a0 0x7f4dd40a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4de000fa70 tx=0x7f4de0019040 comp rx=0 tx=0).stop 2023-07-06T20:18:31.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.831+0000 7f4df25ff700 1 -- 172.21.15.69:0/745663563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec073020 msgr2=0x7f4dec197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:31.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.831+0000 7f4df25ff700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec073020 0x7f4dec197be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00e9c0 tx=0x7f4ddc00ecd0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.831+0000 7f4df25ff700 1 -- 172.21.15.69:0/745663563 shutdown_connections 2023-07-06T20:18:31.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.831+0000 7f4df25ff700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4dd40a12a0 0x7f4dd40a3750 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.831+0000 7f4df25ff700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dec107ae0 0x7f4dec19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.831+0000 7f4df25ff700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dec073020 0x7f4dec197be0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.831+0000 7f4df25ff700 1 --2- 172.21.15.69:0/745663563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4dec074600 0x7f4dec198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:31.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.831+0000 7f4df25ff700 1 -- 172.21.15.69:0/745663563 >> 172.21.15.69:0/745663563 conn(0x7f4dec0fb210 msgr2=0x7f4dec0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:31.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.831+0000 7f4df25ff700 1 -- 172.21.15.69:0/745663563 shutdown_connections 2023-07-06T20:18:31.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:31.831+0000 7f4df25ff700 1 -- 172.21.15.69:0/745663563 wait complete. 2023-07-06T20:18:31.835 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2023-07-06T20:18:31.852 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:18:32.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.074+0000 7f22ccfff700 1 -- 172.21.15.69:0/3351148654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c81017b0 msgr2=0x7f22c8108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:32.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.074+0000 7f22ccfff700 1 --2- 172.21.15.69:0/3351148654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c81017b0 0x7f22c8108070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f22c000b0a0 tx=0x7f22c000b3b0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.074+0000 7f22ccfff700 1 -- 172.21.15.69:0/3351148654 shutdown_connections 2023-07-06T20:18:32.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.074+0000 7f22ccfff700 1 --2- 172.21.15.69:0/3351148654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c81017b0 0x7f22c8108070 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.074+0000 7f22ccfff700 1 --2- 172.21.15.69:0/3351148654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22c80ffc30 0x7f22c8100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.074+0000 7f22ccfff700 1 --2- 172.21.15.69:0/3351148654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22c8100e20 0x7f22c8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.074+0000 7f22ccfff700 1 -- 172.21.15.69:0/3351148654 >> 172.21.15.69:0/3351148654 conn(0x7f22c80fb210 msgr2=0x7f22c80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:32.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.074+0000 7f22ccfff700 1 -- 172.21.15.69:0/3351148654 shutdown_connections 2023-07-06T20:18:32.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.074+0000 7f22ccfff700 1 -- 172.21.15.69:0/3351148654 wait complete. 2023-07-06T20:18:32.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.075+0000 7f22ccfff700 1 Processor -- start 2023-07-06T20:18:32.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.075+0000 7f22ccfff700 1 -- start start 2023-07-06T20:18:32.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.075+0000 7f22ccfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c80ffc30 0x7f22c80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:32.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.075+0000 7f22ccfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22c8100e20 0x7f22c8072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:32.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.075+0000 7f22ccfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22c81017b0 0x7f22c8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:32.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.075+0000 7f22ccfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8073710 con 0x7f22c80ffc30 2023-07-06T20:18:32.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.075+0000 7f22ccfff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8073880 con 0x7f22c8100e20 2023-07-06T20:18:32.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.075+0000 7f22ccfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c819fa50 con 0x7f22c81017b0 2023-07-06T20:18:32.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.076+0000 7f22c659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c80ffc30 0x7f22c80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:32.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.076+0000 7f22c659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c80ffc30 0x7f22c80743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60162/0 (socket says 172.21.15.69:60162) 2023-07-06T20:18:32.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.076+0000 7f22c659c700 1 -- 172.21.15.69:0/2924567925 learned_addr learned my addr 172.21.15.69:0/2924567925 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:32.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.076+0000 7f22c5d9b700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22c8100e20 0x7f22c8072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:32.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.076+0000 7f22c6d9d700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22c81017b0 0x7f22c8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:32.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.076+0000 7f22c659c700 1 -- 172.21.15.69:0/2924567925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22c81017b0 msgr2=0x7f22c8072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:32.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.076+0000 7f22c659c700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22c81017b0 0x7f22c8072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.076+0000 7f22c659c700 1 -- 172.21.15.69:0/2924567925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22c8100e20 msgr2=0x7f22c8072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:32.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.076+0000 7f22c659c700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22c8100e20 0x7f22c8072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.076+0000 7f22c659c700 1 -- 172.21.15.69:0/2924567925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c0009d00 con 0x7f22c80ffc30 2023-07-06T20:18:32.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.076+0000 7f22c659c700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c80ffc30 0x7f22c80743f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f22b800e970 tx=0x7f22b800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:32.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.076+0000 7f22b77fe700 1 -- 172.21.15.69:0/2924567925 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8009790 con 0x7f22c80ffc30 2023-07-06T20:18:32.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.077+0000 7f22ccfff700 1 -- 172.21.15.69:0/2924567925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c819fd30 con 0x7f22c80ffc30 2023-07-06T20:18:32.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.077+0000 7f22b77fe700 1 -- 172.21.15.69:0/2924567925 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22b80044e0 con 0x7f22c80ffc30 2023-07-06T20:18:32.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.077+0000 7f22b77fe700 1 -- 172.21.15.69:0/2924567925 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8003e30 con 0x7f22c80ffc30 2023-07-06T20:18:32.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.077+0000 7f22ccfff700 1 -- 172.21.15.69:0/2924567925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c81a0190 con 0x7f22c80ffc30 2023-07-06T20:18:32.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.078+0000 7f22ccfff700 1 -- 172.21.15.69:0/2924567925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c8107100 con 0x7f22c80ffc30 2023-07-06T20:18:32.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.080+0000 7f22b77fe700 1 -- 172.21.15.69:0/2924567925 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f22b8004000 con 0x7f22c80ffc30 2023-07-06T20:18:32.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.081+0000 7f22b77fe700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f22b00a1220 0x7f22b00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:32.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.081+0000 7f22b77fe700 1 -- 172.21.15.69:0/2924567925 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f22b8014070 con 0x7f22c80ffc30 2023-07-06T20:18:32.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.081+0000 7f22c5d9b700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f22b00a1220 0x7f22b00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:32.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.082+0000 7f22c5d9b700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f22b00a1220 0x7f22b00a36d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009bf0 tx=0x7f22bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:32.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.085+0000 7f22b77fe700 1 -- 172.21.15.69:0/2924567925 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f22b8096360 con 0x7f22c80ffc30 2023-07-06T20:18:32.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:31 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1070407383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:18:32.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:31 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/745663563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:32.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.300+0000 7f22ccfff700 1 -- 172.21.15.69:0/2924567925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22c802c8e0 con 0x7f22c80ffc30 2023-07-06T20:18:32.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.302+0000 7f22b77fe700 1 -- 172.21.15.69:0/2924567925 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22564 (secure 0 0 0) 0x7f22b8099980 con 0x7f22c80ffc30 2023-07-06T20:18:32.305 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:32.305 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":85,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:17:58.736720+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:18:32.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.305+0000 7f22ccfff700 1 -- 172.21.15.69:0/2924567925 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f22b00a1220 msgr2=0x7f22b00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:32.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.305+0000 7f22ccfff700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f22b00a1220 0x7f22b00a36d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009bf0 tx=0x7f22bc009260 comp rx=0 tx=0).stop 2023-07-06T20:18:32.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.305+0000 7f22ccfff700 1 -- 172.21.15.69:0/2924567925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c80ffc30 msgr2=0x7f22c80743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:32.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.305+0000 7f22ccfff700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c80ffc30 0x7f22c80743f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f22b800e970 tx=0x7f22b800ed30 comp rx=0 tx=0).stop 2023-07-06T20:18:32.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.305+0000 7f22ccfff700 1 -- 172.21.15.69:0/2924567925 shutdown_connections 2023-07-06T20:18:32.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.305+0000 7f22ccfff700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f22b00a1220 0x7f22b00a36d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.305+0000 7f22ccfff700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22c81017b0 0x7f22c8072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.305+0000 7f22ccfff700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c80ffc30 0x7f22c80743f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.305+0000 7f22ccfff700 1 --2- 172.21.15.69:0/2924567925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22c8100e20 0x7f22c8072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.305+0000 7f22ccfff700 1 -- 172.21.15.69:0/2924567925 >> 172.21.15.69:0/2924567925 conn(0x7f22c80fb210 msgr2=0x7f22c8104240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:32.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.305+0000 7f22ccfff700 1 -- 172.21.15.69:0/2924567925 shutdown_connections 2023-07-06T20:18:32.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.305+0000 7f22ccfff700 1 -- 172.21.15.69:0/2924567925 wait complete. 2023-07-06T20:18:32.333 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:18:32.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:31 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1070407383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:18:32.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:31 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/745663563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:32.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:31 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1070407383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:18:32.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:31 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/745663563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:32.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.538+0000 7feb79fff700 1 -- 172.21.15.69:0/2467524084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb741006e0 msgr2=0x7feb74102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:32.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.538+0000 7feb79fff700 1 --2- 172.21.15.69:0/2467524084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb741006e0 0x7feb74102ac0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feb64009990 tx=0x7feb64009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.538+0000 7feb79fff700 1 -- 172.21.15.69:0/2467524084 shutdown_connections 2023-07-06T20:18:32.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.538+0000 7feb79fff700 1 --2- 172.21.15.69:0/2467524084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb74105920 0x7feb74107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.538+0000 7feb79fff700 1 --2- 172.21.15.69:0/2467524084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb741006e0 0x7feb74102ac0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.538+0000 7feb79fff700 1 --2- 172.21.15.69:0/2467524084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb74103000 0x7feb741053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.538+0000 7feb79fff700 1 -- 172.21.15.69:0/2467524084 >> 172.21.15.69:0/2467524084 conn(0x7feb740fa100 msgr2=0x7feb740fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:32.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.538+0000 7feb79fff700 1 -- 172.21.15.69:0/2467524084 shutdown_connections 2023-07-06T20:18:32.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.538+0000 7feb79fff700 1 -- 172.21.15.69:0/2467524084 wait complete. 2023-07-06T20:18:32.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.539+0000 7feb79fff700 1 Processor -- start 2023-07-06T20:18:32.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb79fff700 1 -- start start 2023-07-06T20:18:32.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb79fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb741006e0 0x7feb74197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb79fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb74103000 0x7feb74198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb79fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb74105920 0x7feb7419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb79fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7419f6a0 con 0x7feb74103000 2023-07-06T20:18:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb79fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7419f7e0 con 0x7feb741006e0 2023-07-06T20:18:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb79fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7419f970 con 0x7feb74105920 2023-07-06T20:18:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb737fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb741006e0 0x7feb74197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:32.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb72ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb74103000 0x7feb74198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:32.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb737fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb741006e0 0x7feb74197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:42360/0 (socket says 172.21.15.69:42360) 2023-07-06T20:18:32.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb737fe700 1 -- 172.21.15.69:0/2680077358 learned_addr learned my addr 172.21.15.69:0/2680077358 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:32.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb73fff700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb74105920 0x7feb7419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:32.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb737fe700 1 -- 172.21.15.69:0/2680077358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb74105920 msgr2=0x7feb7419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:32.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb737fe700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb74105920 0x7feb7419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb737fe700 1 -- 172.21.15.69:0/2680077358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb74103000 msgr2=0x7feb74198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:32.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.540+0000 7feb737fe700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb74103000 0x7feb74198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.541+0000 7feb737fe700 1 -- 172.21.15.69:0/2680077358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb68009520 con 0x7feb741006e0 2023-07-06T20:18:32.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.541+0000 7feb737fe700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb741006e0 0x7feb74197b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feb64004740 tx=0x7feb64009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:32.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.541+0000 7feb70ff9700 1 -- 172.21.15.69:0/2680077358 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6401d070 con 0x7feb741006e0 2023-07-06T20:18:32.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.541+0000 7feb70ff9700 1 -- 172.21.15.69:0/2680077358 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb64004830 con 0x7feb741006e0 2023-07-06T20:18:32.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.541+0000 7feb79fff700 1 -- 172.21.15.69:0/2680077358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb640095f0 con 0x7feb741006e0 2023-07-06T20:18:32.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.541+0000 7feb70ff9700 1 -- 172.21.15.69:0/2680077358 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb640053c0 con 0x7feb741006e0 2023-07-06T20:18:32.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.541+0000 7feb79fff700 1 -- 172.21.15.69:0/2680077358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7419fe20 con 0x7feb741006e0 2023-07-06T20:18:32.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.542+0000 7feb79fff700 1 -- 172.21.15.69:0/2680077358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb74107490 con 0x7feb741006e0 2023-07-06T20:18:32.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.544+0000 7feb70ff9700 1 -- 172.21.15.69:0/2680077358 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7feb64005580 con 0x7feb741006e0 2023-07-06T20:18:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.545+0000 7feb70ff9700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feb5c0a1220 0x7feb5c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.545+0000 7feb70ff9700 1 -- 172.21.15.69:0/2680077358 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7feb640cdc40 con 0x7feb741006e0 2023-07-06T20:18:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.546+0000 7feb72ffd700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feb5c0a1220 0x7feb5c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.546+0000 7feb72ffd700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feb5c0a1220 0x7feb5c0a36d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feb68009df0 tx=0x7feb68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:32.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.548+0000 7feb70ff9700 1 -- 172.21.15.69:0/2680077358 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7feb64097250 con 0x7feb741006e0 2023-07-06T20:18:32.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.810+0000 7feb79fff700 1 -- 172.21.15.69:0/2680077358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb7402cd30 con 0x7feb741006e0 2023-07-06T20:18:32.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.811+0000 7feb70ff9700 1 -- 172.21.15.69:0/2680077358 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10795 (secure 0 0 0) 0x7feb6409a870 con 0x7feb741006e0 2023-07-06T20:18:32.813 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:32.813 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-06T20:17:28.144582+0000","modified":"2023-07-06T20:17:33.557511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24539,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/3235525224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3235525224},{"type":"v1","addr":"172.21.15.98:6839","nonce":3235525224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:18:32.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.813+0000 7feb79fff700 1 -- 172.21.15.69:0/2680077358 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feb5c0a1220 msgr2=0x7feb5c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:32.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.813+0000 7feb79fff700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feb5c0a1220 0x7feb5c0a36d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feb68009df0 tx=0x7feb68009260 comp rx=0 tx=0).stop 2023-07-06T20:18:32.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.813+0000 7feb79fff700 1 -- 172.21.15.69:0/2680077358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb741006e0 msgr2=0x7feb74197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.813+0000 7feb79fff700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb741006e0 0x7feb74197b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feb64004740 tx=0x7feb64009580 comp rx=0 tx=0).stop 2023-07-06T20:18:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.813+0000 7feb79fff700 1 -- 172.21.15.69:0/2680077358 shutdown_connections 2023-07-06T20:18:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.813+0000 7feb79fff700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7feb5c0a1220 0x7feb5c0a36d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.813+0000 7feb79fff700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb74105920 0x7feb7419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.813+0000 7feb79fff700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb741006e0 0x7feb74197b20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.813+0000 7feb79fff700 1 --2- 172.21.15.69:0/2680077358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb74103000 0x7feb74198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:32.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.813+0000 7feb79fff700 1 -- 172.21.15.69:0/2680077358 >> 172.21.15.69:0/2680077358 conn(0x7feb740fa100 msgr2=0x7feb740fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:32.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.813+0000 7feb79fff700 1 -- 172.21.15.69:0/2680077358 shutdown_connections 2023-07-06T20:18:32.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:32.813+0000 7feb79fff700 1 -- 172.21.15.69:0/2680077358 wait complete. 2023-07-06T20:18:32.817 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2023-07-06T20:18:32.835 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-06T20:17:28.144582+0000', 'modified': '2023-07-06T20:17:33.557511+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24584, 'mds_1': 24539, 'mds_2': 24512}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/2525319359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 2525319359}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 2525319359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24512': {'gid': 24512, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.98:6835/827333150', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 827333150}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 827333150}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24517': {'gid': 24517, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6837/595692705', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 595692705}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 595692705}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24535': {'gid': 24535, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6839/4200610083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 4200610083}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 4200610083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.98:6837/4010267682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 4010267682}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 4010267682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24584': {'gid': 24584, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6839/3235525224', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 3235525224}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 3235525224}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T20:18:32.836 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-06T20:18:32.836 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-07-06T20:18:33.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.045+0000 7fe077fff700 1 -- 172.21.15.69:0/3829007929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe070074620 msgr2=0x7fe070072a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.045+0000 7fe077fff700 1 --2- 172.21.15.69:0/3829007929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe070074620 0x7fe070072a70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009990 tx=0x7fe06c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.045+0000 7fe077fff700 1 -- 172.21.15.69:0/3829007929 shutdown_connections 2023-07-06T20:18:33.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.045+0000 7fe077fff700 1 --2- 172.21.15.69:0/3829007929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe070107b10 0x7fe070109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.045+0000 7fe077fff700 1 --2- 172.21.15.69:0/3829007929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe070074620 0x7fe070072a70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.045+0000 7fe077fff700 1 --2- 172.21.15.69:0/3829007929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070073040 0x7fe0700734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.045+0000 7fe077fff700 1 -- 172.21.15.69:0/3829007929 >> 172.21.15.69:0/3829007929 conn(0x7fe0700fb2a0 msgr2=0x7fe0700fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.045+0000 7fe077fff700 1 -- 172.21.15.69:0/3829007929 shutdown_connections 2023-07-06T20:18:33.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.045+0000 7fe077fff700 1 -- 172.21.15.69:0/3829007929 wait complete. 2023-07-06T20:18:33.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.046+0000 7fe077fff700 1 Processor -- start 2023-07-06T20:18:33.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.047+0000 7fe077fff700 1 -- start start 2023-07-06T20:18:33.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.047+0000 7fe077fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe070073040 0x7fe070197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.047+0000 7fe077fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe070074620 0x7fe0701982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.047+0000 7fe077fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070107b10 0x7fe07019f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.047+0000 7fe077fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07019f8f0 con 0x7fe070107b10 2023-07-06T20:18:33.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.047+0000 7fe077fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07019fa30 con 0x7fe070073040 2023-07-06T20:18:33.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.047+0000 7fe077fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07019fbc0 con 0x7fe070074620 2023-07-06T20:18:33.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.047+0000 7fe07559a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe070074620 0x7fe0701982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.047+0000 7fe07659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070107b10 0x7fe07019f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe07559a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe070074620 0x7fe0701982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:42542/0 (socket says 172.21.15.69:42542) 2023-07-06T20:18:33.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe07559a700 1 -- 172.21.15.69:0/3939643221 learned_addr learned my addr 172.21.15.69:0/3939643221 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe075d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe070073040 0x7fe070197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe07559a700 1 -- 172.21.15.69:0/3939643221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe070073040 msgr2=0x7fe070197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe07559a700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe070073040 0x7fe070197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe07559a700 1 -- 172.21.15.69:0/3939643221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070107b10 msgr2=0x7fe07019f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe07559a700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070107b10 0x7fe07019f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe07559a700 1 -- 172.21.15.69:0/3939643221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c0095f0 con 0x7fe070074620 2023-07-06T20:18:33.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe07659c700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070107b10 0x7fe07019f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:33.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe075d9b700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe070073040 0x7fe070197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:33.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe07559a700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe070074620 0x7fe0701982b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe06000e9a0 tx=0x7fe06000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe066ffd700 1 -- 172.21.15.69:0/3939643221 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060009790 con 0x7fe070074620 2023-07-06T20:18:33.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.048+0000 7fe066ffd700 1 -- 172.21.15.69:0/3939643221 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe060010430 con 0x7fe070074620 2023-07-06T20:18:33.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.049+0000 7fe077fff700 1 -- 172.21.15.69:0/3939643221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07019fea0 con 0x7fe070074620 2023-07-06T20:18:33.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.049+0000 7fe066ffd700 1 -- 172.21.15.69:0/3939643221 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060018680 con 0x7fe070074620 2023-07-06T20:18:33.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.049+0000 7fe077fff700 1 -- 172.21.15.69:0/3939643221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0701a0320 con 0x7fe070074620 2023-07-06T20:18:33.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.052+0000 7fe066ffd700 1 -- 172.21.15.69:0/3939643221 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe0600105a0 con 0x7fe070074620 2023-07-06T20:18:33.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.053+0000 7fe066ffd700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe05c0a11e0 0x7fe05c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.053+0000 7fe066ffd700 1 -- 172.21.15.69:0/3939643221 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe060014070 con 0x7fe070074620 2023-07-06T20:18:33.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.054+0000 7fe075d9b700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe05c0a11e0 0x7fe05c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.054+0000 7fe075d9b700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe05c0a11e0 0x7fe05c0a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00b2e0 tx=0x7fe06c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe08b9b7b80 1 Processor -- start 2023-07-06T20:18:33.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe08b9b7b80 1 -- start start 2023-07-06T20:18:33.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe08b9b7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feec92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe08b9b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 0x5581feec9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe08b9b7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581feeca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe08b9b7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fedcdc60 con 0x5581feec8ed0 2023-07-06T20:18:33.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe08b9b7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fee209f0 con 0x5581feeca340 2023-07-06T20:18:33.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe08b9b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fee19e80 con 0x5581feec9820 2023-07-06T20:18:33.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe07559a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 0x5581feec9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe07659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581feeca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe075d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feec92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe07559a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 0x5581feec9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:42546/0 (socket says 172.21.15.69:42546) 2023-07-06T20:18:33.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.055+0000 7fe07559a700 1 -- 172.21.15.69:0/3116554181 learned_addr learned my addr 172.21.15.69:0/3116554181 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.056+0000 7fe07559a700 1 -- 172.21.15.69:0/3116554181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 msgr2=0x5581feeca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.056+0000 7fe07559a700 1 --2- 172.21.15.69:0/3116554181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581feeca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.056+0000 7fe07559a700 1 -- 172.21.15.69:0/3116554181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 msgr2=0x5581feec92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.056+0000 7fe07559a700 1 --2- 172.21.15.69:0/3116554181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feec92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.056+0000 7fe07559a700 1 -- 172.21.15.69:0/3116554181 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581fec47d90 con 0x5581feec9820 2023-07-06T20:18:33.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.056+0000 7fe07659c700 1 --2- 172.21.15.69:0/3116554181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581feeca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:33.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.056+0000 7fe07559a700 1 --2- 172.21.15.69:0/3116554181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 0x5581feec9db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe0600a1e20 tx=0x7fe060099990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0f2ff573eeafc9d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.056+0000 7fe05bfff700 1 -- 172.21.15.69:0/3116554181 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600a2350 con 0x5581feec9820 2023-07-06T20:18:33.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.056+0000 7fe05bfff700 1 -- 172.21.15.69:0/3116554181 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe060010880 con 0x5581feec9820 2023-07-06T20:18:33.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.056+0000 7fe05bfff700 1 -- 172.21.15.69:0/3116554181 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600a1ec0 con 0x5581feec9820 2023-07-06T20:18:33.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.057+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/3116554181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 msgr2=0x5581feec9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.057+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/3116554181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 0x5581feec9db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe0600a1e20 tx=0x7fe060099990 comp rx=0 tx=0).stop 2023-07-06T20:18:33.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.057+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/3116554181 shutdown_connections 2023-07-06T20:18:33.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.057+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/3116554181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581feeca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.057+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/3116554181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feec92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.057+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/3116554181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 0x5581feec9db0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.057+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/3116554181 >> 172.21.15.69:0/3116554181 conn(0x5581feebe5f0 msgr2=0x5581feec39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.057+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/3116554181 shutdown_connections 2023-07-06T20:18:33.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.057+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/3116554181 wait complete. 2023-07-06T20:18:33.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.058+0000 7fe08b9b7b80 1 -- start start 2023-07-06T20:18:33.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.058+0000 7fe08b9b7b80 1 Processor -- start 2023-07-06T20:18:33.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.059+0000 7fe08b9b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec8ed0 0x5581feed28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.059+0000 7fe08b9b7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec9820 0x5581feed2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.059+0000 7fe08b9b7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581feed9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.059+0000 7fe08b9b7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fedb9a70 con 0x5581feec9820 2023-07-06T20:18:33.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.059+0000 7fe08b9b7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fedb9770 con 0x5581feeca340 2023-07-06T20:18:33.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.059+0000 7fe08b9b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fec9d8d0 con 0x5581feec8ed0 2023-07-06T20:18:33.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.059+0000 7fe08b9b7b80 10 client.0 _ll_register_callbacks cb 0x5581fedb7710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:18:33.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.059+0000 7fe07559a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec9820 0x5581feed2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.059+0000 7fe07659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581feed9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.059+0000 7fe07559a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec9820 0x5581feed2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58858/0 (socket says 172.21.15.69:58858) 2023-07-06T20:18:33.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.059+0000 7fe07559a700 1 -- 172.21.15.69:0/1634502113 learned_addr learned my addr 172.21.15.69:0/1634502113 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.059+0000 7fe07659c700 1 -- 172.21.15.69:0/1634502113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec8ed0 msgr2=0x5581feed28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe07659c700 1 --2- 172.21.15.69:0/1634502113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec8ed0 0x5581feed28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe07659c700 1 -- 172.21.15.69:0/1634502113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec9820 msgr2=0x5581feed2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe07659c700 1 --2- 172.21.15.69:0/1634502113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec9820 0x5581feed2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe07659c700 1 -- 172.21.15.69:0/1634502113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe068003a60 con 0x5581feeca340 2023-07-06T20:18:33.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe07559a700 1 --2- 172.21.15.69:0/1634502113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec9820 0x5581feed2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:33.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe07659c700 1 --2- 172.21.15.69:0/1634502113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581feed9e30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe068003db0 tx=0x7fe06800e7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e581ea07d2f2bc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe0597fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:18:33.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe0597fa700 1 -- 172.21.15.69:0/1634502113 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068004db0 con 0x5581feeca340 2023-07-06T20:18:33.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/1634502113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581fed734a0 con 0x5581feeca340 2023-07-06T20:18:33.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe0597fa700 1 -- 172.21.15.69:0/1634502113 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe0680090d0 con 0x5581feeca340 2023-07-06T20:18:33.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe0597fa700 1 -- 172.21.15.69:0/1634502113 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe068009290 con 0x5581feeca340 2023-07-06T20:18:33.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe08b9b7b80 10 client.24980 fetch_fsmap learned FSMap version 23 2023-07-06T20:18:33.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.060+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/1634502113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fe0701a0320 con 0x5581feeca340 2023-07-06T20:18:33.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.061+0000 7fe0597fa700 1 -- 172.21.15.69:0/1634502113 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5581fed734a0 con 0x5581feeca340 2023-07-06T20:18:33.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.061+0000 7fe0597fa700 1 -- 172.21.15.69:0/1634502113 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe06801c030 con 0x5581feeca340 2023-07-06T20:18:33.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.061+0000 7fe08b9b7b80 10 client.24980 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:18:33.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.061+0000 7fe08b9b7b80 10 client.24980 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-06T20:18:33.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.061+0000 7fe08b9b7b80 20 client.24980 populate_metadata read hostname 'smithi069' 2023-07-06T20:18:33.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.061+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/1634502113 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5581feeddcc0 0x5581feedd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.061+0000 7fe08b9b7b80 4 client.24980 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:18:33.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.061+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/1634502113 --> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581fee97d10 con 0x5581feeddcc0 2023-07-06T20:18:33.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.061+0000 7fe075d9b700 1 --2- 172.21.15.69:0/1634502113 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5581feeddcc0 0x5581feedd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.062+0000 7fe0597fa700 10 client.24980 ms_handle_connect on v2:172.21.15.98:6834/827333150 2023-07-06T20:18:33.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.062+0000 7fe075d9b700 1 --2- 172.21.15.69:0/1634502113 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5581feeddcc0 0x5581feedd010 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60c10dbdc1d79cc4 server_cookie=8e47b0c0502df6a2 in_seq=0 out_seq=0 2023-07-06T20:18:33.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.062+0000 7fe0597fa700 1 -- 172.21.15.69:0/1634502113 <== mds.2 v2:172.21.15.98:6834/827333150 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5581fee97d10 con 0x5581feeddcc0 2023-07-06T20:18:33.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.062+0000 7fe0597fa700 10 client.24980 handle_command_reply: tid=0 2023-07-06T20:18:33.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.062+0000 7fe08b9b7b80 1 client.24980 shutdown 2023-07-06T20:18:33.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.063+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/1634502113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 msgr2=0x5581feed9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.063+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/1634502113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581feed9e30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe068003db0 tx=0x7fe06800e7d0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.063+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/1634502113 shutdown_connections 2023-07-06T20:18:33.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.063+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/1634502113 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5581feeddcc0 0x5581feedd010 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.063+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/1634502113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581feed9e30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.063+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/1634502113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec8ed0 0x5581feed28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.063+0000 7fe0597fa700 0 client.24980 ms_handle_reset on v2:172.21.15.98:6834/827333150 2023-07-06T20:18:33.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.063+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/1634502113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec9820 0x5581feed2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.063+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/1634502113 >> 172.21.15.69:0/1634502113 conn(0x5581feebe5f0 msgr2=0x5581feec3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.063+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/1634502113 shutdown_connections 2023-07-06T20:18:33.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.063+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/1634502113 wait complete. 2023-07-06T20:18:33.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.063+0000 7fe08b9b7b80 20 client.24980 trim_cache size 0 max 16384 2023-07-06T20:18:33.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.081+0000 7fe08b9b7b80 1 Processor -- start 2023-07-06T20:18:33.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.081+0000 7fe08b9b7b80 1 -- start start 2023-07-06T20:18:33.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe08b9b7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feed15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe08b9b7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feec9820 0x5581feed9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe08b9b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feeca340 0x5581feed0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe08b9b7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581feed2480 con 0x5581feec8ed0 2023-07-06T20:18:33.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe08b9b7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fed734a0 con 0x5581feec9820 2023-07-06T20:18:33.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe08b9b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fede8790 con 0x5581feeca340 2023-07-06T20:18:33.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe075d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feed15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe07559a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feec9820 0x5581feed9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe07659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feeca340 0x5581feed0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe075d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feed15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58872/0 (socket says 172.21.15.69:58872) 2023-07-06T20:18:33.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe075d9b700 1 -- 172.21.15.69:0/452296499 learned_addr learned my addr 172.21.15.69:0/452296499 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe07659c700 1 -- 172.21.15.69:0/452296499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feec9820 msgr2=0x5581feed9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.082+0000 7fe07659c700 1 --2- 172.21.15.69:0/452296499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feec9820 0x5581feed9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.083+0000 7fe07659c700 1 -- 172.21.15.69:0/452296499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 msgr2=0x5581feed15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.083+0000 7fe07659c700 1 --2- 172.21.15.69:0/452296499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feed15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.083+0000 7fe07659c700 1 -- 172.21.15.69:0/452296499 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581fec30c10 con 0x5581feeca340 2023-07-06T20:18:33.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.083+0000 7fe075d9b700 1 --2- 172.21.15.69:0/452296499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feed15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:33.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.083+0000 7fe07659c700 1 --2- 172.21.15.69:0/452296499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feeca340 0x5581feed0700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe06801e400 tx=0x7fe06800afd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb15b0652d8bbaef server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.083+0000 7fe064ff9700 1 -- 172.21.15.69:0/452296499 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06801e770 con 0x5581feeca340 2023-07-06T20:18:33.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.083+0000 7fe064ff9700 1 -- 172.21.15.69:0/452296499 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe06800f040 con 0x5581feeca340 2023-07-06T20:18:33.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.083+0000 7fe064ff9700 1 -- 172.21.15.69:0/452296499 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06801ee10 con 0x5581feeca340 2023-07-06T20:18:33.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.084+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/452296499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feeca340 msgr2=0x5581feed0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.084+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/452296499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feeca340 0x5581feed0700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe06801e400 tx=0x7fe06800afd0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.084+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/452296499 shutdown_connections 2023-07-06T20:18:33.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.084+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/452296499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feeca340 0x5581feed0700 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.084+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/452296499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feed15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.084+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/452296499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feec9820 0x5581feed9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.084+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/452296499 >> 172.21.15.69:0/452296499 conn(0x5581feebe5f0 msgr2=0x5581feec3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.084+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/452296499 shutdown_connections 2023-07-06T20:18:33.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.084+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/452296499 wait complete. 2023-07-06T20:18:33.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.084+0000 7fe08b9b7b80 1 -- start start 2023-07-06T20:18:33.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.085+0000 7fe08b9b7b80 1 Processor -- start 2023-07-06T20:18:33.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.085+0000 7fe08b9b7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feed4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.085+0000 7fe08b9b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 0x5581feed5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.085+0000 7fe08b9b7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581fef31b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.093 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.085+0000 7fe08b9b7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581feed5c30 con 0x5581feec8ed0 2023-07-06T20:18:33.093 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.085+0000 7fe08b9b7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fef320a0 con 0x5581feeca340 2023-07-06T20:18:33.093 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.085+0000 7fe08b9b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fef321e0 con 0x5581feec9820 2023-07-06T20:18:33.093 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.085+0000 7fe08b9b7b80 10 client.0 _ll_register_callbacks cb 0x7fe0680043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:18:33.093 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe07559a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 0x5581feed5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.093 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe075d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feed4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.093 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe07559a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 0x5581feed5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:42552/0 (socket says 172.21.15.69:42552) 2023-07-06T20:18:33.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe07659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581fef31b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe07559a700 1 -- 172.21.15.69:0/2740416810 learned_addr learned my addr 172.21.15.69:0/2740416810 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe07559a700 1 -- 172.21.15.69:0/2740416810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 msgr2=0x5581fef31b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe07559a700 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581fef31b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe07559a700 1 -- 172.21.15.69:0/2740416810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 msgr2=0x5581feed4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe07559a700 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feed4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe07559a700 1 -- 172.21.15.69:0/2740416810 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07010abb0 con 0x5581feec9820 2023-07-06T20:18:33.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe07659c700 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581fef31b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:33.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe075d9b700 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feed4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:33.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe07559a700 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 0x5581feed5560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe06000c2d0 tx=0x7fe0600996a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ade914488ad5eb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe0597fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:18:33.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe0597fa700 1 -- 172.21.15.69:0/2740416810 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600109f0 con 0x5581feec9820 2023-07-06T20:18:33.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe0597fa700 1 -- 172.21.15.69:0/2740416810 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe0600a2350 con 0x5581feec9820 2023-07-06T20:18:33.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.086+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/2740416810 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe060010880 con 0x5581feec9820 2023-07-06T20:18:33.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.087+0000 7fe0597fa700 1 -- 172.21.15.69:0/2740416810 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0600b81f0 con 0x5581feec9820 2023-07-06T20:18:33.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.087+0000 7fe08b9b7b80 10 client.24958 fetch_fsmap learned FSMap version 23 2023-07-06T20:18:33.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.087+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/2740416810 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fe0701a0320 con 0x5581feec9820 2023-07-06T20:18:33.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.087+0000 7fe0597fa700 1 -- 172.21.15.69:0/2740416810 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe060010880 con 0x5581feec9820 2023-07-06T20:18:33.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.087+0000 7fe0597fa700 1 -- 172.21.15.69:0/2740416810 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe06000ca10 con 0x5581feec9820 2023-07-06T20:18:33.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.087+0000 7fe08b9b7b80 10 client.24958 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:18:33.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.087+0000 7fe08b9b7b80 10 client.24958 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-06T20:18:33.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.087+0000 7fe08b9b7b80 20 client.24958 populate_metadata read hostname 'smithi069' 2023-07-06T20:18:33.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.087+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5581fef36c30 0x5581fef335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.087+0000 7fe08b9b7b80 4 client.24958 mds_command: new command op to 24512 tid=0[{"prefix": "status"}] 2023-07-06T20:18:33.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.087+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/2740416810 --> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5581fee12e60 con 0x5581fef36c30 2023-07-06T20:18:33.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.088+0000 7fe075d9b700 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5581fef36c30 0x5581fef335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.088+0000 7fe075d9b700 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5581fef36c30 0x5581fef335c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=266525c95b9371c8 server_cookie=d2675577b2a00ef in_seq=0 out_seq=0 2023-07-06T20:18:33.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.088+0000 7fe0597fa700 10 client.24958 ms_handle_connect on v2:172.21.15.98:6834/827333150 2023-07-06T20:18:33.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.088+0000 7fe0597fa700 1 -- 172.21.15.69:0/2740416810 <== mds.2 v2:172.21.15.98:6834/827333150 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5581fee12e60 con 0x5581fef36c30 2023-07-06T20:18:33.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.088+0000 7fe0597fa700 10 client.24958 handle_command_reply: tid=0 2023-07-06T20:18:33.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.088+0000 7fe08b9b7b80 1 client.24958 shutdown 2023-07-06T20:18:33.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.088+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/2740416810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 msgr2=0x5581feed5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.088+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 0x5581feed5560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe06000c2d0 tx=0x7fe0600996a0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/2740416810 shutdown_connections 2023-07-06T20:18:33.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5581fef36c30 0x5581fef335c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581feeca340 0x5581fef31b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581feec8ed0 0x5581feed4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe08b9b7b80 1 --2- 172.21.15.69:0/2740416810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581feec9820 0x5581feed5560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe0597fa700 0 client.24958 ms_handle_reset on v2:172.21.15.98:6834/827333150 2023-07-06T20:18:33.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/2740416810 >> 172.21.15.69:0/2740416810 conn(0x5581feebe5f0 msgr2=0x5581feed0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/2740416810 shutdown_connections 2023-07-06T20:18:33.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe08b9b7b80 1 -- 172.21.15.69:0/2740416810 wait complete. 2023-07-06T20:18:33.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe08b9b7b80 20 client.24958 trim_cache size 0 max 16384 2023-07-06T20:18:33.102 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:18:33.102 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "e3e8ec0c-1c38-11ee-9b2f-001a4aab830c", 2023-07-06T20:18:33.102 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2023-07-06T20:18:33.103 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24512, 2023-07-06T20:18:33.103 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2023-07-06T20:18:33.103 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2023-07-06T20:18:33.103 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2023-07-06T20:18:33.104 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 60.536089050000001, 2023-07-06T20:18:33.104 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 23, 2023-07-06T20:18:33.104 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 85, 2023-07-06T20:18:33.104 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 83, 2023-07-06T20:18:33.104 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 88.287677728999995 2023-07-06T20:18:33.105 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:18:33.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe077fff700 1 -- 172.21.15.69:0/3939643221 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe05c0a11e0 msgr2=0x7fe05c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe077fff700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe05c0a11e0 0x7fe05c0a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00b2e0 tx=0x7fe06c005fb0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe077fff700 1 -- 172.21.15.69:0/3939643221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe070074620 msgr2=0x7fe0701982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe077fff700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe070074620 0x7fe0701982b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe06000e9a0 tx=0x7fe06000ecb0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe077fff700 1 -- 172.21.15.69:0/3939643221 shutdown_connections 2023-07-06T20:18:33.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe077fff700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe05c0a11e0 0x7fe05c0a3690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe077fff700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe070107b10 0x7fe07019f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.089+0000 7fe077fff700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe070073040 0x7fe070197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.090+0000 7fe077fff700 1 --2- 172.21.15.69:0/3939643221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe070074620 0x7fe0701982b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.090+0000 7fe077fff700 1 -- 172.21.15.69:0/3939643221 >> 172.21.15.69:0/3939643221 conn(0x7fe0700fb2a0 msgr2=0x7fe070109e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.090+0000 7fe077fff700 1 -- 172.21.15.69:0/3939643221 shutdown_connections 2023-07-06T20:18:33.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.090+0000 7fe077fff700 1 -- 172.21.15.69:0/3939643221 wait complete. 2023-07-06T20:18:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:32 smithi069 ceph-mon[126399]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-06T20:18:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:32 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2924567925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:33.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:32 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2680077358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:33.110 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-06T20:18:33.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.314+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/2585473405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e54102f90 msgr2=0x7f7e54103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.314+0000 7f7e5b9ff700 1 --2- 172.21.15.69:0/2585473405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e54102f90 0x7f7e54103400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009990 tx=0x7f7e44009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.314+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/2585473405 shutdown_connections 2023-07-06T20:18:33.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.314+0000 7f7e5b9ff700 1 --2- 172.21.15.69:0/2585473405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e54103940 0x7f7e5410a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.314+0000 7f7e5b9ff700 1 --2- 172.21.15.69:0/2585473405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e54101e40 0x7f7e54102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.314+0000 7f7e5b9ff700 1 --2- 172.21.15.69:0/2585473405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e54102f90 0x7f7e54103400 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.314+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/2585473405 >> 172.21.15.69:0/2585473405 conn(0x7f7e540fd480 msgr2=0x7f7e540ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.314+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/2585473405 shutdown_connections 2023-07-06T20:18:33.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.314+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/2585473405 wait complete. 2023-07-06T20:18:33.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.315+0000 7f7e5b9ff700 1 Processor -- start 2023-07-06T20:18:33.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.316+0000 7f7e5b9ff700 1 -- start start 2023-07-06T20:18:33.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.316+0000 7f7e5b9ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e54101e40 0x7f7e54197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.316+0000 7f7e5b9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e54102f90 0x7f7e54198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.316+0000 7f7e5b9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e54103940 0x7f7e5419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.316+0000 7f7e5b9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5419f770 con 0x7f7e54103940 2023-07-06T20:18:33.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.316+0000 7f7e5b9ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5419f8b0 con 0x7f7e54101e40 2023-07-06T20:18:33.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.316+0000 7f7e5b9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5419fa40 con 0x7f7e54102f90 2023-07-06T20:18:33.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.316+0000 7f7e5979b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e54101e40 0x7f7e54197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.317+0000 7f7e5979b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e54101e40 0x7f7e54197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:56954/0 (socket says 172.21.15.69:56954) 2023-07-06T20:18:33.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.317+0000 7f7e5979b700 1 -- 172.21.15.69:0/1359872811 learned_addr learned my addr 172.21.15.69:0/1359872811 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.317+0000 7f7e59f9c700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e54103940 0x7f7e5419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.317+0000 7f7e58f9a700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e54102f90 0x7f7e54198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.317+0000 7f7e5979b700 1 -- 172.21.15.69:0/1359872811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e54102f90 msgr2=0x7f7e54198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.317+0000 7f7e5979b700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e54102f90 0x7f7e54198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.317+0000 7f7e5979b700 1 -- 172.21.15.69:0/1359872811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e54103940 msgr2=0x7f7e5419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.317+0000 7f7e5979b700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e54103940 0x7f7e5419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.317+0000 7f7e5979b700 1 -- 172.21.15.69:0/1359872811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e440095f0 con 0x7f7e54101e40 2023-07-06T20:18:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.317+0000 7f7e5979b700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e54101e40 0x7f7e54197bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000e9a0 tx=0x7f7e5000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.317+0000 7f7e4a7fc700 1 -- 172.21.15.69:0/1359872811 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50009790 con 0x7f7e54101e40 2023-07-06T20:18:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.318+0000 7f7e4a7fc700 1 -- 172.21.15.69:0/1359872811 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e50010430 con 0x7f7e54101e40 2023-07-06T20:18:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.318+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/1359872811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5419fd20 con 0x7f7e54101e40 2023-07-06T20:18:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.318+0000 7f7e4a7fc700 1 -- 172.21.15.69:0/1359872811 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50018680 con 0x7f7e54101e40 2023-07-06T20:18:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.318+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/1359872811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e541a01d0 con 0x7f7e54101e40 2023-07-06T20:18:33.324 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.322+0000 7f7e4a7fc700 1 -- 172.21.15.69:0/1359872811 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7e500105a0 con 0x7f7e54101e40 2023-07-06T20:18:33.324 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.323+0000 7f7e6f3d8b80 1 Processor -- start 2023-07-06T20:18:33.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e6f3d8b80 1 -- start start 2023-07-06T20:18:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e6f3d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec3ed0 0x555e4fec42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e6f3d8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec4820 0x555e4fec4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e6f3d8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec5340 0x555e4fec57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e6f3d8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fdc8c60 con 0x555e4fec5340 2023-07-06T20:18:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e6f3d8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fe1b9f0 con 0x555e4fec4820 2023-07-06T20:18:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e6f3d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fe14e80 con 0x555e4fec3ed0 2023-07-06T20:18:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e5979b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec4820 0x555e4fec4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e59f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec3ed0 0x555e4fec42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e59f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec3ed0 0x555e4fec42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:42586/0 (socket says 172.21.15.69:42586) 2023-07-06T20:18:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e59f9c700 1 -- 172.21.15.69:0/697033195 learned_addr learned my addr 172.21.15.69:0/697033195 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e4a7fc700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7e400a1260 0x7f7e400a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.324+0000 7f7e58f9a700 1 --2- 172.21.15.69:0/697033195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec5340 0x555e4fec57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.325+0000 7f7e4a7fc700 1 -- 172.21.15.69:0/1359872811 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f7e50014070 con 0x7f7e54101e40 2023-07-06T20:18:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.325+0000 7f7e59f9c700 1 -- 172.21.15.69:0/697033195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec4820 msgr2=0x555e4fec4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.325+0000 7f7e59f9c700 1 --2- 172.21.15.69:0/697033195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec4820 0x555e4fec4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.325+0000 7f7e59f9c700 1 -- 172.21.15.69:0/697033195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec5340 msgr2=0x555e4fec57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.325+0000 7f7e59f9c700 1 --2- 172.21.15.69:0/697033195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec5340 0x555e4fec57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.325+0000 7f7e59f9c700 1 -- 172.21.15.69:0/697033195 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4fc42d90 con 0x555e4fec3ed0 2023-07-06T20:18:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.325+0000 7f7e58f9a700 1 --2- 172.21.15.69:0/697033195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec5340 0x555e4fec57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.325+0000 7f7e59f9c700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7e400a1260 0x7f7e400a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.325+0000 7f7e59f9c700 1 --2- 172.21.15.69:0/697033195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec3ed0 0x555e4fec42e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00a5b0 tx=0x7f7e4c00a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f67c26faca3da4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.326+0000 7f7e3f7fe700 1 -- 172.21.15.69:0/697033195 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c018d00 con 0x555e4fec3ed0 2023-07-06T20:18:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.326+0000 7f7e3f7fe700 1 -- 172.21.15.69:0/697033195 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e4c018e60 con 0x555e4fec3ed0 2023-07-06T20:18:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.326+0000 7f7e3f7fe700 1 -- 172.21.15.69:0/697033195 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c010070 con 0x555e4fec3ed0 2023-07-06T20:18:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.326+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/697033195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec3ed0 msgr2=0x555e4fec42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.326+0000 7f7e59f9c700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7e400a1260 0x7f7e400a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c020000 tx=0x7f7e4c021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.326+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/697033195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec3ed0 0x555e4fec42e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00a5b0 tx=0x7f7e4c00a8c0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.334 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.327+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/697033195 shutdown_connections 2023-07-06T20:18:33.334 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.327+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/697033195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec5340 0x555e4fec57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.334 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.327+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/697033195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec3ed0 0x555e4fec42e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.334 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.327+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/697033195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec4820 0x555e4fec4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.334 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.327+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/697033195 >> 172.21.15.69:0/697033195 conn(0x555e4feb95f0 msgr2=0x555e4febe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.335 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.327+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/697033195 shutdown_connections 2023-07-06T20:18:33.335 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.327+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/697033195 wait complete. 2023-07-06T20:18:33.335 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.327+0000 7f7e6f3d8b80 1 -- start start 2023-07-06T20:18:33.335 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.327+0000 7f7e6f3d8b80 1 Processor -- start 2023-07-06T20:18:33.335 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e6f3d8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec3ed0 0x555e4fecd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e6f3d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fecddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e6f3d8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec5340 0x555e4fed4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e6f3d8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fdb4a70 con 0x555e4fec3ed0 2023-07-06T20:18:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e6f3d8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fdb4770 con 0x555e4fec5340 2023-07-06T20:18:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e6f3d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fc988d0 con 0x555e4fec4820 2023-07-06T20:18:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e6f3d8b80 10 client.0 _ll_register_callbacks cb 0x555e4fdb2710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:18:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e58f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fecddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e59f9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec5340 0x555e4fed4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e58f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fecddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:42590/0 (socket says 172.21.15.69:42590) 2023-07-06T20:18:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e58f9a700 1 -- 172.21.15.69:0/141173933 learned_addr learned my addr 172.21.15.69:0/141173933 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e5979b700 1 --2- 172.21.15.69:0/141173933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec3ed0 0x555e4fecd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e58f9a700 1 -- 172.21.15.69:0/141173933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec5340 msgr2=0x555e4fed4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e58f9a700 1 --2- 172.21.15.69:0/141173933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec5340 0x555e4fed4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.328+0000 7f7e58f9a700 1 -- 172.21.15.69:0/141173933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec3ed0 msgr2=0x555e4fecd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.329+0000 7f7e58f9a700 1 --2- 172.21.15.69:0/141173933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec3ed0 0x555e4fecd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.329+0000 7f7e58f9a700 1 -- 172.21.15.69:0/141173933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4c009d50 con 0x555e4fec4820 2023-07-06T20:18:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.329+0000 7f7e58f9a700 1 --2- 172.21.15.69:0/141173933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fecddd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7e44015000 tx=0x7f7e44005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34529384615c2ac5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.329+0000 7f7e3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:18:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.329+0000 7f7e3cff9700 1 -- 172.21.15.69:0/141173933 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4401d040 con 0x555e4fec4820 2023-07-06T20:18:33.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.329+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/141173933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e4fd6e4a0 con 0x555e4fec4820 2023-07-06T20:18:33.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.329+0000 7f7e3cff9700 1 -- 172.21.15.69:0/141173933 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f7e44004700 con 0x555e4fec4820 2023-07-06T20:18:33.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.329+0000 7f7e3cff9700 1 -- 172.21.15.69:0/141173933 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e440048c0 con 0x555e4fec4820 2023-07-06T20:18:33.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.329+0000 7f7e6f3d8b80 10 client.24967 fetch_fsmap learned FSMap version 23 2023-07-06T20:18:33.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.329+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/141173933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555e4fc2bc10 con 0x555e4fec4820 2023-07-06T20:18:33.340 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.331+0000 7f7e3cff9700 1 -- 172.21.15.69:0/141173933 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e44032020 con 0x555e4fec4820 2023-07-06T20:18:33.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.331+0000 7f7e3cff9700 1 -- 172.21.15.69:0/141173933 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7e44023b70 con 0x555e4fec4820 2023-07-06T20:18:33.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.331+0000 7f7e6f3d8b80 10 client.24967 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:18:33.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.331+0000 7f7e6f3d8b80 10 client.24967 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-06T20:18:33.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.331+0000 7f7e6f3d8b80 20 client.24967 populate_metadata read hostname 'smithi069' 2023-07-06T20:18:33.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.331+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/141173933 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x555e4fed8bb0 0x555e4fed7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.331+0000 7f7e6f3d8b80 4 client.24967 mds_command: new command op to 24539 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:18:33.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.331+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/141173933 --> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e4fe92d10 con 0x555e4fed8bb0 2023-07-06T20:18:33.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.331+0000 7f7e5979b700 1 --2- 172.21.15.69:0/141173933 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x555e4fed8bb0 0x555e4fed7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.332+0000 7f7e5979b700 1 --2- 172.21.15.69:0/141173933 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x555e4fed8bb0 0x555e4fed7f60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b14db796056eaa4 server_cookie=b214b8e9949fb3da in_seq=0 out_seq=0 2023-07-06T20:18:33.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.332+0000 7f7e3cff9700 10 client.24967 ms_handle_connect on v2:172.21.15.98:6836/4010267682 2023-07-06T20:18:33.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.332+0000 7f7e3cff9700 1 -- 172.21.15.69:0/141173933 <== mds.1 v2:172.21.15.98:6836/4010267682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555e4fe92d10 con 0x555e4fed8bb0 2023-07-06T20:18:33.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.332+0000 7f7e3cff9700 10 client.24967 handle_command_reply: tid=0 2023-07-06T20:18:33.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.332+0000 7f7e6f3d8b80 1 client.24967 shutdown 2023-07-06T20:18:33.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.333+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/141173933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 msgr2=0x555e4fecddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.333+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/141173933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fecddd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7e44015000 tx=0x7f7e44005040 comp rx=0 tx=0).stop 2023-07-06T20:18:33.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.333+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/141173933 shutdown_connections 2023-07-06T20:18:33.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.333+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/141173933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec5340 0x555e4fed4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.333+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/141173933 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x555e4fed8bb0 0x555e4fed7f60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.333+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/141173933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec3ed0 0x555e4fecd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.333+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/141173933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fecddd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.344 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.333+0000 7f7e3cff9700 0 client.24967 ms_handle_reset on v2:172.21.15.98:6836/4010267682 2023-07-06T20:18:33.344 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.333+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/141173933 >> 172.21.15.69:0/141173933 conn(0x555e4feb95f0 msgr2=0x555e4febe060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.344 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.333+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/141173933 shutdown_connections 2023-07-06T20:18:33.344 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.333+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/141173933 wait complete. 2023-07-06T20:18:33.344 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.333+0000 7f7e6f3d8b80 20 client.24967 trim_cache size 0 max 16384 2023-07-06T20:18:33.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.352+0000 7f7e6f3d8b80 1 Processor -- start 2023-07-06T20:18:33.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.352+0000 7f7e6f3d8b80 1 -- start start 2023-07-06T20:18:33.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.353+0000 7f7e6f3d8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec3ed0 0x555e4fecc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.353+0000 7f7e6f3d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fed4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.353+0000 7f7e6f3d8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec5340 0x555e4fecb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.353+0000 7f7e6f3d8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4feceab0 con 0x555e4fec3ed0 2023-07-06T20:18:33.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.353+0000 7f7e6f3d8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fde37c0 con 0x555e4fec5340 2023-07-06T20:18:33.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.353+0000 7f7e6f3d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fdd3480 con 0x555e4fec4820 2023-07-06T20:18:33.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.353+0000 7f7e5979b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec3ed0 0x555e4fecc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.353+0000 7f7e59f9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec5340 0x555e4fecb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.353+0000 7f7e5979b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec3ed0 0x555e4fecc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58938/0 (socket says 172.21.15.69:58938) 2023-07-06T20:18:33.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.353+0000 7f7e5979b700 1 -- 172.21.15.69:0/2951500919 learned_addr learned my addr 172.21.15.69:0/2951500919 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.354+0000 7f7e5979b700 1 -- 172.21.15.69:0/2951500919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 msgr2=0x555e4fed4dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:18:33.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.354+0000 7f7e5979b700 1 --2- 172.21.15.69:0/2951500919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fed4dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.354+0000 7f7e5979b700 1 -- 172.21.15.69:0/2951500919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec5340 msgr2=0x555e4fecb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.358 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.354+0000 7f7e5979b700 1 --2- 172.21.15.69:0/2951500919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec5340 0x555e4fecb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.358 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.354+0000 7f7e5979b700 1 -- 172.21.15.69:0/2951500919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4fec5c30 con 0x555e4fec3ed0 2023-07-06T20:18:33.358 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.354+0000 7f7e59f9c700 1 --2- 172.21.15.69:0/2951500919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec5340 0x555e4fecb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:33.358 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.354+0000 7f7e5979b700 1 --2- 172.21.15.69:0/2951500919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec3ed0 0x555e4fecc540 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c2d0 tx=0x7f7e500d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5210e4ad5de83d6e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:32 smithi156 ceph-mon[129280]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-06T20:18:33.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:32 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2924567925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:33.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:32 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2680077358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:33.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.354+0000 7f7e3ffff700 1 -- 172.21.15.69:0/2951500919 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50099ea0 con 0x555e4fec3ed0 2023-07-06T20:18:33.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.354+0000 7f7e3ffff700 1 -- 172.21.15.69:0/2951500919 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e5009ded0 con 0x555e4fec3ed0 2023-07-06T20:18:33.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.354+0000 7f7e3ffff700 1 -- 172.21.15.69:0/2951500919 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5009a360 con 0x555e4fec3ed0 2023-07-06T20:18:33.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.355+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/2951500919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec3ed0 msgr2=0x555e4fecc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.355+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/2951500919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec3ed0 0x555e4fecc540 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c2d0 tx=0x7f7e500d1040 comp rx=0 tx=0).stop 2023-07-06T20:18:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.355+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/2951500919 shutdown_connections 2023-07-06T20:18:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.355+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/2951500919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec5340 0x555e4fecb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.355+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/2951500919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec3ed0 0x555e4fecc540 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.355+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/2951500919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fed4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.355+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/2951500919 >> 172.21.15.69:0/2951500919 conn(0x555e4feb95f0 msgr2=0x555e4febe060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.355+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/2951500919 shutdown_connections 2023-07-06T20:18:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.355+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/2951500919 wait complete. 2023-07-06T20:18:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.355+0000 7f7e6f3d8b80 1 -- start start 2023-07-06T20:18:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.356+0000 7f7e6f3d8b80 1 Processor -- start 2023-07-06T20:18:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.356+0000 7f7e6f3d8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec3ed0 0x555e4fed01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.356+0000 7f7e6f3d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fed0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.356+0000 7f7e6f3d8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec5340 0x555e4fed3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.356+0000 7f7e6f3d8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fed3c60 con 0x555e4fec5340 2023-07-06T20:18:33.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.356+0000 7f7e6f3d8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fed3dd0 con 0x555e4fec3ed0 2023-07-06T20:18:33.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.356+0000 7f7e6f3d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fed3f90 con 0x555e4fec4820 2023-07-06T20:18:33.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e6f3d8b80 10 client.0 _ll_register_callbacks cb 0x7f7e4400bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:18:33.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e58f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fed0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e58f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fed0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:42612/0 (socket says 172.21.15.69:42612) 2023-07-06T20:18:33.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e58f9a700 1 -- 172.21.15.69:0/808242659 learned_addr learned my addr 172.21.15.69:0/808242659 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e5979b700 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec3ed0 0x555e4fed01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e59f9c700 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec5340 0x555e4fed3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e58f9a700 1 -- 172.21.15.69:0/808242659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec3ed0 msgr2=0x555e4fed01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e58f9a700 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec3ed0 0x555e4fed01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e58f9a700 1 -- 172.21.15.69:0/808242659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec5340 msgr2=0x555e4fed3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e58f9a700 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec5340 0x555e4fed3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e58f9a700 1 -- 172.21.15.69:0/808242659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e4fc42d90 con 0x555e4fec4820 2023-07-06T20:18:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e58f9a700 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fed0700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e440098e0 tx=0x7f7e4400e690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb8ccd83e4120eb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.357+0000 7f7e3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:18:33.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.360+0000 7f7e3cff9700 1 -- 172.21.15.69:0/808242659 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4401d040 con 0x555e4fec4820 2023-07-06T20:18:33.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.360+0000 7f7e3cff9700 1 -- 172.21.15.69:0/808242659 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f7e44000e40 con 0x555e4fec4820 2023-07-06T20:18:33.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.360+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/808242659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e4c018e60 con 0x555e4fec4820 2023-07-06T20:18:33.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.360+0000 7f7e3cff9700 1 -- 172.21.15.69:0/808242659 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e44005390 con 0x555e4fec4820 2023-07-06T20:18:33.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.360+0000 7f7e6f3d8b80 10 client.24970 fetch_fsmap learned FSMap version 23 2023-07-06T20:18:33.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.360+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/808242659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f7e541a01d0 con 0x555e4fec4820 2023-07-06T20:18:33.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.360+0000 7f7e3cff9700 1 -- 172.21.15.69:0/808242659 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e44044020 con 0x555e4fec4820 2023-07-06T20:18:33.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.361+0000 7f7e3cff9700 1 -- 172.21.15.69:0/808242659 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7e44035070 con 0x555e4fec4820 2023-07-06T20:18:33.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.361+0000 7f7e6f3d8b80 10 client.24970 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:18:33.369 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.361+0000 7f7e6f3d8b80 10 client.24970 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-06T20:18:33.369 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.361+0000 7f7e6f3d8b80 20 client.24970 populate_metadata read hostname 'smithi069' 2023-07-06T20:18:33.369 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.361+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x555e4ff31f70 0x555e4ff2ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.369 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.361+0000 7f7e6f3d8b80 4 client.24970 mds_command: new command op to 24539 tid=0[{"prefix": "status"}] 2023-07-06T20:18:33.369 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.361+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/808242659 --> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555e4fe0de60 con 0x555e4ff31f70 2023-07-06T20:18:33.369 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.361+0000 7f7e5979b700 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x555e4ff31f70 0x555e4ff2ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.369 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.361+0000 7f7e5979b700 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x555e4ff31f70 0x555e4ff2ea40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6005332f6ab38352 server_cookie=a825e81980bcdd8a in_seq=0 out_seq=0 2023-07-06T20:18:33.370 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.361+0000 7f7e3cff9700 10 client.24970 ms_handle_connect on v2:172.21.15.98:6836/4010267682 2023-07-06T20:18:33.370 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e3cff9700 1 -- 172.21.15.69:0/808242659 <== mds.1 v2:172.21.15.98:6836/4010267682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x555e4fe0de60 con 0x555e4ff31f70 2023-07-06T20:18:33.370 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e3cff9700 10 client.24970 handle_command_reply: tid=0 2023-07-06T20:18:33.370 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e6f3d8b80 1 client.24970 shutdown 2023-07-06T20:18:33.370 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/808242659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 msgr2=0x555e4fed0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fed0700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e440098e0 tx=0x7f7e4400e690 comp rx=0 tx=0).stop 2023-07-06T20:18:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/808242659 shutdown_connections 2023-07-06T20:18:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x555e4ff31f70 0x555e4ff2ea40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e4fec5340 0x555e4fed3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e4fec3ed0 0x555e4fed01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e3cff9700 0 client.24970 ms_handle_reset on v2:172.21.15.98:6836/4010267682 2023-07-06T20:18:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e6f3d8b80 1 --2- 172.21.15.69:0/808242659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4fec4820 0x555e4fed0700 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/808242659 >> 172.21.15.69:0/808242659 conn(0x555e4feb95f0 msgr2=0x555e4fed7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/808242659 shutdown_connections 2023-07-06T20:18:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e6f3d8b80 1 -- 172.21.15.69:0/808242659 wait complete. 2023-07-06T20:18:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.362+0000 7f7e6f3d8b80 20 client.24970 trim_cache size 0 max 16384 2023-07-06T20:18:33.373 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:18:33.373 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "e3e8ec0c-1c38-11ee-9b2f-001a4aab830c", 2023-07-06T20:18:33.373 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2023-07-06T20:18:33.373 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24539, 2023-07-06T20:18:33.373 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2023-07-06T20:18:33.374 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2023-07-06T20:18:33.374 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2023-07-06T20:18:33.374 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 61.752869726999997, 2023-07-06T20:18:33.374 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 23, 2023-07-06T20:18:33.374 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 85, 2023-07-06T20:18:33.374 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 83, 2023-07-06T20:18:33.374 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 81.719374158999997 2023-07-06T20:18:33.375 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:18:33.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:32 smithi098 ceph-mon[128968]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-06T20:18:33.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:32 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2924567925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:33.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:32 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2680077358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:33.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.363+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/1359872811 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7e400a1260 msgr2=0x7f7e400a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.363+0000 7f7e5b9ff700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7e400a1260 0x7f7e400a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c020000 tx=0x7f7e4c021000 comp rx=0 tx=0).stop 2023-07-06T20:18:33.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.363+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/1359872811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e54101e40 msgr2=0x7f7e54197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.363+0000 7f7e5b9ff700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e54101e40 0x7f7e54197bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000e9a0 tx=0x7f7e5000ecb0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.363+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/1359872811 shutdown_connections 2023-07-06T20:18:33.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.363+0000 7f7e5b9ff700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7e400a1260 0x7f7e400a3710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.363+0000 7f7e5b9ff700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e54103940 0x7f7e5419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.363+0000 7f7e5b9ff700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e54101e40 0x7f7e54197bf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.378 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.363+0000 7f7e5b9ff700 1 --2- 172.21.15.69:0/1359872811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e54102f90 0x7f7e54198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.378 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.363+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/1359872811 >> 172.21.15.69:0/1359872811 conn(0x7f7e540fd480 msgr2=0x7f7e541064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.378 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.363+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/1359872811 shutdown_connections 2023-07-06T20:18:33.378 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.363+0000 7f7e5b9ff700 1 -- 172.21.15.69:0/1359872811 wait complete. 2023-07-06T20:18:33.384 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-06T20:18:33.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.588+0000 7f1e023ff700 1 -- 172.21.15.69:0/2312804423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc0ffc10 msgr2=0x7f1dfc100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.588+0000 7f1e023ff700 1 --2- 172.21.15.69:0/2312804423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc0ffc10 0x7f1dfc100020 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009990 tx=0x7f1dec009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.589+0000 7f1e023ff700 1 -- 172.21.15.69:0/2312804423 shutdown_connections 2023-07-06T20:18:33.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.589+0000 7f1e023ff700 1 --2- 172.21.15.69:0/2312804423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dfc101790 0x7f1dfc108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.589+0000 7f1e023ff700 1 --2- 172.21.15.69:0/2312804423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc0ffc10 0x7f1dfc100020 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.589+0000 7f1e023ff700 1 --2- 172.21.15.69:0/2312804423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dfc100e00 0x7f1dfc101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.589+0000 7f1e023ff700 1 -- 172.21.15.69:0/2312804423 >> 172.21.15.69:0/2312804423 conn(0x7f1dfc0fb210 msgr2=0x7f1dfc0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.589+0000 7f1e023ff700 1 -- 172.21.15.69:0/2312804423 shutdown_connections 2023-07-06T20:18:33.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.589+0000 7f1e023ff700 1 -- 172.21.15.69:0/2312804423 wait complete. 2023-07-06T20:18:33.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1e023ff700 1 Processor -- start 2023-07-06T20:18:33.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1e023ff700 1 -- start start 2023-07-06T20:18:33.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1e023ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc0ffc10 0x7f1dfc197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1e023ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dfc100e00 0x7f1dfc198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1e023ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dfc101790 0x7f1dfc19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1e023ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc19f740 con 0x7f1dfc0ffc10 2023-07-06T20:18:33.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1e023ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc19f880 con 0x7f1dfc101790 2023-07-06T20:18:33.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1e023ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc19fa10 con 0x7f1dfc100e00 2023-07-06T20:18:33.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc0ffc10 0x7f1dfc197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1dfb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dfc100e00 0x7f1dfc198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1dfb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dfc100e00 0x7f1dfc198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:42636/0 (socket says 172.21.15.69:42636) 2023-07-06T20:18:33.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1dfb7fe700 1 -- 172.21.15.69:0/4183721106 learned_addr learned my addr 172.21.15.69:0/4183721106 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc0ffc10 0x7f1dfc197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58968/0 (socket says 172.21.15.69:58968) 2023-07-06T20:18:33.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.590+0000 7f1e0099c700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dfc101790 0x7f1dfc19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1dfb7fe700 1 -- 172.21.15.69:0/4183721106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dfc101790 msgr2=0x7f1dfc19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1dfb7fe700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dfc101790 0x7f1dfc19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1dfb7fe700 1 -- 172.21.15.69:0/4183721106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc0ffc10 msgr2=0x7f1dfc197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1dfb7fe700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc0ffc10 0x7f1dfc197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1dfb7fe700 1 -- 172.21.15.69:0/4183721106 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec0095f0 con 0x7f1dfc100e00 2023-07-06T20:18:33.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1dfbfff700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc0ffc10 0x7f1dfc197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:33.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1e0099c700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dfc101790 0x7f1dfc19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:33.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1dfb7fe700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dfc100e00 0x7f1dfc198100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1df000e9a0 tx=0x7f1df000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1df97fa700 1 -- 172.21.15.69:0/4183721106 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0009790 con 0x7f1dfc100e00 2023-07-06T20:18:33.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1e023ff700 1 -- 172.21.15.69:0/4183721106 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc19fcf0 con 0x7f1dfc100e00 2023-07-06T20:18:33.598 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1df97fa700 1 -- 172.21.15.69:0/4183721106 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df0010430 con 0x7f1dfc100e00 2023-07-06T20:18:33.598 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1df97fa700 1 -- 172.21.15.69:0/4183721106 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0018680 con 0x7f1dfc100e00 2023-07-06T20:18:33.598 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.591+0000 7f1e023ff700 1 -- 172.21.15.69:0/4183721106 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc1a01f0 con 0x7f1dfc100e00 2023-07-06T20:18:33.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.594+0000 7f1df97fa700 1 -- 172.21.15.69:0/4183721106 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1df00105a0 con 0x7f1dfc100e00 2023-07-06T20:18:33.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.596+0000 7f1e15cdab80 1 Processor -- start 2023-07-06T20:18:33.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.596+0000 7f1e15cdab80 1 -- start start 2023-07-06T20:18:33.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.596+0000 7f1e15cdab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae58159ed0 0x55ae5815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1e15cdab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815a820 0x55ae5815adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1e15cdab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5815b340 0x55ae5815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1e15cdab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5805ec60 con 0x55ae5815a820 2023-07-06T20:18:33.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1e15cdab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae580b19f0 con 0x55ae5815b340 2023-07-06T20:18:33.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1e15cdab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae580aae80 con 0x55ae58159ed0 2023-07-06T20:18:33.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1df97fa700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1de40a1260 0x7f1de40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815a820 0x55ae5815adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815a820 0x55ae5815adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58972/0 (socket says 172.21.15.69:58972) 2023-07-06T20:18:33.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1dfbfff700 1 -- 172.21.15.69:0/3796093025 learned_addr learned my addr 172.21.15.69:0/3796093025 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1df97fa700 1 -- 172.21.15.69:0/4183721106 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1df0014070 con 0x7f1dfc100e00 2023-07-06T20:18:33.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1dfb7fe700 1 --2- 172.21.15.69:0/3796093025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5815b340 0x55ae5815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1e0099c700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1de40a1260 0x7f1de40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1dfbfff700 1 -- 172.21.15.69:0/3796093025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae58159ed0 msgr2=0x55ae5815a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:18:33.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1dfbfff700 1 --2- 172.21.15.69:0/3796093025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae58159ed0 0x55ae5815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1dfbfff700 1 -- 172.21.15.69:0/3796093025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5815b340 msgr2=0x55ae5815b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1dfbfff700 1 --2- 172.21.15.69:0/3796093025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5815b340 0x55ae5815b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1dfbfff700 1 -- 172.21.15.69:0/3796093025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae57ed8d90 con 0x55ae5815a820 2023-07-06T20:18:33.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.597+0000 7f1dfb7fe700 1 --2- 172.21.15.69:0/3796093025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5815b340 0x55ae5815b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:33.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.598+0000 7f1dfbfff700 1 --2- 172.21.15.69:0/3796093025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815a820 0x55ae5815adb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc100c60 tx=0x7f1dec005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4371f1dacf8b9c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.598+0000 7f1e0099c700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1de40a1260 0x7f1de40a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1df4003ec0 tx=0x7f1df4009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.598+0000 7f1de27fc700 1 -- 172.21.15.69:0/3796093025 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec01d070 con 0x55ae5815a820 2023-07-06T20:18:33.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.598+0000 7f1de27fc700 1 -- 172.21.15.69:0/3796093025 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dec01e040 con 0x55ae5815a820 2023-07-06T20:18:33.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.598+0000 7f1de27fc700 1 -- 172.21.15.69:0/3796093025 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec004dc0 con 0x55ae5815a820 2023-07-06T20:18:33.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.598+0000 7f1e15cdab80 1 -- 172.21.15.69:0/3796093025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815a820 msgr2=0x55ae5815adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.598+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/3796093025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815a820 0x55ae5815adb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc100c60 tx=0x7f1dec005ee0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.599+0000 7f1e15cdab80 1 -- 172.21.15.69:0/3796093025 shutdown_connections 2023-07-06T20:18:33.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.599+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/3796093025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5815b340 0x55ae5815b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.599+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/3796093025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae58159ed0 0x55ae5815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.599+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/3796093025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815a820 0x55ae5815adb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.599+0000 7f1e15cdab80 1 -- 172.21.15.69:0/3796093025 >> 172.21.15.69:0/3796093025 conn(0x55ae5814f5f0 msgr2=0x55ae581549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.599+0000 7f1e15cdab80 1 -- 172.21.15.69:0/3796093025 shutdown_connections 2023-07-06T20:18:33.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.599+0000 7f1e15cdab80 1 -- 172.21.15.69:0/3796093025 wait complete. 2023-07-06T20:18:33.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.599+0000 7f1e15cdab80 1 -- start start 2023-07-06T20:18:33.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.599+0000 7f1e15cdab80 1 Processor -- start 2023-07-06T20:18:33.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1e15cdab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae58159ed0 0x55ae58163800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1e15cdab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5815a820 0x55ae58163d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1e15cdab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815b340 0x55ae5816ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1e15cdab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5804aa70 con 0x55ae5815b340 2023-07-06T20:18:33.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1e15cdab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5804a770 con 0x55ae5815a820 2023-07-06T20:18:33.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1e15cdab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae57f2e8d0 con 0x55ae58159ed0 2023-07-06T20:18:33.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1e15cdab80 10 client.0 _ll_register_callbacks cb 0x55ae58048710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:18:33.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae58159ed0 0x55ae58163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae58159ed0 0x55ae58163800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:42648/0 (socket says 172.21.15.69:42648) 2023-07-06T20:18:33.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1dfbfff700 1 -- 172.21.15.69:0/4183740675 learned_addr learned my addr 172.21.15.69:0/4183740675 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1e0099c700 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815b340 0x55ae5816ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1dfb7fe700 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5815a820 0x55ae58163d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1dfbfff700 1 -- 172.21.15.69:0/4183740675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5815a820 msgr2=0x55ae58163d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1dfbfff700 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5815a820 0x55ae58163d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1dfbfff700 1 -- 172.21.15.69:0/4183740675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815b340 msgr2=0x55ae5816ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1dfbfff700 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815b340 0x55ae5816ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1dfbfff700 1 -- 172.21.15.69:0/4183740675 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc1a01f0 con 0x55ae58159ed0 2023-07-06T20:18:33.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.600+0000 7f1e0099c700 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815b340 0x55ae5816ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:33.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.601+0000 7f1dfbfff700 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae58159ed0 0x55ae58163800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0098e0 tx=0x7f1dec022b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13a33fdc36dea109 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.601+0000 7f1ddb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:18:33.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.601+0000 7f1ddb7fe700 1 -- 172.21.15.69:0/4183740675 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec01d030 con 0x55ae58159ed0 2023-07-06T20:18:33.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.601+0000 7f1ddb7fe700 1 -- 172.21.15.69:0/4183740675 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1dec00f460 con 0x55ae58159ed0 2023-07-06T20:18:33.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.601+0000 7f1e15cdab80 1 -- 172.21.15.69:0/4183740675 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae580044a0 con 0x55ae58159ed0 2023-07-06T20:18:33.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.601+0000 7f1ddb7fe700 1 -- 172.21.15.69:0/4183740675 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dec004760 con 0x55ae58159ed0 2023-07-06T20:18:33.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.601+0000 7f1e15cdab80 10 client.24979 fetch_fsmap learned FSMap version 23 2023-07-06T20:18:33.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.601+0000 7f1e15cdab80 1 -- 172.21.15.69:0/4183740675 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f1dfc108980 con 0x55ae58159ed0 2023-07-06T20:18:33.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.602+0000 7f1ddb7fe700 1 -- 172.21.15.69:0/4183740675 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae580044a0 con 0x55ae58159ed0 2023-07-06T20:18:33.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.602+0000 7f1ddb7fe700 1 -- 172.21.15.69:0/4183740675 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1dec02c490 con 0x55ae58159ed0 2023-07-06T20:18:33.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.602+0000 7f1e15cdab80 10 client.24979 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:18:33.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.602+0000 7f1e15cdab80 10 client.24979 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-06T20:18:33.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.602+0000 7f1e15cdab80 20 client.24979 populate_metadata read hostname 'smithi069' 2023-07-06T20:18:33.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.602+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ae5816e620 0x55ae5816dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.602+0000 7f1e15cdab80 4 client.24979 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:18:33.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.602+0000 7f1e15cdab80 1 -- 172.21.15.69:0/4183740675 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae58128d10 con 0x55ae5816e620 2023-07-06T20:18:33.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.602+0000 7f1dfb7fe700 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ae5816e620 0x55ae5816dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.603+0000 7f1ddb7fe700 10 client.24979 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:18:33.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.603+0000 7f1dfb7fe700 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ae5816e620 0x55ae5816dcb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db3e169925868640 server_cookie=fb5fccbaaf78717 in_seq=0 out_seq=0 2023-07-06T20:18:33.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.603+0000 7f1ddb7fe700 1 -- 172.21.15.69:0/4183740675 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ae58128d10 con 0x55ae5816e620 2023-07-06T20:18:33.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.603+0000 7f1ddb7fe700 10 client.24979 handle_command_reply: tid=0 2023-07-06T20:18:33.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.603+0000 7f1e15cdab80 1 client.24979 shutdown 2023-07-06T20:18:33.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.604+0000 7f1e15cdab80 1 -- 172.21.15.69:0/4183740675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae58159ed0 msgr2=0x55ae58163800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.604+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae58159ed0 0x55ae58163800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0098e0 tx=0x7f1dec022b10 comp rx=0 tx=0).stop 2023-07-06T20:18:33.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.604+0000 7f1e15cdab80 1 -- 172.21.15.69:0/4183740675 shutdown_connections 2023-07-06T20:18:33.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.604+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ae5816e620 0x55ae5816dcb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.604+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815b340 0x55ae5816ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.604+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae58159ed0 0x55ae58163800 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.604+0000 7f1ddb7fe700 0 client.24979 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:18:33.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.604+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/4183740675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5815a820 0x55ae58163d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.604+0000 7f1e15cdab80 1 -- 172.21.15.69:0/4183740675 >> 172.21.15.69:0/4183740675 conn(0x55ae5814f5f0 msgr2=0x55ae58154060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.604+0000 7f1e15cdab80 1 -- 172.21.15.69:0/4183740675 shutdown_connections 2023-07-06T20:18:33.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.604+0000 7f1e15cdab80 1 -- 172.21.15.69:0/4183740675 wait complete. 2023-07-06T20:18:33.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.604+0000 7f1e15cdab80 20 client.24979 trim_cache size 0 max 16384 2023-07-06T20:18:33.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.622+0000 7f1e15cdab80 1 Processor -- start 2023-07-06T20:18:33.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.622+0000 7f1e15cdab80 1 -- start start 2023-07-06T20:18:33.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1e15cdab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5814f5f0 0x55ae5816dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1e15cdab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae58159ed0 0x55ae5816aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1e15cdab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815a820 0x55ae58154060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1e15cdab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae58079790 con 0x55ae5815a820 2023-07-06T20:18:33.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1e15cdab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae57f3ade0 con 0x55ae58159ed0 2023-07-06T20:18:33.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1e15cdab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae58069480 con 0x55ae5814f5f0 2023-07-06T20:18:33.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5814f5f0 0x55ae5816dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5814f5f0 0x55ae5816dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:42652/0 (socket says 172.21.15.69:42652) 2023-07-06T20:18:33.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1e0099c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815a820 0x55ae58154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1dfbfff700 1 -- 172.21.15.69:0/3107462172 learned_addr learned my addr 172.21.15.69:0/3107462172 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1dfb7fe700 1 --2- 172.21.15.69:0/3107462172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae58159ed0 0x55ae5816aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1dfbfff700 1 -- 172.21.15.69:0/3107462172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae58159ed0 msgr2=0x55ae5816aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1dfbfff700 1 --2- 172.21.15.69:0/3107462172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae58159ed0 0x55ae5816aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1dfbfff700 1 -- 172.21.15.69:0/3107462172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815a820 msgr2=0x55ae58154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1dfbfff700 1 --2- 172.21.15.69:0/3107462172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815a820 0x55ae58154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.623+0000 7f1dfbfff700 1 -- 172.21.15.69:0/3107462172 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae5815bc30 con 0x55ae5814f5f0 2023-07-06T20:18:33.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.624+0000 7f1dfbfff700 1 --2- 172.21.15.69:0/3107462172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5814f5f0 0x55ae5816dcb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00b620 tx=0x7f1dec009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b66d9d36d348c2c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.624+0000 7f1de2ffd700 1 -- 172.21.15.69:0/3107462172 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec01d040 con 0x55ae5814f5f0 2023-07-06T20:18:33.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.624+0000 7f1de2ffd700 1 -- 172.21.15.69:0/3107462172 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dec004710 con 0x55ae5814f5f0 2023-07-06T20:18:33.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.624+0000 7f1de2ffd700 1 -- 172.21.15.69:0/3107462172 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec02b680 con 0x55ae5814f5f0 2023-07-06T20:18:33.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.624+0000 7f1e15cdab80 1 -- 172.21.15.69:0/3107462172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5814f5f0 msgr2=0x55ae5816dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.624+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/3107462172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5814f5f0 0x55ae5816dcb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00b620 tx=0x7f1dec009310 comp rx=0 tx=0).stop 2023-07-06T20:18:33.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.624+0000 7f1e15cdab80 1 -- 172.21.15.69:0/3107462172 shutdown_connections 2023-07-06T20:18:33.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.624+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/3107462172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae5815a820 0x55ae58154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.624+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/3107462172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5814f5f0 0x55ae5816dcb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.624+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/3107462172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae58159ed0 0x55ae5816aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.624+0000 7f1e15cdab80 1 -- 172.21.15.69:0/3107462172 >> 172.21.15.69:0/3107462172 conn(0x55ae581628b0 msgr2=0x55ae58177350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.625+0000 7f1e15cdab80 1 -- 172.21.15.69:0/3107462172 shutdown_connections 2023-07-06T20:18:33.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.625+0000 7f1e15cdab80 1 -- 172.21.15.69:0/3107462172 wait complete. 2023-07-06T20:18:33.633 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.625+0000 7f1e15cdab80 1 -- start start 2023-07-06T20:18:33.638 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.625+0000 7f1e15cdab80 1 Processor -- start 2023-07-06T20:18:33.638 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1e15cdab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5814f5f0 0x55ae581604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.638 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1e15cdab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae58159ed0 0x55ae5816aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1e15cdab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5815a820 0x55ae581609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1e15cdab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae58161050 con 0x55ae58159ed0 2023-07-06T20:18:33.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1e15cdab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae581c2480 con 0x55ae5814f5f0 2023-07-06T20:18:33.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1e15cdab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae581c2610 con 0x55ae5815a820 2023-07-06T20:18:33.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1e15cdab80 10 client.0 _ll_register_callbacks cb 0x7f1dec009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:18:33.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1dfb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae58159ed0 0x55ae5816aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5814f5f0 0x55ae581604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.641 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1dfbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5814f5f0 0x55ae581604a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:57062/0 (socket says 172.21.15.69:57062) 2023-07-06T20:18:33.641 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1dfbfff700 1 -- 172.21.15.69:0/749166696 learned_addr learned my addr 172.21.15.69:0/749166696 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:33.641 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1dfb7fe700 1 -- 172.21.15.69:0/749166696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5815a820 msgr2=0x55ae581609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.641 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1dfb7fe700 1 --2- 172.21.15.69:0/749166696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5815a820 0x55ae581609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.641 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1dfb7fe700 1 -- 172.21.15.69:0/749166696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5814f5f0 msgr2=0x55ae581604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:33.641 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1dfb7fe700 1 --2- 172.21.15.69:0/749166696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5814f5f0 0x55ae581604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.626+0000 7f1dfb7fe700 1 -- 172.21.15.69:0/749166696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc108980 con 0x55ae58159ed0 2023-07-06T20:18:33.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.627+0000 7f1dfb7fe700 1 --2- 172.21.15.69:0/749166696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae58159ed0 0x55ae5816aca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1df00ca570 tx=0x7f1df0010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57cf766e8bc0020 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:33.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.627+0000 7f1ddb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:18:33.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.627+0000 7f1ddb7fe700 1 -- 172.21.15.69:0/749166696 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df009a100 con 0x55ae58159ed0 2023-07-06T20:18:33.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.627+0000 7f1ddb7fe700 1 -- 172.21.15.69:0/749166696 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1df00b8200 con 0x55ae58159ed0 2023-07-06T20:18:33.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.627+0000 7f1e15cdab80 1 -- 172.21.15.69:0/749166696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dec01e040 con 0x55ae58159ed0 2023-07-06T20:18:33.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.627+0000 7f1ddb7fe700 1 -- 172.21.15.69:0/749166696 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df009a260 con 0x55ae58159ed0 2023-07-06T20:18:33.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.627+0000 7f1e15cdab80 10 client.14970 fetch_fsmap learned FSMap version 23 2023-07-06T20:18:33.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.627+0000 7f1e15cdab80 1 -- 172.21.15.69:0/749166696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ae57ec1c10 con 0x55ae58159ed0 2023-07-06T20:18:33.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.627+0000 7f1ddb7fe700 1 -- 172.21.15.69:0/749166696 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dec01e040 con 0x55ae58159ed0 2023-07-06T20:18:33.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.627+0000 7f1ddb7fe700 1 -- 172.21.15.69:0/749166696 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1df00a1e60 con 0x55ae58159ed0 2023-07-06T20:18:33.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.627+0000 7f1e15cdab80 10 client.14970 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:18:33.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.628+0000 7f1e15cdab80 10 client.14970 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-06T20:18:33.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.628+0000 7f1e15cdab80 20 client.14970 populate_metadata read hostname 'smithi069' 2023-07-06T20:18:33.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.628+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/749166696 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ae5815b340 0x55ae581c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:33.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.628+0000 7f1e15cdab80 4 client.14970 mds_command: new command op to 24584 tid=0[{"prefix": "status"}] 2023-07-06T20:18:33.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.628+0000 7f1e15cdab80 1 -- 172.21.15.69:0/749166696 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ae580a3e60 con 0x55ae5815b340 2023-07-06T20:18:33.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.628+0000 7f1dfbfff700 1 --2- 172.21.15.69:0/749166696 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ae5815b340 0x55ae581c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:33.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.628+0000 7f1ddb7fe700 10 client.14970 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:18:33.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.628+0000 7f1dfbfff700 1 --2- 172.21.15.69:0/749166696 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ae5815b340 0x55ae581c3dc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6b3caf0d10eb9d server_cookie=98cf61e6cb0dea10 in_seq=0 out_seq=0 2023-07-06T20:18:33.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.628+0000 7f1ddb7fe700 1 -- 172.21.15.69:0/749166696 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ae580a3e60 con 0x55ae5815b340 2023-07-06T20:18:33.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.628+0000 7f1ddb7fe700 10 client.14970 handle_command_reply: tid=0 2023-07-06T20:18:33.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.628+0000 7f1e15cdab80 1 client.14970 shutdown 2023-07-06T20:18:33.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e15cdab80 1 -- 172.21.15.69:0/749166696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae58159ed0 msgr2=0x55ae5816aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/749166696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae58159ed0 0x55ae5816aca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1df00ca570 tx=0x7f1df0010c80 comp rx=0 tx=0).stop 2023-07-06T20:18:33.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e15cdab80 1 -- 172.21.15.69:0/749166696 shutdown_connections 2023-07-06T20:18:33.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/749166696 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ae5815b340 0x55ae581c3dc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/749166696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5815a820 0x55ae581609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/749166696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5814f5f0 0x55ae581604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.646 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e15cdab80 1 --2- 172.21.15.69:0/749166696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ae58159ed0 0x55ae5816aca0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.646 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1ddb7fe700 0 client.14970 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:18:33.646 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e15cdab80 1 -- 172.21.15.69:0/749166696 >> 172.21.15.69:0/749166696 conn(0x55ae581628b0 msgr2=0x55ae5816dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.646 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e15cdab80 1 -- 172.21.15.69:0/749166696 shutdown_connections 2023-07-06T20:18:33.646 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e15cdab80 1 -- 172.21.15.69:0/749166696 wait complete. 2023-07-06T20:18:33.646 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e15cdab80 20 client.14970 trim_cache size 0 max 16384 2023-07-06T20:18:33.647 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:18:33.647 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "e3e8ec0c-1c38-11ee-9b2f-001a4aab830c", 2023-07-06T20:18:33.647 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2023-07-06T20:18:33.647 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24584, 2023-07-06T20:18:33.647 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2023-07-06T20:18:33.647 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2023-07-06T20:18:33.648 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2023-07-06T20:18:33.648 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 65.043386022999996, 2023-07-06T20:18:33.648 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 23, 2023-07-06T20:18:33.648 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 85, 2023-07-06T20:18:33.648 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 83, 2023-07-06T20:18:33.648 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 71.255297228000003 2023-07-06T20:18:33.648 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:18:33.650 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e023ff700 1 -- 172.21.15.69:0/4183721106 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1de40a1260 msgr2=0x7f1de40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.650 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e023ff700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1de40a1260 0x7f1de40a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1df4003ec0 tx=0x7f1df4009920 comp rx=0 tx=0).stop 2023-07-06T20:18:33.650 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e023ff700 1 -- 172.21.15.69:0/4183721106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dfc100e00 msgr2=0x7f1dfc198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:33.650 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.629+0000 7f1e023ff700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dfc100e00 0x7f1dfc198100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1df000e9a0 tx=0x7f1df000ecb0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.650 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.630+0000 7f1e023ff700 1 -- 172.21.15.69:0/4183721106 shutdown_connections 2023-07-06T20:18:33.650 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.630+0000 7f1e023ff700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1de40a1260 0x7f1de40a3710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.650 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.630+0000 7f1e023ff700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dfc101790 0x7f1dfc19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.651 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.630+0000 7f1e023ff700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1dfc0ffc10 0x7f1dfc197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.651 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.630+0000 7f1e023ff700 1 --2- 172.21.15.69:0/4183721106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1dfc100e00 0x7f1dfc198100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:33.651 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.630+0000 7f1e023ff700 1 -- 172.21.15.69:0/4183721106 >> 172.21.15.69:0/4183721106 conn(0x7f1dfc0fb210 msgr2=0x7f1dfc0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:33.651 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.630+0000 7f1e023ff700 1 -- 172.21.15.69:0/4183721106 shutdown_connections 2023-07-06T20:18:33.651 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:33.630+0000 7f1e023ff700 1 -- 172.21.15.69:0/4183721106 wait complete. 2023-07-06T20:18:33.652 INFO:tasks.cephadm:Setup complete, yielding 2023-07-06T20:18:33.652 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-06T20:18:33.663 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2023-07-06T20:18:33.664 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- bash -c 'ceph orch status' 2023-07-06T20:18:34.415 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:34.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:34 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:34 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:34 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:34 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:34 smithi156 ceph-mon[129280]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:34.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:34 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:34 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:34 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:34 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:34 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:34 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:34 smithi098 ceph-mon[128968]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:34.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:34 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:34 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.684 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:34 smithi069 ceph-mon[126399]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:34.685 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:34.685 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:34 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:35.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:35 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:18:35.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:35 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:18:35.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:35 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:35.642 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:35 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:18:35.642 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:35 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:18:35.642 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:35 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:35.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:35 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:18:35.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:35 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:18:35.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:35 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:18:35.905 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.903+0000 7fb0b0864700 1 -- 172.21.15.69:0/2957479551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0ac0ffcc0 msgr2=0x7fb0ac1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:35.905 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.903+0000 7fb0b0864700 1 --2- 172.21.15.69:0/2957479551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0ac0ffcc0 0x7fb0ac1000d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0098f0 tx=0x7fb09c009c00 comp rx=0 tx=0).stop 2023-07-06T20:18:35.906 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.903+0000 7fb0b0864700 1 -- 172.21.15.69:0/2957479551 shutdown_connections 2023-07-06T20:18:35.906 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.903+0000 7fb0b0864700 1 --2- 172.21.15.69:0/2957479551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ac101850 0x7fb0ac108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:35.906 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.903+0000 7fb0b0864700 1 --2- 172.21.15.69:0/2957479551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0ac0ffcc0 0x7fb0ac1000d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:35.906 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.903+0000 7fb0b0864700 1 --2- 172.21.15.69:0/2957479551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0ac100ec0 0x7fb0ac101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:35.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.903+0000 7fb0b0864700 1 -- 172.21.15.69:0/2957479551 >> 172.21.15.69:0/2957479551 conn(0x7fb0ac0fb2d0 msgr2=0x7fb0ac0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:35.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.903+0000 7fb0b0864700 1 -- 172.21.15.69:0/2957479551 shutdown_connections 2023-07-06T20:18:35.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.903+0000 7fb0b0864700 1 -- 172.21.15.69:0/2957479551 wait complete. 2023-07-06T20:18:35.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.904+0000 7fb0b0864700 1 Processor -- start 2023-07-06T20:18:35.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.904+0000 7fb0b0864700 1 -- start start 2023-07-06T20:18:35.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.904+0000 7fb0b0864700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0ac0ffcc0 0x7fb0ac197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:35.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.904+0000 7fb0b0864700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0ac100ec0 0x7fb0ac1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:35.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0b0864700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ac101850 0x7fb0ac19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:35.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0b0864700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac19f800 con 0x7fb0ac100ec0 2023-07-06T20:18:35.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0b0864700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac19f940 con 0x7fb0ac0ffcc0 2023-07-06T20:18:35.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0b0864700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac19fb00 con 0x7fb0ac101850 2023-07-06T20:18:35.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0a9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0ac100ec0 0x7fb0ac1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:35.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0aa59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0ac0ffcc0 0x7fb0ac197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:35.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0aa59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0ac0ffcc0 0x7fb0ac197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:57086/0 (socket says 172.21.15.69:57086) 2023-07-06T20:18:35.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0aa59c700 1 -- 172.21.15.69:0/2085173869 learned_addr learned my addr 172.21.15.69:0/2085173869 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:35.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0aad9d700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ac101850 0x7fb0ac19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:35.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0a9d9b700 1 -- 172.21.15.69:0/2085173869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ac101850 msgr2=0x7fb0ac19f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:35.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0a9d9b700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ac101850 0x7fb0ac19f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:35.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0a9d9b700 1 -- 172.21.15.69:0/2085173869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0ac0ffcc0 msgr2=0x7fb0ac197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:35.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0a9d9b700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0ac0ffcc0 0x7fb0ac197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:35.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0a9d9b700 1 -- 172.21.15.69:0/2085173869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09c009520 con 0x7fb0ac100ec0 2023-07-06T20:18:35.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0aa59c700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0ac0ffcc0 0x7fb0ac197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:35.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.905+0000 7fb0a9d9b700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0ac100ec0 0x7fb0ac1981c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000e9c0 tx=0x7fb0a000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:35.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.906+0000 7fb09b7fe700 1 -- 172.21.15.69:0/2085173869 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a000c9c0 con 0x7fb0ac100ec0 2023-07-06T20:18:35.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.906+0000 7fb0b0864700 1 -- 172.21.15.69:0/2085173869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ac19fde0 con 0x7fb0ac100ec0 2023-07-06T20:18:35.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.906+0000 7fb09b7fe700 1 -- 172.21.15.69:0/2085173869 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0a0010430 con 0x7fb0ac100ec0 2023-07-06T20:18:35.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.906+0000 7fb09b7fe700 1 -- 172.21.15.69:0/2085173869 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0018680 con 0x7fb0ac100ec0 2023-07-06T20:18:35.912 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.906+0000 7fb0b0864700 1 -- 172.21.15.69:0/2085173869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ac1a0260 con 0x7fb0ac100ec0 2023-07-06T20:18:35.912 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.907+0000 7fb0b0864700 1 -- 172.21.15.69:0/2085173869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ac192390 con 0x7fb0ac100ec0 2023-07-06T20:18:35.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.909+0000 7fb09b7fe700 1 -- 172.21.15.69:0/2085173869 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb0a0010860 con 0x7fb0ac100ec0 2023-07-06T20:18:35.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.910+0000 7fb09b7fe700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb0940a11e0 0x7fb0940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:35.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.910+0000 7fb09b7fe700 1 -- 172.21.15.69:0/2085173869 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb0a0014070 con 0x7fb0ac100ec0 2023-07-06T20:18:35.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.910+0000 7fb0aa59c700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb0940a11e0 0x7fb0940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:35.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.911+0000 7fb0aa59c700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb0940a11e0 0x7fb0940a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00fa40 tx=0x7fb09c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:35.916 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:35.914+0000 7fb09b7fe700 1 -- 172.21.15.69:0/2085173869 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb0a0051770 con 0x7fb0ac100ec0 2023-07-06T20:18:36.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.125+0000 7fb0b0864700 1 -- 172.21.15.69:0/2085173869 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0ac060a80 con 0x7fb0940a11e0 2023-07-06T20:18:36.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.126+0000 7fb09b7fe700 1 -- 172.21.15.69:0/2085173869 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb0ac060a80 con 0x7fb0940a11e0 2023-07-06T20:18:36.129 INFO:teuthology.orchestra.run.smithi069.stdout:Backend: cephadm 2023-07-06T20:18:36.129 INFO:teuthology.orchestra.run.smithi069.stdout:Available: Yes 2023-07-06T20:18:36.129 INFO:teuthology.orchestra.run.smithi069.stdout:Paused: No 2023-07-06T20:18:36.131 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.129+0000 7fb0b0864700 1 -- 172.21.15.69:0/2085173869 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb0940a11e0 msgr2=0x7fb0940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:36.131 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.129+0000 7fb0b0864700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb0940a11e0 0x7fb0940a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00fa40 tx=0x7fb09c019040 comp rx=0 tx=0).stop 2023-07-06T20:18:36.131 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.129+0000 7fb0b0864700 1 -- 172.21.15.69:0/2085173869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0ac100ec0 msgr2=0x7fb0ac1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:36.131 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.129+0000 7fb0b0864700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0ac100ec0 0x7fb0ac1981c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000e9c0 tx=0x7fb0a000ecd0 comp rx=0 tx=0).stop 2023-07-06T20:18:36.131 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.129+0000 7fb0b0864700 1 -- 172.21.15.69:0/2085173869 shutdown_connections 2023-07-06T20:18:36.131 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.129+0000 7fb0b0864700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb0940a11e0 0x7fb0940a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:36.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.129+0000 7fb0b0864700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ac101850 0x7fb0ac19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:36.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.129+0000 7fb0b0864700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0ac0ffcc0 0x7fb0ac197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:36.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.129+0000 7fb0b0864700 1 --2- 172.21.15.69:0/2085173869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0ac100ec0 0x7fb0ac1981c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:36.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.129+0000 7fb0b0864700 1 -- 172.21.15.69:0/2085173869 >> 172.21.15.69:0/2085173869 conn(0x7fb0ac0fb2d0 msgr2=0x7fb0ac106890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:36.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.129+0000 7fb0b0864700 1 -- 172.21.15.69:0/2085173869 shutdown_connections 2023-07-06T20:18:36.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:36.129+0000 7fb0b0864700 1 -- 172.21.15.69:0/2085173869 wait complete. 2023-07-06T20:18:36.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:36 smithi069 ceph-mon[126399]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-06T20:18:36.591 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-06T20:18:36.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:36 smithi156 ceph-mon[129280]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-06T20:18:36.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:36 smithi098 ceph-mon[128968]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-06T20:18:37.308 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:37.659 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:37 smithi069 ceph-mon[126399]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:37.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:37 smithi156 ceph-mon[129280]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:37.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:37 smithi098 ceph-mon[128968]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:38.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.811+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/347569538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40ffcc0 msgr2=0x7fb2f41000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:38.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.811+0000 7fb2fb2fd700 1 --2- 172.21.15.69:0/347569538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40ffcc0 0x7fb2f41000d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009990 tx=0x7fb2f0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:38.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.811+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/347569538 shutdown_connections 2023-07-06T20:18:38.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.811+0000 7fb2fb2fd700 1 --2- 172.21.15.69:0/347569538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2f4101850 0x7fb2f4108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:38.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.811+0000 7fb2fb2fd700 1 --2- 172.21.15.69:0/347569538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40ffcc0 0x7fb2f41000d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:38.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.811+0000 7fb2fb2fd700 1 --2- 172.21.15.69:0/347569538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2f4100ec0 0x7fb2f4101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:38.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.811+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/347569538 >> 172.21.15.69:0/347569538 conn(0x7fb2f40fb2d0 msgr2=0x7fb2f40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:38.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.811+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/347569538 shutdown_connections 2023-07-06T20:18:38.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.811+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/347569538 wait complete. 2023-07-06T20:18:38.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.812+0000 7fb2fb2fd700 1 Processor -- start 2023-07-06T20:18:38.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.812+0000 7fb2fb2fd700 1 -- start start 2023-07-06T20:18:38.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.813+0000 7fb2fb2fd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2f40ffcc0 0x7fb2f41959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:38.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.813+0000 7fb2fb2fd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4100ec0 0x7fb2f4195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:38.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.813+0000 7fb2fb2fd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2f4101850 0x7fb2f419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:38.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.813+0000 7fb2fb2fd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f419d580 con 0x7fb2f4100ec0 2023-07-06T20:18:38.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.813+0000 7fb2fb2fd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f419d6c0 con 0x7fb2f40ffcc0 2023-07-06T20:18:38.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.813+0000 7fb2fb2fd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f419d850 con 0x7fb2f4101850 2023-07-06T20:18:38.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.813+0000 7fb2f8898700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4100ec0 0x7fb2f4195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:38.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.813+0000 7fb2f8898700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4100ec0 0x7fb2f4195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59042/0 (socket says 172.21.15.69:59042) 2023-07-06T20:18:38.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.813+0000 7fb2f8898700 1 -- 172.21.15.69:0/3169710857 learned_addr learned my addr 172.21.15.69:0/3169710857 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:38.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.813+0000 7fb2f9099700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2f40ffcc0 0x7fb2f41959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:38.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.813+0000 7fb2f989a700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2f4101850 0x7fb2f419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:38.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2f8898700 1 -- 172.21.15.69:0/3169710857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2f4101850 msgr2=0x7fb2f419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:38.818 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2f8898700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2f4101850 0x7fb2f419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:38.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2f8898700 1 -- 172.21.15.69:0/3169710857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2f40ffcc0 msgr2=0x7fb2f41959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:38.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2f8898700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2f40ffcc0 0x7fb2f41959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:38.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2f8898700 1 -- 172.21.15.69:0/3169710857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f00095f0 con 0x7fb2f4100ec0 2023-07-06T20:18:38.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2f989a700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2f4101850 0x7fb2f419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:38.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2f9099700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2f40ffcc0 0x7fb2f41959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:38.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2f8898700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4100ec0 0x7fb2f4195f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e9d0 tx=0x7fb2e400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:38.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2ea7fc700 1 -- 172.21.15.69:0/3169710857 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4009790 con 0x7fb2f4100ec0 2023-07-06T20:18:38.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/3169710857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f419db30 con 0x7fb2f4100ec0 2023-07-06T20:18:38.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2ea7fc700 1 -- 172.21.15.69:0/3169710857 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2e400cbd0 con 0x7fb2f4100ec0 2023-07-06T20:18:38.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2ea7fc700 1 -- 172.21.15.69:0/3169710857 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4018680 con 0x7fb2f4100ec0 2023-07-06T20:18:38.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.814+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/3169710857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f419e030 con 0x7fb2f4100ec0 2023-07-06T20:18:38.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.815+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/3169710857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f41072c0 con 0x7fb2f4100ec0 2023-07-06T20:18:38.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.817+0000 7fb2ea7fc700 1 -- 172.21.15.69:0/3169710857 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb2e400cd40 con 0x7fb2f4100ec0 2023-07-06T20:18:38.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.818+0000 7fb2ea7fc700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb2e00a11e0 0x7fb2e00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:38.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.819+0000 7fb2ea7fc700 1 -- 172.21.15.69:0/3169710857 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb2e4014070 con 0x7fb2f4100ec0 2023-07-06T20:18:38.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.819+0000 7fb2f9099700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb2e00a11e0 0x7fb2e00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:38.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.819+0000 7fb2f9099700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb2e00a11e0 0x7fb2e00a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000b2e0 tx=0x7fb2f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:38.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:38.822+0000 7fb2ea7fc700 1 -- 172.21.15.69:0/3169710857 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb2e4051770 con 0x7fb2f4100ec0 2023-07-06T20:18:38.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:38 smithi156 ceph-mon[129280]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:38.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:38 smithi098 ceph-mon[128968]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:39.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:38 smithi069 ceph-mon[126399]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:39.030 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.028+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/3169710857 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2f4060a80 con 0x7fb2e00a11e0 2023-07-06T20:18:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.042+0000 7fb2ea7fc700 1 -- 172.21.15.69:0/3169710857 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fb2f4060a80 con 0x7fb2e00a11e0 2023-07-06T20:18:39.044 INFO:teuthology.orchestra.run.smithi069.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-06T20:18:39.044 INFO:teuthology.orchestra.run.smithi069.stdout:mds.a smithi069 running (79s) 4s ago 78s 13.7M - 18.0.0-4742-g79365bb3 d8383422d536 43fbceeea5e9 2023-07-06T20:18:39.044 INFO:teuthology.orchestra.run.smithi069.stdout:mds.b smithi098 running (77s) 5s ago 76s 29.9M - 18.0.0-4742-g79365bb3 d8383422d536 dca64888f178 2023-07-06T20:18:39.044 INFO:teuthology.orchestra.run.smithi069.stdout:mds.c smithi156 running (90s) 5s ago 89s 18.1M - 18.0.0-4742-g79365bb3 d8383422d536 4c54792087d8 2023-07-06T20:18:39.045 INFO:teuthology.orchestra.run.smithi069.stdout:mds.d smithi069 running (96s) 4s ago 96s 13.8M - 18.0.0-4742-g79365bb3 d8383422d536 83dd9508bcab 2023-07-06T20:18:39.045 INFO:teuthology.orchestra.run.smithi069.stdout:mds.e smithi098 running (94s) 5s ago 94s 19.5M - 18.0.0-4742-g79365bb3 d8383422d536 9ad1d4ad1424 2023-07-06T20:18:39.045 INFO:teuthology.orchestra.run.smithi069.stdout:mds.f smithi156 running (92s) 5s ago 91s 18.0M - 18.0.0-4742-g79365bb3 d8383422d536 147cd4571e9e 2023-07-06T20:18:39.045 INFO:teuthology.orchestra.run.smithi069.stdout:mds.g smithi069 running (82s) 4s ago 82s 13.7M - 18.0.0-4742-g79365bb3 d8383422d536 137d376ad724 2023-07-06T20:18:39.045 INFO:teuthology.orchestra.run.smithi069.stdout:mds.h smithi098 running (87s) 5s ago 87s 20.8M - 18.0.0-4742-g79365bb3 d8383422d536 e3ba5aef4eb5 2023-07-06T20:18:39.045 INFO:teuthology.orchestra.run.smithi069.stdout:mds.i smithi156 running (85s) 5s ago 85s 21.7M - 18.0.0-4742-g79365bb3 d8383422d536 760cf5fd834a 2023-07-06T20:18:39.045 INFO:teuthology.orchestra.run.smithi069.stdout:mds.j smithi069 running (99s) 4s ago 99s 13.7M - 18.0.0-4742-g79365bb3 d8383422d536 e88abd072023 2023-07-06T20:18:39.046 INFO:teuthology.orchestra.run.smithi069.stdout:mds.k smithi098 running (102s) 5s ago 102s 16.5M - 18.0.0-4742-g79365bb3 d8383422d536 9964ef061164 2023-07-06T20:18:39.046 INFO:teuthology.orchestra.run.smithi069.stdout:mds.l smithi156 running (104s) 5s ago 104s 17.4M - 18.0.0-4742-g79365bb3 d8383422d536 9e1555364e4c 2023-07-06T20:18:39.046 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.x smithi069 *:9283,8765 running (8m) 4s ago 8m 521M - 18.0.0-4742-g79365bb3 d8383422d536 568a2d077ffd 2023-07-06T20:18:39.046 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.y smithi098 *:8443,8765 running (6m) 5s ago 6m 448M - 18.0.0-4742-g79365bb3 d8383422d536 f12ac3e248e3 2023-07-06T20:18:39.046 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.z smithi156 *:8443,8765 running (6m) 5s ago 6m 446M - 18.0.0-4742-g79365bb3 d8383422d536 2761d1a14d2b 2023-07-06T20:18:39.046 INFO:teuthology.orchestra.run.smithi069.stdout:mon.a smithi069 running (8m) 4s ago 8m 92.9M 2048M 18.0.0-4742-g79365bb3 d8383422d536 28dcfc1fcd48 2023-07-06T20:18:39.046 INFO:teuthology.orchestra.run.smithi069.stdout:mon.b smithi098 running (6m) 5s ago 6m 61.6M 2048M 18.0.0-4742-g79365bb3 d8383422d536 cd355ba922fb 2023-07-06T20:18:39.046 INFO:teuthology.orchestra.run.smithi069.stdout:mon.c smithi156 running (7m) 5s ago 7m 62.1M 2048M 18.0.0-4742-g79365bb3 d8383422d536 04b7f1be6647 2023-07-06T20:18:39.047 INFO:teuthology.orchestra.run.smithi069.stdout:osd.0 smithi069 running (6m) 4s ago 6m 63.8M 4316M 18.0.0-4742-g79365bb3 d8383422d536 70b530e92573 2023-07-06T20:18:39.047 INFO:teuthology.orchestra.run.smithi069.stdout:osd.1 smithi098 running (5m) 5s ago 5m 71.5M 4292M 18.0.0-4742-g79365bb3 d8383422d536 6b945fac7e21 2023-07-06T20:18:39.047 INFO:teuthology.orchestra.run.smithi069.stdout:osd.2 smithi156 running (5m) 5s ago 5m 67.2M 4292M 18.0.0-4742-g79365bb3 d8383422d536 3614a48a82b3 2023-07-06T20:18:39.047 INFO:teuthology.orchestra.run.smithi069.stdout:osd.3 smithi069 running (5m) 4s ago 5m 61.2M 4316M 18.0.0-4742-g79365bb3 d8383422d536 0ab34039c200 2023-07-06T20:18:39.047 INFO:teuthology.orchestra.run.smithi069.stdout:osd.4 smithi098 running (4m) 5s ago 4m 61.3M 4292M 18.0.0-4742-g79365bb3 d8383422d536 06808bcd2b8b 2023-07-06T20:18:39.047 INFO:teuthology.orchestra.run.smithi069.stdout:osd.5 smithi156 running (4m) 5s ago 4m 63.6M 4292M 18.0.0-4742-g79365bb3 d8383422d536 641c4358f8fe 2023-07-06T20:18:39.047 INFO:teuthology.orchestra.run.smithi069.stdout:osd.6 smithi069 running (3m) 4s ago 3m 58.0M 4316M 18.0.0-4742-g79365bb3 d8383422d536 9c258581f06c 2023-07-06T20:18:39.047 INFO:teuthology.orchestra.run.smithi069.stdout:osd.7 smithi098 running (3m) 5s ago 3m 62.6M 4292M 18.0.0-4742-g79365bb3 d8383422d536 598a55fb39ba 2023-07-06T20:18:39.048 INFO:teuthology.orchestra.run.smithi069.stdout:osd.8 smithi156 running (3m) 5s ago 3m 58.7M 4292M 18.0.0-4742-g79365bb3 d8383422d536 035dc79e142e 2023-07-06T20:18:39.048 INFO:teuthology.orchestra.run.smithi069.stdout:osd.9 smithi069 running (2m) 4s ago 2m 54.7M 4316M 18.0.0-4742-g79365bb3 d8383422d536 2205bedc805a 2023-07-06T20:18:39.048 INFO:teuthology.orchestra.run.smithi069.stdout:osd.10 smithi098 running (2m) 5s ago 2m 60.5M 4292M 18.0.0-4742-g79365bb3 d8383422d536 b1e62d30b7a0 2023-07-06T20:18:39.048 INFO:teuthology.orchestra.run.smithi069.stdout:osd.11 smithi156 running (2m) 5s ago 2m 53.8M 4292M 18.0.0-4742-g79365bb3 d8383422d536 628978ab2f59 2023-07-06T20:18:39.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.044+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/3169710857 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb2e00a11e0 msgr2=0x7fb2e00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:39.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.044+0000 7fb2fb2fd700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb2e00a11e0 0x7fb2e00a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000b2e0 tx=0x7fb2f0005fb0 comp rx=0 tx=0).stop 2023-07-06T20:18:39.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.044+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/3169710857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4100ec0 msgr2=0x7fb2f4195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:39.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.044+0000 7fb2fb2fd700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4100ec0 0x7fb2f4195f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e9d0 tx=0x7fb2e400ece0 comp rx=0 tx=0).stop 2023-07-06T20:18:39.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.044+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/3169710857 shutdown_connections 2023-07-06T20:18:39.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.044+0000 7fb2fb2fd700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb2e00a11e0 0x7fb2e00a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:39.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.044+0000 7fb2fb2fd700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2f4101850 0x7fb2f419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:39.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.044+0000 7fb2fb2fd700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2f40ffcc0 0x7fb2f41959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:39.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.044+0000 7fb2fb2fd700 1 --2- 172.21.15.69:0/3169710857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4100ec0 0x7fb2f4195f30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:39.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.044+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/3169710857 >> 172.21.15.69:0/3169710857 conn(0x7fb2f40fb2d0 msgr2=0x7fb2f4104400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:39.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.044+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/3169710857 shutdown_connections 2023-07-06T20:18:39.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:39.044+0000 7fb2fb2fd700 1 -- 172.21.15.69:0/3169710857 wait complete. 2023-07-06T20:18:39.482 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-06T20:18:39.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:39 smithi069 ceph-mon[126399]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:39.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:39 smithi156 ceph-mon[129280]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:39.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:39 smithi098 ceph-mon[128968]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:40.227 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:40.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:40 smithi069 ceph-mon[126399]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-06T20:18:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:40 smithi156 ceph-mon[129280]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-06T20:18:40.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:40 smithi098 ceph-mon[128968]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-06T20:18:41.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.719+0000 7f012ba45700 1 -- 172.21.15.69:0/1359476814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0124100ec0 msgr2=0x7f0124101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:41.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.719+0000 7f012ba45700 1 --2- 172.21.15.69:0/1359476814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0124100ec0 0x7f0124101310 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0114009990 tx=0x7f0114009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:41.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.719+0000 7f012ba45700 1 -- 172.21.15.69:0/1359476814 shutdown_connections 2023-07-06T20:18:41.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.719+0000 7f012ba45700 1 --2- 172.21.15.69:0/1359476814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0124101850 0x7f0124108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:41.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.719+0000 7f012ba45700 1 --2- 172.21.15.69:0/1359476814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01240ffcc0 0x7f01241000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:41.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.719+0000 7f012ba45700 1 --2- 172.21.15.69:0/1359476814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0124100ec0 0x7f0124101310 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:41.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.719+0000 7f012ba45700 1 -- 172.21.15.69:0/1359476814 >> 172.21.15.69:0/1359476814 conn(0x7f01240fb2d0 msgr2=0x7f01240fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:41.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.719+0000 7f012ba45700 1 -- 172.21.15.69:0/1359476814 shutdown_connections 2023-07-06T20:18:41.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.719+0000 7f012ba45700 1 -- 172.21.15.69:0/1359476814 wait complete. 2023-07-06T20:18:41.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.720+0000 7f012ba45700 1 Processor -- start 2023-07-06T20:18:41.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.720+0000 7f012ba45700 1 -- start start 2023-07-06T20:18:41.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f012ba45700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01240ffcc0 0x7f0124197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:41.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f012ba45700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0124100ec0 0x7f01241980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:41.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f012ba45700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0124101850 0x7f012419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:41.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f012ba45700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012419f730 con 0x7f01240ffcc0 2023-07-06T20:18:41.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f012ba45700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012419f870 con 0x7f0124101850 2023-07-06T20:18:41.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f012ba45700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012419fa00 con 0x7f0124100ec0 2023-07-06T20:18:41.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f01297e1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01240ffcc0 0x7f0124197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:41.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f01297e1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01240ffcc0 0x7f0124197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59066/0 (socket says 172.21.15.69:59066) 2023-07-06T20:18:41.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f01297e1700 1 -- 172.21.15.69:0/2751185646 learned_addr learned my addr 172.21.15.69:0/2751185646 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:41.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f0129fe2700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0124101850 0x7f012419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:41.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f01297e1700 1 -- 172.21.15.69:0/2751185646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0124100ec0 msgr2=0x7f01241980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:41.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f01297e1700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0124100ec0 0x7f01241980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:41.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f01297e1700 1 -- 172.21.15.69:0/2751185646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0124101850 msgr2=0x7f012419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:41.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f01297e1700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0124101850 0x7f012419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:41.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f01297e1700 1 -- 172.21.15.69:0/2751185646 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01140095f0 con 0x7f01240ffcc0 2023-07-06T20:18:41.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f0129fe2700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0124101850 0x7f012419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:41.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.721+0000 7f01297e1700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01240ffcc0 0x7f0124197ba0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f012000d9b0 tx=0x7f012000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:41.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.722+0000 7f011a7fc700 1 -- 172.21.15.69:0/2751185646 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120004020 con 0x7f01240ffcc0 2023-07-06T20:18:41.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.722+0000 7f012ba45700 1 -- 172.21.15.69:0/2751185646 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012419fce0 con 0x7f01240ffcc0 2023-07-06T20:18:41.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.722+0000 7f011a7fc700 1 -- 172.21.15.69:0/2751185646 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0120009730 con 0x7f01240ffcc0 2023-07-06T20:18:41.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.722+0000 7f011a7fc700 1 -- 172.21.15.69:0/2751185646 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120003a40 con 0x7f01240ffcc0 2023-07-06T20:18:41.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.722+0000 7f012ba45700 1 -- 172.21.15.69:0/2751185646 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01241a0180 con 0x7f01240ffcc0 2023-07-06T20:18:41.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.723+0000 7f012ba45700 1 -- 172.21.15.69:0/2751185646 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012410b4f0 con 0x7f01240ffcc0 2023-07-06T20:18:41.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.725+0000 7f011a7fc700 1 -- 172.21.15.69:0/2751185646 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f01200098a0 con 0x7f01240ffcc0 2023-07-06T20:18:41.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.726+0000 7f011a7fc700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f01100a1260 0x7f01100a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.726+0000 7f011a7fc700 1 -- 172.21.15.69:0/2751185646 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f01200cc730 con 0x7f01240ffcc0 2023-07-06T20:18:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.726+0000 7f0128fe0700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f01100a1260 0x7f01100a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.727+0000 7f0128fe0700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f01100a1260 0x7f01100a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f01140095c0 tx=0x7f0114009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:41.732 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.730+0000 7f011a7fc700 1 -- 172.21.15.69:0/2751185646 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0120095cb0 con 0x7f01240ffcc0 2023-07-06T20:18:41.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.934+0000 7f012ba45700 1 -- 172.21.15.69:0/2751185646 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0124107820 con 0x7f01100a1260 2023-07-06T20:18:41.941 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.940+0000 7f011a7fc700 1 -- 172.21.15.69:0/2751185646 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f0124107820 con 0x7f01100a1260 2023-07-06T20:18:41.942 INFO:teuthology.orchestra.run.smithi069.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-06T20:18:41.942 INFO:teuthology.orchestra.run.smithi069.stdout:mds.all 12/12 8s ago 110s smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-07-06T20:18:41.942 INFO:teuthology.orchestra.run.smithi069.stdout:mgr 3/3 8s ago 6m smithi069=x;smithi098=y;smithi156=z;count:3 2023-07-06T20:18:41.942 INFO:teuthology.orchestra.run.smithi069.stdout:mon 3/3 8s ago 7m smithi069:172.21.15.69=a;smithi098:172.21.15.98=b;smithi156:172.21.15.156=c;count:3 2023-07-06T20:18:41.942 INFO:teuthology.orchestra.run.smithi069.stdout:osd 12 8s ago - 2023-07-06T20:18:41.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.942+0000 7f012ba45700 1 -- 172.21.15.69:0/2751185646 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f01100a1260 msgr2=0x7f01100a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:41.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.942+0000 7f012ba45700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f01100a1260 0x7f01100a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f01140095c0 tx=0x7f0114009550 comp rx=0 tx=0).stop 2023-07-06T20:18:41.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.942+0000 7f012ba45700 1 -- 172.21.15.69:0/2751185646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01240ffcc0 msgr2=0x7f0124197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:41.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.942+0000 7f012ba45700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01240ffcc0 0x7f0124197ba0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f012000d9b0 tx=0x7f012000dcc0 comp rx=0 tx=0).stop 2023-07-06T20:18:41.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.942+0000 7f012ba45700 1 -- 172.21.15.69:0/2751185646 shutdown_connections 2023-07-06T20:18:41.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.942+0000 7f012ba45700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f01100a1260 0x7f01100a3710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:41.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.942+0000 7f012ba45700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0124101850 0x7f012419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:41.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.942+0000 7f012ba45700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01240ffcc0 0x7f0124197ba0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:41.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.942+0000 7f012ba45700 1 --2- 172.21.15.69:0/2751185646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0124100ec0 0x7f01241980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:41.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.942+0000 7f012ba45700 1 -- 172.21.15.69:0/2751185646 >> 172.21.15.69:0/2751185646 conn(0x7f01240fb2d0 msgr2=0x7f01240fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:41.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.942+0000 7f012ba45700 1 -- 172.21.15.69:0/2751185646 shutdown_connections 2023-07-06T20:18:41.946 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:41.942+0000 7f012ba45700 1 -- 172.21.15.69:0/2751185646 wait complete. 2023-07-06T20:18:42.397 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-06T20:18:42.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:42 smithi069 ceph-mon[126399]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-06T20:18:42.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:42 smithi069 ceph-mon[126399]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:42.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:42 smithi156 ceph-mon[129280]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-06T20:18:42.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:42 smithi156 ceph-mon[129280]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:42.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:42 smithi098 ceph-mon[128968]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-06T20:18:42.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:42 smithi098 ceph-mon[128968]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:43.120 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:44.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.637+0000 7fb65be76700 1 -- 172.21.15.69:0/3052479516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6540ffcc0 msgr2=0x7fb6541000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:44.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.637+0000 7fb65be76700 1 --2- 172.21.15.69:0/3052479516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6540ffcc0 0x7fb6541000d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb650009990 tx=0x7fb650009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:44.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.637+0000 7fb65be76700 1 -- 172.21.15.69:0/3052479516 shutdown_connections 2023-07-06T20:18:44.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.637+0000 7fb65be76700 1 --2- 172.21.15.69:0/3052479516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb654101850 0x7fb654108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:44.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.637+0000 7fb65be76700 1 --2- 172.21.15.69:0/3052479516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6540ffcc0 0x7fb6541000d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:44.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.637+0000 7fb65be76700 1 --2- 172.21.15.69:0/3052479516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb654100ec0 0x7fb654101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:44.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.637+0000 7fb65be76700 1 -- 172.21.15.69:0/3052479516 >> 172.21.15.69:0/3052479516 conn(0x7fb6540fb2d0 msgr2=0x7fb6540fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:44.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.637+0000 7fb65be76700 1 -- 172.21.15.69:0/3052479516 shutdown_connections 2023-07-06T20:18:44.640 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.637+0000 7fb65be76700 1 -- 172.21.15.69:0/3052479516 wait complete. 2023-07-06T20:18:44.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.638+0000 7fb65be76700 1 Processor -- start 2023-07-06T20:18:44.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.638+0000 7fb65be76700 1 -- start start 2023-07-06T20:18:44.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.638+0000 7fb65be76700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6540ffcc0 0x7fb6541959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:44.642 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb65be76700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654100ec0 0x7fb654195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb65be76700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb654101850 0x7fb65419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb65be76700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65419d580 con 0x7fb654100ec0 2023-07-06T20:18:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb65be76700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65419d6c0 con 0x7fb654101850 2023-07-06T20:18:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb65be76700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65419d850 con 0x7fb6540ffcc0 2023-07-06T20:18:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb659411700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654100ec0 0x7fb654195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb659411700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654100ec0 0x7fb654195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33652/0 (socket says 172.21.15.69:33652) 2023-07-06T20:18:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb659411700 1 -- 172.21.15.69:0/3537522226 learned_addr learned my addr 172.21.15.69:0/3537522226 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb659c12700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6540ffcc0 0x7fb6541959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb65a413700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb654101850 0x7fb65419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb659411700 1 -- 172.21.15.69:0/3537522226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6540ffcc0 msgr2=0x7fb6541959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb659411700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6540ffcc0 0x7fb6541959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb659411700 1 -- 172.21.15.69:0/3537522226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb654101850 msgr2=0x7fb65419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb659411700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb654101850 0x7fb65419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.639+0000 7fb659411700 1 -- 172.21.15.69:0/3537522226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6500095f0 con 0x7fb654100ec0 2023-07-06T20:18:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.640+0000 7fb659411700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654100ec0 0x7fb654195f30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb64400d9b0 tx=0x7fb64400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:44.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.640+0000 7fb64affd700 1 -- 172.21.15.69:0/3537522226 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644004220 con 0x7fb654100ec0 2023-07-06T20:18:44.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.640+0000 7fb65be76700 1 -- 172.21.15.69:0/3537522226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65419db30 con 0x7fb654100ec0 2023-07-06T20:18:44.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.640+0000 7fb64affd700 1 -- 172.21.15.69:0/3537522226 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb644007640 con 0x7fb654100ec0 2023-07-06T20:18:44.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.640+0000 7fb64affd700 1 -- 172.21.15.69:0/3537522226 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644005590 con 0x7fb654100ec0 2023-07-06T20:18:44.645 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.640+0000 7fb65be76700 1 -- 172.21.15.69:0/3537522226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65419e080 con 0x7fb654100ec0 2023-07-06T20:18:44.646 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.641+0000 7fb65be76700 1 -- 172.21.15.69:0/3537522226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6541072c0 con 0x7fb654100ec0 2023-07-06T20:18:44.646 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.643+0000 7fb64affd700 1 -- 172.21.15.69:0/3537522226 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb644004750 con 0x7fb654100ec0 2023-07-06T20:18:44.647 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.644+0000 7fb64affd700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb6400a1260 0x7fb6400a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:44.647 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.644+0000 7fb64affd700 1 -- 172.21.15.69:0/3537522226 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb6440ccd60 con 0x7fb654100ec0 2023-07-06T20:18:44.647 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.645+0000 7fb659c12700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb6400a1260 0x7fb6400a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:44.647 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.645+0000 7fb659c12700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb6400a1260 0x7fb6400a3710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb65000b5c0 tx=0x7fb650005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:44.648 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.646+0000 7fb64affd700 1 -- 172.21.15.69:0/3537522226 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb6440962e0 con 0x7fb654100ec0 2023-07-06T20:18:44.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:44 smithi069 ceph-mon[126399]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:44.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.849+0000 7fb65be76700 1 -- 172.21.15.69:0/3537522226 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb654060a80 con 0x7fb6400a1260 2023-07-06T20:18:44.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.851+0000 7fb64affd700 1 -- 172.21.15.69:0/3537522226 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fb654060a80 con 0x7fb6400a1260 2023-07-06T20:18:44.853 INFO:teuthology.orchestra.run.smithi069.stdout:HOST ADDR LABELS STATUS 2023-07-06T20:18:44.853 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 172.21.15.69 2023-07-06T20:18:44.853 INFO:teuthology.orchestra.run.smithi069.stdout:smithi098 172.21.15.98 2023-07-06T20:18:44.854 INFO:teuthology.orchestra.run.smithi069.stdout:smithi156 172.21.15.156 2023-07-06T20:18:44.854 INFO:teuthology.orchestra.run.smithi069.stdout:3 hosts in cluster 2023-07-06T20:18:44.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.853+0000 7fb65be76700 1 -- 172.21.15.69:0/3537522226 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb6400a1260 msgr2=0x7fb6400a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:44.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.853+0000 7fb65be76700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb6400a1260 0x7fb6400a3710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb65000b5c0 tx=0x7fb650005fd0 comp rx=0 tx=0).stop 2023-07-06T20:18:44.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.853+0000 7fb65be76700 1 -- 172.21.15.69:0/3537522226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654100ec0 msgr2=0x7fb654195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:44.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.853+0000 7fb65be76700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654100ec0 0x7fb654195f30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb64400d9b0 tx=0x7fb64400dcc0 comp rx=0 tx=0).stop 2023-07-06T20:18:44.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.853+0000 7fb65be76700 1 -- 172.21.15.69:0/3537522226 shutdown_connections 2023-07-06T20:18:44.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.853+0000 7fb65be76700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb6400a1260 0x7fb6400a3710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:44.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.853+0000 7fb65be76700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb654101850 0x7fb65419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:44.856 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.853+0000 7fb65be76700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6540ffcc0 0x7fb6541959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:44.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.853+0000 7fb65be76700 1 --2- 172.21.15.69:0/3537522226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654100ec0 0x7fb654195f30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:44.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.853+0000 7fb65be76700 1 -- 172.21.15.69:0/3537522226 >> 172.21.15.69:0/3537522226 conn(0x7fb6540fb2d0 msgr2=0x7fb654104400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:44.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.854+0000 7fb65be76700 1 -- 172.21.15.69:0/3537522226 shutdown_connections 2023-07-06T20:18:44.857 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:44.854+0000 7fb65be76700 1 -- 172.21.15.69:0/3537522226 wait complete. 2023-07-06T20:18:44.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:44 smithi156 ceph-mon[129280]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:44.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:44 smithi098 ceph-mon[128968]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:45.292 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-06T20:18:45.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:45 smithi156 ceph-mon[129280]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:45.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:45 smithi098 ceph-mon[128968]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:46.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:45 smithi069 ceph-mon[126399]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:46.012 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:46.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:46 smithi156 ceph-mon[129280]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-06T20:18:46.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:46 smithi069 ceph-mon[126399]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-06T20:18:46.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:46 smithi098 ceph-mon[128968]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-06T20:18:47.519 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.517+0000 7ff8f9407700 1 -- 172.21.15.69:0/3612361934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8f4101850 msgr2=0x7ff8f4108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:47.519 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.517+0000 7ff8f9407700 1 --2- 172.21.15.69:0/3612361934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8f4101850 0x7ff8f4108110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00b0a0 tx=0x7ff8ec00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:18:47.520 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.517+0000 7ff8f9407700 1 -- 172.21.15.69:0/3612361934 shutdown_connections 2023-07-06T20:18:47.520 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.517+0000 7ff8f9407700 1 --2- 172.21.15.69:0/3612361934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8f4101850 0x7ff8f4108110 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:47.520 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.517+0000 7ff8f9407700 1 --2- 172.21.15.69:0/3612361934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8f40ffcc0 0x7ff8f41000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:47.520 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.517+0000 7ff8f9407700 1 --2- 172.21.15.69:0/3612361934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8f4100ec0 0x7ff8f4101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:47.520 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.517+0000 7ff8f9407700 1 -- 172.21.15.69:0/3612361934 >> 172.21.15.69:0/3612361934 conn(0x7ff8f40fb2d0 msgr2=0x7ff8f40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:47.520 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.517+0000 7ff8f9407700 1 -- 172.21.15.69:0/3612361934 shutdown_connections 2023-07-06T20:18:47.521 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.517+0000 7ff8f9407700 1 -- 172.21.15.69:0/3612361934 wait complete. 2023-07-06T20:18:47.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.518+0000 7ff8f9407700 1 Processor -- start 2023-07-06T20:18:47.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.518+0000 7ff8f9407700 1 -- start start 2023-07-06T20:18:47.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.519+0000 7ff8f9407700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8f40ffcc0 0x7ff8f4195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:47.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.519+0000 7ff8f9407700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8f4100ec0 0x7ff8f4196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:47.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.519+0000 7ff8f9407700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8f4101850 0x7ff8f419d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:47.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.519+0000 7ff8f9407700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f419d680 con 0x7ff8f4101850 2023-07-06T20:18:47.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.519+0000 7ff8f9407700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f419d7c0 con 0x7ff8f4100ec0 2023-07-06T20:18:47.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.519+0000 7ff8f9407700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f419d950 con 0x7ff8f40ffcc0 2023-07-06T20:18:47.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.519+0000 7ff8f37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8f4101850 0x7ff8f419d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:47.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.519+0000 7ff8f37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8f4101850 0x7ff8f419d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33662/0 (socket says 172.21.15.69:33662) 2023-07-06T20:18:47.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.519+0000 7ff8f37fe700 1 -- 172.21.15.69:0/1007363390 learned_addr learned my addr 172.21.15.69:0/1007363390 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.519+0000 7ff8f2ffd700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8f40ffcc0 0x7ff8f4195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.519+0000 7ff8f27fc700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8f4100ec0 0x7ff8f4196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.520+0000 7ff8f37fe700 1 -- 172.21.15.69:0/1007363390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8f40ffcc0 msgr2=0x7ff8f4195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.520+0000 7ff8f37fe700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8f40ffcc0 0x7ff8f4195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.520+0000 7ff8f37fe700 1 -- 172.21.15.69:0/1007363390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8f4100ec0 msgr2=0x7ff8f4196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.520+0000 7ff8f37fe700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8f4100ec0 0x7ff8f4196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.520+0000 7ff8f37fe700 1 -- 172.21.15.69:0/1007363390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec009d00 con 0x7ff8f4101850 2023-07-06T20:18:47.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.520+0000 7ff8f37fe700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8f4101850 0x7ff8f419d0b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00b070 tx=0x7ff8ec00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:47.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.520+0000 7ff8dbfff700 1 -- 172.21.15.69:0/1007363390 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec00e040 con 0x7ff8f4101850 2023-07-06T20:18:47.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.520+0000 7ff8dbfff700 1 -- 172.21.15.69:0/1007363390 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8ec0094a0 con 0x7ff8f4101850 2023-07-06T20:18:47.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.520+0000 7ff8f9407700 1 -- 172.21.15.69:0/1007363390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f419dbd0 con 0x7ff8f4101850 2023-07-06T20:18:47.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.520+0000 7ff8dbfff700 1 -- 172.21.15.69:0/1007363390 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec01b6c0 con 0x7ff8f4101850 2023-07-06T20:18:47.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.520+0000 7ff8f9407700 1 -- 172.21.15.69:0/1007363390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f419dfe0 con 0x7ff8f4101850 2023-07-06T20:18:47.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.521+0000 7ff8f9407700 1 -- 172.21.15.69:0/1007363390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f4107240 con 0x7ff8f4101850 2023-07-06T20:18:47.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.523+0000 7ff8dbfff700 1 -- 172.21.15.69:0/1007363390 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff8ec01b820 con 0x7ff8f4101850 2023-07-06T20:18:47.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.525+0000 7ff8dbfff700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff8dc0a11e0 0x7ff8dc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:47.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.525+0000 7ff8dbfff700 1 -- 172.21.15.69:0/1007363390 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff8ec0ce100 con 0x7ff8f4101850 2023-07-06T20:18:47.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.525+0000 7ff8f2ffd700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff8dc0a11e0 0x7ff8dc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:47.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.525+0000 7ff8f2ffd700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff8dc0a11e0 0x7ff8dc0a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4100d20 tx=0x7ff8e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:47.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.527+0000 7ff8dbfff700 1 -- 172.21.15.69:0/1007363390 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff8ec097770 con 0x7ff8f4101850 2023-07-06T20:18:47.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.733+0000 7ff8f9407700 1 -- 172.21.15.69:0/1007363390 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8f4060a80 con 0x7ff8dc0a11e0 2023-07-06T20:18:47.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.735+0000 7ff8dbfff700 1 -- 172.21.15.69:0/1007363390 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7ff8f4060a80 con 0x7ff8dc0a11e0 2023-07-06T20:18:47.737 INFO:teuthology.orchestra.run.smithi069.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-06T20:18:47.737 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN 372G No 2m ago LVM detected, locked 2023-07-06T20:18:47.737 INFO:teuthology.orchestra.run.smithi069.stdout:smithi098 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010H400BGN 372G No 2m ago LVM detected, locked 2023-07-06T20:18:47.737 INFO:teuthology.orchestra.run.smithi069.stdout:smithi156 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760040480DGN 447G No 2m ago LVM detected, locked 2023-07-06T20:18:47.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.737+0000 7ff8f9407700 1 -- 172.21.15.69:0/1007363390 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff8dc0a11e0 msgr2=0x7ff8dc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:47.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.737+0000 7ff8f9407700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff8dc0a11e0 0x7ff8dc0a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4100d20 tx=0x7ff8e4009260 comp rx=0 tx=0).stop 2023-07-06T20:18:47.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.737+0000 7ff8f9407700 1 -- 172.21.15.69:0/1007363390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8f4101850 msgr2=0x7ff8f419d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:47.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.737+0000 7ff8f9407700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8f4101850 0x7ff8f419d0b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00b070 tx=0x7ff8ec00bce0 comp rx=0 tx=0).stop 2023-07-06T20:18:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.737+0000 7ff8f9407700 1 -- 172.21.15.69:0/1007363390 shutdown_connections 2023-07-06T20:18:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.737+0000 7ff8f9407700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff8dc0a11e0 0x7ff8dc0a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.737+0000 7ff8f9407700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8f4101850 0x7ff8f419d0b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.737+0000 7ff8f9407700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8f40ffcc0 0x7ff8f4195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.737+0000 7ff8f9407700 1 --2- 172.21.15.69:0/1007363390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8f4100ec0 0x7ff8f4196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.737+0000 7ff8f9407700 1 -- 172.21.15.69:0/1007363390 >> 172.21.15.69:0/1007363390 conn(0x7ff8f40fb2d0 msgr2=0x7ff8f4104380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:47.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.737+0000 7ff8f9407700 1 -- 172.21.15.69:0/1007363390 shutdown_connections 2023-07-06T20:18:47.741 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:47.737+0000 7ff8f9407700 1 -- 172.21.15.69:0/1007363390 wait complete. 2023-07-06T20:18:48.184 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-06T20:18:48.194 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2023-07-06T20:18:48.194 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-06T20:18:48.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:48 smithi156 ceph-mon[129280]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:48.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:48 smithi156 ceph-mon[129280]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:48.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:48 smithi098 ceph-mon[128968]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:48.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:48 smithi098 ceph-mon[128968]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:48.917 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:49.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:48 smithi069 ceph-mon[126399]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:49.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:48 smithi069 ceph-mon[126399]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-06T20:18:50.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.422+0000 7f3469b07700 1 -- 172.21.15.69:0/2762983077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3464100ec0 msgr2=0x7f3464101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:50.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.422+0000 7f3469b07700 1 --2- 172.21.15.69:0/2762983077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3464100ec0 0x7f3464101310 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3458009990 tx=0x7f3458009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:50.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.422+0000 7f3469b07700 1 -- 172.21.15.69:0/2762983077 shutdown_connections 2023-07-06T20:18:50.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.422+0000 7f3469b07700 1 --2- 172.21.15.69:0/2762983077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3464101850 0x7f3464108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:50.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.422+0000 7f3469b07700 1 --2- 172.21.15.69:0/2762983077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34640ffcc0 0x7f34641000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:50.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.422+0000 7f3469b07700 1 --2- 172.21.15.69:0/2762983077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3464100ec0 0x7f3464101310 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:50.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.422+0000 7f3469b07700 1 -- 172.21.15.69:0/2762983077 >> 172.21.15.69:0/2762983077 conn(0x7f34640fb2d0 msgr2=0x7f34640fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:50.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.422+0000 7f3469b07700 1 -- 172.21.15.69:0/2762983077 shutdown_connections 2023-07-06T20:18:50.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.422+0000 7f3469b07700 1 -- 172.21.15.69:0/2762983077 wait complete. 2023-07-06T20:18:50.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.423+0000 7f3469b07700 1 Processor -- start 2023-07-06T20:18:50.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.423+0000 7f3469b07700 1 -- start start 2023-07-06T20:18:50.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.423+0000 7f3469b07700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34640ffcc0 0x7f3464197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:50.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.423+0000 7f3469b07700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3464100ec0 0x7f3464198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:50.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.423+0000 7f3469b07700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3464101850 0x7f346419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:50.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.423+0000 7f3469b07700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346419f760 con 0x7f3464100ec0 2023-07-06T20:18:50.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.423+0000 7f3469b07700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346419f8a0 con 0x7f3464101850 2023-07-06T20:18:50.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.423+0000 7f3469b07700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346419fa60 con 0x7f34640ffcc0 2023-07-06T20:18:50.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.424+0000 7f3462ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3464100ec0 0x7f3464198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:50.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.424+0000 7f34637fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34640ffcc0 0x7f3464197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:50.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.424+0000 7f3462ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3464100ec0 0x7f3464198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33694/0 (socket says 172.21.15.69:33694) 2023-07-06T20:18:50.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.424+0000 7f3462ffd700 1 -- 172.21.15.69:0/1775479449 learned_addr learned my addr 172.21.15.69:0/1775479449 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:50.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.424+0000 7f3463fff700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3464101850 0x7f346419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:50.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.424+0000 7f3462ffd700 1 -- 172.21.15.69:0/1775479449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34640ffcc0 msgr2=0x7f3464197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:50.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.424+0000 7f3462ffd700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34640ffcc0 0x7f3464197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:50.429 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.424+0000 7f3462ffd700 1 -- 172.21.15.69:0/1775479449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3464101850 msgr2=0x7f346419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:50.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.424+0000 7f3462ffd700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3464101850 0x7f346419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:50.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.424+0000 7f3462ffd700 1 -- 172.21.15.69:0/1775479449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34580095f0 con 0x7f3464100ec0 2023-07-06T20:18:50.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.424+0000 7f3462ffd700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3464100ec0 0x7f3464198150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3458004730 tx=0x7f3458004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:50.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.425+0000 7f3460ff9700 1 -- 172.21.15.69:0/1775479449 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345801d070 con 0x7f3464100ec0 2023-07-06T20:18:50.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.425+0000 7f3460ff9700 1 -- 172.21.15.69:0/1775479449 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3458004c90 con 0x7f3464100ec0 2023-07-06T20:18:50.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.425+0000 7f3469b07700 1 -- 172.21.15.69:0/1775479449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346419fce0 con 0x7f3464100ec0 2023-07-06T20:18:50.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.425+0000 7f3460ff9700 1 -- 172.21.15.69:0/1775479449 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34580054b0 con 0x7f3464100ec0 2023-07-06T20:18:50.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.425+0000 7f3469b07700 1 -- 172.21.15.69:0/1775479449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34641a0090 con 0x7f3464100ec0 2023-07-06T20:18:50.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.426+0000 7f3469b07700 1 -- 172.21.15.69:0/1775479449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346410b540 con 0x7f3464100ec0 2023-07-06T20:18:50.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.428+0000 7f3460ff9700 1 -- 172.21.15.69:0/1775479449 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f345800fbc0 con 0x7f3464100ec0 2023-07-06T20:18:50.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.429+0000 7f3460ff9700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f344c0a11e0 0x7f344c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:50.432 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.429+0000 7f3460ff9700 1 -- 172.21.15.69:0/1775479449 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f34580cdd90 con 0x7f3464100ec0 2023-07-06T20:18:50.432 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.429+0000 7f34637fe700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f344c0a11e0 0x7f344c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:50.432 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.430+0000 7f34637fe700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f344c0a11e0 0x7f344c0a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f34641990f0 tx=0x7f345400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:50.435 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.433+0000 7f3460ff9700 1 -- 172.21.15.69:0/1775479449 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f345805a770 con 0x7f3464100ec0 2023-07-06T20:18:50.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.670+0000 7f3469b07700 1 -- 172.21.15.69:0/1775479449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f34641a06a0 con 0x7f3464100ec0 2023-07-06T20:18:50.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.671+0000 7f3460ff9700 1 -- 172.21.15.69:0/1775479449 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2891 (secure 0 0 0) 0x7f34580973a0 con 0x7f3464100ec0 2023-07-06T20:18:50.673 INFO:teuthology.orchestra.run.smithi069.stdout:e23 2023-07-06T20:18:50.673 INFO:teuthology.orchestra.run.smithi069.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-06T20:18:50.673 INFO:teuthology.orchestra.run.smithi069.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-06T20:18:50.674 INFO:teuthology.orchestra.run.smithi069.stdout:legacy client fscid: 1 2023-07-06T20:18:50.674 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:50.674 INFO:teuthology.orchestra.run.smithi069.stdout:Filesystem 'cephfs' (1) 2023-07-06T20:18:50.674 INFO:teuthology.orchestra.run.smithi069.stdout:fs_name cephfs 2023-07-06T20:18:50.674 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 23 2023-07-06T20:18:50.674 INFO:teuthology.orchestra.run.smithi069.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-06T20:18:50.674 INFO:teuthology.orchestra.run.smithi069.stdout:created 2023-07-06T20:17:28.144582+0000 2023-07-06T20:18:50.675 INFO:teuthology.orchestra.run.smithi069.stdout:modified 2023-07-06T20:17:33.557511+0000 2023-07-06T20:18:50.675 INFO:teuthology.orchestra.run.smithi069.stdout:tableserver 0 2023-07-06T20:18:50.675 INFO:teuthology.orchestra.run.smithi069.stdout:root 0 2023-07-06T20:18:50.675 INFO:teuthology.orchestra.run.smithi069.stdout:session_timeout 300 2023-07-06T20:18:50.675 INFO:teuthology.orchestra.run.smithi069.stdout:session_autoclose 300 2023-07-06T20:18:50.675 INFO:teuthology.orchestra.run.smithi069.stdout:max_file_size 1099511627776 2023-07-06T20:18:50.675 INFO:teuthology.orchestra.run.smithi069.stdout:max_xattr_size 65536 2023-07-06T20:18:50.675 INFO:teuthology.orchestra.run.smithi069.stdout:required_client_features {} 2023-07-06T20:18:50.676 INFO:teuthology.orchestra.run.smithi069.stdout:last_failure 0 2023-07-06T20:18:50.676 INFO:teuthology.orchestra.run.smithi069.stdout:last_failure_osd_epoch 0 2023-07-06T20:18:50.676 INFO:teuthology.orchestra.run.smithi069.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-06T20:18:50.676 INFO:teuthology.orchestra.run.smithi069.stdout:max_mds 3 2023-07-06T20:18:50.676 INFO:teuthology.orchestra.run.smithi069.stdout:in 0,1,2 2023-07-06T20:18:50.676 INFO:teuthology.orchestra.run.smithi069.stdout:up {0=24584,1=24539,2=24512} 2023-07-06T20:18:50.676 INFO:teuthology.orchestra.run.smithi069.stdout:failed 2023-07-06T20:18:50.677 INFO:teuthology.orchestra.run.smithi069.stdout:damaged 2023-07-06T20:18:50.677 INFO:teuthology.orchestra.run.smithi069.stdout:stopped 2023-07-06T20:18:50.677 INFO:teuthology.orchestra.run.smithi069.stdout:data_pools [3] 2023-07-06T20:18:50.677 INFO:teuthology.orchestra.run.smithi069.stdout:metadata_pool 2 2023-07-06T20:18:50.677 INFO:teuthology.orchestra.run.smithi069.stdout:inline_data disabled 2023-07-06T20:18:50.677 INFO:teuthology.orchestra.run.smithi069.stdout:balancer 2023-07-06T20:18:50.677 INFO:teuthology.orchestra.run.smithi069.stdout:bal_rank_mask -1 2023-07-06T20:18:50.677 INFO:teuthology.orchestra.run.smithi069.stdout:standby_count_wanted 0 2023-07-06T20:18:50.678 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.b{0:24584} state up:active seq 3 addr [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] compat {c=[1],r=[1],i=[7ff]}] 2023-07-06T20:18:50.678 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.i{0:24535} state up:standby-replay seq 1 addr [v2:172.21.15.156:6838/4200610083,v1:172.21.15.156:6839/4200610083] compat {c=[1],r=[1],i=[7ff]}] 2023-07-06T20:18:50.678 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.h{1:24539} state up:active seq 6 addr [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] compat {c=[1],r=[1],i=[7ff]}] 2023-07-06T20:18:50.678 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.c{1:24517} state up:standby-replay seq 1 addr [v2:172.21.15.156:6836/595692705,v1:172.21.15.156:6837/595692705] compat {c=[1],r=[1],i=[7ff]}] 2023-07-06T20:18:50.678 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.e{2:24512} state up:active seq 8 addr [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] compat {c=[1],r=[1],i=[7ff]}] 2023-07-06T20:18:50.678 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.f{2:24505} state up:standby-replay seq 1 addr [v2:172.21.15.156:6834/2525319359,v1:172.21.15.156:6835/2525319359] compat {c=[1],r=[1],i=[7ff]}] 2023-07-06T20:18:50.678 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:50.678 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:50.679 INFO:teuthology.orchestra.run.smithi069.stdout:Standby daemons: 2023-07-06T20:18:50.679 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:50.679 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.j{-1:14448} state up:standby seq 1 addr [v2:172.21.15.69:6834/3728004874,v1:172.21.15.69:6835/3728004874] compat {c=[1],r=[1],i=[7ff]}] 2023-07-06T20:18:50.679 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.d{-1:14460} state up:standby seq 1 addr [v2:172.21.15.69:6836/896595344,v1:172.21.15.69:6837/896595344] compat {c=[1],r=[1],i=[7ff]}] 2023-07-06T20:18:50.679 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.g{-1:14517} state up:standby seq 1 addr [v2:172.21.15.69:6838/1973475728,v1:172.21.15.69:6839/1973475728] compat {c=[1],r=[1],i=[7ff]}] 2023-07-06T20:18:50.679 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.a{-1:14529} state up:standby seq 1 addr [v2:172.21.15.69:6840/1464129965,v1:172.21.15.69:6841/1464129965] compat {c=[1],r=[1],i=[7ff]}] 2023-07-06T20:18:50.679 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.l{-1:24457} state up:standby seq 1 addr [v2:172.21.15.156:6832/1091328636,v1:172.21.15.156:6833/1091328636] compat {c=[1],r=[1],i=[7ff]}] 2023-07-06T20:18:50.679 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.k{-1:24476} state up:standby seq 1 addr [v2:172.21.15.98:6832/3115382908,v1:172.21.15.98:6833/3115382908] compat {c=[1],r=[1],i=[7ff]}] 2023-07-06T20:18:50.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.673+0000 7f3469b07700 1 -- 172.21.15.69:0/1775479449 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f344c0a11e0 msgr2=0x7f344c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:50.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.673+0000 7f3469b07700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f344c0a11e0 0x7f344c0a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f34641990f0 tx=0x7f345400b410 comp rx=0 tx=0).stop 2023-07-06T20:18:50.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.673+0000 7f3469b07700 1 -- 172.21.15.69:0/1775479449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3464100ec0 msgr2=0x7f3464198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:50.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.673+0000 7f3469b07700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3464100ec0 0x7f3464198150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3458004730 tx=0x7f3458004760 comp rx=0 tx=0).stop 2023-07-06T20:18:50.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.673+0000 7f3469b07700 1 -- 172.21.15.69:0/1775479449 shutdown_connections 2023-07-06T20:18:50.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.673+0000 7f3469b07700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f344c0a11e0 0x7f344c0a3690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:50.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.673+0000 7f3469b07700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3464101850 0x7f346419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:50.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.673+0000 7f3469b07700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34640ffcc0 0x7f3464197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:50.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.673+0000 7f3469b07700 1 --2- 172.21.15.69:0/1775479449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3464100ec0 0x7f3464198150 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:50.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.673+0000 7f3469b07700 1 -- 172.21.15.69:0/1775479449 >> 172.21.15.69:0/1775479449 conn(0x7f34640fb2d0 msgr2=0x7f34640fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:50.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.673+0000 7f3469b07700 1 -- 172.21.15.69:0/1775479449 shutdown_connections 2023-07-06T20:18:50.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:50.673+0000 7f3469b07700 1 -- 172.21.15.69:0/1775479449 wait complete. 2023-07-06T20:18:50.682 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2023-07-06T20:18:50.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:50 smithi156 ceph-mon[129280]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-06T20:18:50.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:50 smithi098 ceph-mon[128968]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-06T20:18:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:50 smithi069 ceph-mon[126399]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-06T20:18:51.089 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-06T20:18:51.808 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:18:51.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:51 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1775479449' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-06T20:18:52.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:51 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1775479449' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-06T20:18:52.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:51 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1775479449' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-06T20:18:52.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:52 smithi098 ceph-mon[128968]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-06T20:18:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:52 smithi069 ceph-mon[126399]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-06T20:18:53.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:52 smithi156 ceph-mon[129280]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-06T20:18:53.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.281+0000 7fdd402bc700 1 -- 172.21.15.69:0/2595840115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd380730a0 msgr2=0x7fdd38073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:53.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.281+0000 7fdd402bc700 1 --2- 172.21.15.69:0/2595840115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd380730a0 0x7fdd38073510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009990 tx=0x7fdd28009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:53.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.282+0000 7fdd402bc700 1 -- 172.21.15.69:0/2595840115 shutdown_connections 2023-07-06T20:18:53.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.282+0000 7fdd402bc700 1 --2- 172.21.15.69:0/2595840115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd38107b90 0x7fdd38109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:53.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.282+0000 7fdd402bc700 1 --2- 172.21.15.69:0/2595840115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd38074670 0x7fdd38072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:53.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.282+0000 7fdd402bc700 1 --2- 172.21.15.69:0/2595840115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd380730a0 0x7fdd38073510 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:53.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.282+0000 7fdd402bc700 1 -- 172.21.15.69:0/2595840115 >> 172.21.15.69:0/2595840115 conn(0x7fdd380fb2d0 msgr2=0x7fdd380fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:53.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.282+0000 7fdd402bc700 1 -- 172.21.15.69:0/2595840115 shutdown_connections 2023-07-06T20:18:53.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.282+0000 7fdd402bc700 1 -- 172.21.15.69:0/2595840115 wait complete. 2023-07-06T20:18:53.288 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.283+0000 7fdd402bc700 1 Processor -- start 2023-07-06T20:18:53.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.283+0000 7fdd402bc700 1 -- start start 2023-07-06T20:18:53.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.283+0000 7fdd402bc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd380730a0 0x7fdd38197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:53.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.283+0000 7fdd402bc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd38074670 0x7fdd38198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:53.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.283+0000 7fdd402bc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd38107b90 0x7fdd3819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:53.290 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.283+0000 7fdd402bc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3819f790 con 0x7fdd380730a0 2023-07-06T20:18:53.290 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.283+0000 7fdd402bc700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3819f8d0 con 0x7fdd38107b90 2023-07-06T20:18:53.290 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.283+0000 7fdd402bc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3819fa90 con 0x7fdd38074670 2023-07-06T20:18:53.290 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.283+0000 7fdd3e058700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd380730a0 0x7fdd38197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:53.291 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd3e058700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd380730a0 0x7fdd38197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56834/0 (socket says 172.21.15.69:56834) 2023-07-06T20:18:53.291 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd3e058700 1 -- 172.21.15.69:0/942628965 learned_addr learned my addr 172.21.15.69:0/942628965 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:53.291 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd3d857700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd38074670 0x7fdd38198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:53.292 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd3e859700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd38107b90 0x7fdd3819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:53.292 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd3e058700 1 -- 172.21.15.69:0/942628965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd38074670 msgr2=0x7fdd38198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:53.292 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd3e058700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd38074670 0x7fdd38198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:53.292 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd3e058700 1 -- 172.21.15.69:0/942628965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd38107b90 msgr2=0x7fdd3819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:53.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd3e058700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd38107b90 0x7fdd3819f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:53.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd3e058700 1 -- 172.21.15.69:0/942628965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd30009c40 con 0x7fdd380730a0 2023-07-06T20:18:53.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd3e859700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd38107b90 0x7fdd3819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:53.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd3e058700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd380730a0 0x7fdd38197c40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400e9a0 tx=0x7fdd3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:53.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd2f7fe700 1 -- 172.21.15.69:0/942628965 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34009790 con 0x7fdd380730a0 2023-07-06T20:18:53.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd2f7fe700 1 -- 172.21.15.69:0/942628965 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd3400ce80 con 0x7fdd380730a0 2023-07-06T20:18:53.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.284+0000 7fdd402bc700 1 -- 172.21.15.69:0/942628965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd280095f0 con 0x7fdd380730a0 2023-07-06T20:18:53.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.285+0000 7fdd2f7fe700 1 -- 172.21.15.69:0/942628965 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34018680 con 0x7fdd380730a0 2023-07-06T20:18:53.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.285+0000 7fdd402bc700 1 -- 172.21.15.69:0/942628965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd381a0130 con 0x7fdd380730a0 2023-07-06T20:18:53.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.286+0000 7fdd402bc700 1 -- 172.21.15.69:0/942628965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd38106240 con 0x7fdd380730a0 2023-07-06T20:18:53.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.288+0000 7fdd2f7fe700 1 -- 172.21.15.69:0/942628965 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdd34010430 con 0x7fdd380730a0 2023-07-06T20:18:53.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.289+0000 7fdd2f7fe700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fdd240a1260 0x7fdd240a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:53.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.289+0000 7fdd2f7fe700 1 -- 172.21.15.69:0/942628965 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fdd34014070 con 0x7fdd380730a0 2023-07-06T20:18:53.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.289+0000 7fdd3d857700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fdd240a1260 0x7fdd240a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:53.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.290+0000 7fdd3d857700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fdd240a1260 0x7fdd240a3710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800b2e0 tx=0x7fdd28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:53.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.293+0000 7fdd2f7fe700 1 -- 172.21.15.69:0/942628965 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdd34051770 con 0x7fdd380730a0 2023-07-06T20:18:53.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.488+0000 7fdd402bc700 1 -- 172.21.15.69:0/942628965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fdd3802cb50 con 0x7fdd380730a0 2023-07-06T20:18:53.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.489+0000 7fdd2f7fe700 1 -- 172.21.15.69:0/942628965 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) v1 ==== 56+0+5291 (secure 0 0 0) 0x7fdd340961d0 con 0x7fdd380730a0 2023-07-06T20:18:53.491 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 85 2023-07-06T20:18:53.491 INFO:teuthology.orchestra.run.smithi069.stdout:fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c 2023-07-06T20:18:53.492 INFO:teuthology.orchestra.run.smithi069.stdout:created 2023-07-06T20:09:56.506880+0000 2023-07-06T20:18:53.492 INFO:teuthology.orchestra.run.smithi069.stdout:modified 2023-07-06T20:17:58.736720+0000 2023-07-06T20:18:53.492 INFO:teuthology.orchestra.run.smithi069.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-06T20:18:53.492 INFO:teuthology.orchestra.run.smithi069.stdout:crush_version 26 2023-07-06T20:18:53.492 INFO:teuthology.orchestra.run.smithi069.stdout:full_ratio 0.95 2023-07-06T20:18:53.492 INFO:teuthology.orchestra.run.smithi069.stdout:backfillfull_ratio 0.9 2023-07-06T20:18:53.492 INFO:teuthology.orchestra.run.smithi069.stdout:nearfull_ratio 0.85 2023-07-06T20:18:53.493 INFO:teuthology.orchestra.run.smithi069.stdout:require_min_compat_client luminous 2023-07-06T20:18:53.493 INFO:teuthology.orchestra.run.smithi069.stdout:min_compat_client luminous 2023-07-06T20:18:53.493 INFO:teuthology.orchestra.run.smithi069.stdout:require_osd_release reef 2023-07-06T20:18:53.493 INFO:teuthology.orchestra.run.smithi069.stdout:stretch_mode_enabled false 2023-07-06T20:18:53.493 INFO:teuthology.orchestra.run.smithi069.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-06T20:18:53.493 INFO:teuthology.orchestra.run.smithi069.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-07-06T20:18:53.493 INFO:teuthology.orchestra.run.smithi069.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-07-06T20:18:53.494 INFO:teuthology.orchestra.run.smithi069.stdout:max_osd 12 2023-07-06T20:18:53.494 INFO:teuthology.orchestra.run.smithi069.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6802/2593880786,v1:172.21.15.69:6803/2593880786] [v2:172.21.15.69:6804/2593880786,v1:172.21.15.69:6805/2593880786] exists,up 02b7a479-153d-48fb-bac6-606262f65f1a 2023-07-06T20:18:53.494 INFO:teuthology.orchestra.run.smithi069.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6800/3741678125,v1:172.21.15.98:6801/3741678125] [v2:172.21.15.98:6802/3741678125,v1:172.21.15.98:6803/3741678125] exists,up 85f52335-a4a7-42f5-8fbc-6414205fe241 2023-07-06T20:18:53.494 INFO:teuthology.orchestra.run.smithi069.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6800/1470225946,v1:172.21.15.156:6801/1470225946] [v2:172.21.15.156:6802/1470225946,v1:172.21.15.156:6803/1470225946] exists,up dea6f798-761a-460a-a87b-f7b2bc6e28a6 2023-07-06T20:18:53.494 INFO:teuthology.orchestra.run.smithi069.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6810/3049408200,v1:172.21.15.69:6811/3049408200] [v2:172.21.15.69:6812/3049408200,v1:172.21.15.69:6813/3049408200] exists,up 8cca1e09-50d7-4a90-bc9e-a62e092c5d55 2023-07-06T20:18:53.494 INFO:teuthology.orchestra.run.smithi069.stdout:osd.4 up in weight 1 up_from 33 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6808/3859428162,v1:172.21.15.98:6809/3859428162] [v2:172.21.15.98:6810/3859428162,v1:172.21.15.98:6811/3859428162] exists,up 66a50ae1-cca5-474e-a2f0-ace23bbc0ed8 2023-07-06T20:18:53.494 INFO:teuthology.orchestra.run.smithi069.stdout:osd.5 up in weight 1 up_from 38 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6808/428236634,v1:172.21.15.156:6809/428236634] [v2:172.21.15.156:6810/428236634,v1:172.21.15.156:6811/428236634] exists,up f62add7f-8f7d-46a4-b28b-e26110346d85 2023-07-06T20:18:53.494 INFO:teuthology.orchestra.run.smithi069.stdout:osd.6 up in weight 1 up_from 46 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6818/2001926544,v1:172.21.15.69:6819/2001926544] [v2:172.21.15.69:6820/2001926544,v1:172.21.15.69:6821/2001926544] exists,up 3fdcdab4-b7d2-4e02-908a-f7ce47c8c735 2023-07-06T20:18:53.495 INFO:teuthology.orchestra.run.smithi069.stdout:osd.7 up in weight 1 up_from 51 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6816/238502140,v1:172.21.15.98:6817/238502140] [v2:172.21.15.98:6818/238502140,v1:172.21.15.98:6819/238502140] exists,up ce45b288-d0c7-4ce3-9b99-0b48a168cd74 2023-07-06T20:18:53.495 INFO:teuthology.orchestra.run.smithi069.stdout:osd.8 up in weight 1 up_from 58 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6816/3522956813,v1:172.21.15.156:6817/3522956813] [v2:172.21.15.156:6818/3522956813,v1:172.21.15.156:6819/3522956813] exists,up 8288c36a-d97e-4279-95ac-a215817b4ff7 2023-07-06T20:18:53.495 INFO:teuthology.orchestra.run.smithi069.stdout:osd.9 up in weight 1 up_from 66 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6826/2424610097,v1:172.21.15.69:6827/2424610097] [v2:172.21.15.69:6828/2424610097,v1:172.21.15.69:6829/2424610097] exists,up 3fc7fc17-09f7-4d5c-b267-46605c603c75 2023-07-06T20:18:53.495 INFO:teuthology.orchestra.run.smithi069.stdout:osd.10 up in weight 1 up_from 71 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6824/2499526049,v1:172.21.15.98:6825/2499526049] [v2:172.21.15.98:6826/2499526049,v1:172.21.15.98:6827/2499526049] exists,up e2f95a02-3a23-48bb-8b7e-3ecc462b3192 2023-07-06T20:18:53.495 INFO:teuthology.orchestra.run.smithi069.stdout:osd.11 up in weight 1 up_from 78 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6824/266940539,v1:172.21.15.156:6825/266940539] [v2:172.21.15.156:6826/266940539,v1:172.21.15.156:6827/266940539] exists,up 5d0deb6c-92a0-4ae9-897e-51ddfc190bf4 2023-07-06T20:18:53.495 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.10 [8,11,2,1] 2023-07-06T20:18:53.495 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.16 [2,4] 2023-07-06T20:18:53.496 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.1a [8,4] 2023-07-06T20:18:53.496 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.e [4,7] 2023-07-06T20:18:53.496 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.21 [4,8] 2023-07-06T20:18:53.496 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.25 [5,9] 2023-07-06T20:18:53.496 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.2d [3,9] 2023-07-06T20:18:53.496 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.31 [4,6] 2023-07-06T20:18:53.496 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.33 [5,9] 2023-07-06T20:18:53.497 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/1409752664 expires 2023-07-07T20:10:57.432754+0000 2023-07-06T20:18:53.497 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6801/9699389 expires 2023-07-07T20:10:57.432754+0000 2023-07-06T20:18:53.497 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6800/9699389 expires 2023-07-07T20:10:57.432754+0000 2023-07-06T20:18:53.497 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/20539643 expires 2023-07-07T20:10:57.432754+0000 2023-07-06T20:18:53.497 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/3832776393 expires 2023-07-07T20:10:31.404980+0000 2023-07-06T20:18:53.497 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/1479828612 expires 2023-07-07T20:10:31.404980+0000 2023-07-06T20:18:53.497 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6800/1829009377 expires 2023-07-07T20:10:31.404980+0000 2023-07-06T20:18:53.497 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6801/1829009377 expires 2023-07-07T20:10:31.404980+0000 2023-07-06T20:18:53.498 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/4048964594 expires 2023-07-07T20:10:31.404980+0000 2023-07-06T20:18:53.498 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/2854344202 expires 2023-07-07T20:10:57.432754+0000 2023-07-06T20:18:53.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.491+0000 7fdd402bc700 1 -- 172.21.15.69:0/942628965 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fdd240a1260 msgr2=0x7fdd240a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.491+0000 7fdd402bc700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fdd240a1260 0x7fdd240a3710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800b2e0 tx=0x7fdd28005fb0 comp rx=0 tx=0).stop 2023-07-06T20:18:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.491+0000 7fdd402bc700 1 -- 172.21.15.69:0/942628965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd380730a0 msgr2=0x7fdd38197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.491+0000 7fdd402bc700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd380730a0 0x7fdd38197c40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400e9a0 tx=0x7fdd3400ecb0 comp rx=0 tx=0).stop 2023-07-06T20:18:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.491+0000 7fdd402bc700 1 -- 172.21.15.69:0/942628965 shutdown_connections 2023-07-06T20:18:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.491+0000 7fdd402bc700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fdd240a1260 0x7fdd240a3710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.491+0000 7fdd402bc700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd38107b90 0x7fdd3819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.491+0000 7fdd402bc700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd380730a0 0x7fdd38197c40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:53.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.491+0000 7fdd402bc700 1 --2- 172.21.15.69:0/942628965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd38074670 0x7fdd38198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:53.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.491+0000 7fdd402bc700 1 -- 172.21.15.69:0/942628965 >> 172.21.15.69:0/942628965 conn(0x7fdd380fb2d0 msgr2=0x7fdd380fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:53.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.492+0000 7fdd402bc700 1 -- 172.21.15.69:0/942628965 shutdown_connections 2023-07-06T20:18:53.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:53.492+0000 7fdd402bc700 1 -- 172.21.15.69:0/942628965 wait complete. 2023-07-06T20:18:53.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:53 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/942628965' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-06T20:18:53.883 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:53 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/942628965' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-06T20:18:53.979 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-06T20:18:53.992 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:18:54.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:53 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/942628965' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-06T20:18:54.191 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.189+0000 7f96793ff700 1 -- 172.21.15.69:0/466016276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96741017b0 msgr2=0x7f9674108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:54.191 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.189+0000 7f96793ff700 1 --2- 172.21.15.69:0/466016276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96741017b0 0x7f9674108070 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f966c00b0a0 tx=0x7f966c00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.191 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.189+0000 7f96793ff700 1 -- 172.21.15.69:0/466016276 shutdown_connections 2023-07-06T20:18:54.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.189+0000 7f96793ff700 1 --2- 172.21.15.69:0/466016276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96741017b0 0x7f9674108070 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.189+0000 7f96793ff700 1 --2- 172.21.15.69:0/466016276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96740ffc30 0x7f9674100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.189+0000 7f96793ff700 1 --2- 172.21.15.69:0/466016276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9674100e20 0x7f9674101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.189+0000 7f96793ff700 1 -- 172.21.15.69:0/466016276 >> 172.21.15.69:0/466016276 conn(0x7f96740fb210 msgr2=0x7f96740fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:54.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.189+0000 7f96793ff700 1 -- 172.21.15.69:0/466016276 shutdown_connections 2023-07-06T20:18:54.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.189+0000 7f96793ff700 1 -- 172.21.15.69:0/466016276 wait complete. 2023-07-06T20:18:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.190+0000 7f96793ff700 1 Processor -- start 2023-07-06T20:18:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.190+0000 7f96793ff700 1 -- start start 2023-07-06T20:18:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f96793ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96740ffc30 0x7f96740743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f96793ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9674100e20 0x7f9674072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f96793ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96741017b0 0x7f9674072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f96793ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9674073710 con 0x7f96740ffc30 2023-07-06T20:18:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f96793ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9674073880 con 0x7f9674100e20 2023-07-06T20:18:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f96793ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967419fa90 con 0x7f96741017b0 2023-07-06T20:18:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f9672ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96740ffc30 0x7f96740743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f9672ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96740ffc30 0x7f96740743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56848/0 (socket says 172.21.15.69:56848) 2023-07-06T20:18:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f9672ffd700 1 -- 172.21.15.69:0/3477858806 learned_addr learned my addr 172.21.15.69:0/3477858806 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f96727fc700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9674100e20 0x7f9674072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f96737fe700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96741017b0 0x7f9674072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f9672ffd700 1 -- 172.21.15.69:0/3477858806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96741017b0 msgr2=0x7f9674072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f9672ffd700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96741017b0 0x7f9674072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f9672ffd700 1 -- 172.21.15.69:0/3477858806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9674100e20 msgr2=0x7f9674072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f9672ffd700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9674100e20 0x7f9674072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f9672ffd700 1 -- 172.21.15.69:0/3477858806 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966c009d00 con 0x7f96740ffc30 2023-07-06T20:18:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f96737fe700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96741017b0 0x7f9674072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.191+0000 7f9672ffd700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96740ffc30 0x7f96740743f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f966400e970 tx=0x7f966400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.192+0000 7f965bfff700 1 -- 172.21.15.69:0/3477858806 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664009790 con 0x7f96740ffc30 2023-07-06T20:18:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.192+0000 7f96793ff700 1 -- 172.21.15.69:0/3477858806 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967419fd70 con 0x7f96740ffc30 2023-07-06T20:18:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.192+0000 7f965bfff700 1 -- 172.21.15.69:0/3477858806 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f966400eec0 con 0x7f96740ffc30 2023-07-06T20:18:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.192+0000 7f965bfff700 1 -- 172.21.15.69:0/3477858806 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664003cf0 con 0x7f96740ffc30 2023-07-06T20:18:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.192+0000 7f96793ff700 1 -- 172.21.15.69:0/3477858806 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96741a0210 con 0x7f96740ffc30 2023-07-06T20:18:54.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.193+0000 7f96793ff700 1 -- 172.21.15.69:0/3477858806 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967410b460 con 0x7f96740ffc30 2023-07-06T20:18:54.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.195+0000 7f965bfff700 1 -- 172.21.15.69:0/3477858806 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f966400ccd0 con 0x7f96740ffc30 2023-07-06T20:18:54.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.196+0000 7f965bfff700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f965c0a1220 0x7f965c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:54.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.196+0000 7f965bfff700 1 -- 172.21.15.69:0/3477858806 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9664014070 con 0x7f96740ffc30 2023-07-06T20:18:54.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.196+0000 7f96727fc700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f965c0a1220 0x7f965c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:54.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.197+0000 7f96727fc700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f965c0a1220 0x7f965c0a36d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9668009c60 tx=0x7f9668009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.200+0000 7f965bfff700 1 -- 172.21.15.69:0/3477858806 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f96640962d0 con 0x7f96740ffc30 2023-07-06T20:18:54.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.399+0000 7f96793ff700 1 -- 172.21.15.69:0/3477858806 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f967410b650 con 0x7f96740ffc30 2023-07-06T20:18:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.401+0000 7f965bfff700 1 -- 172.21.15.69:0/3477858806 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22564 (secure 0 0 0) 0x7f96640998f0 con 0x7f96740ffc30 2023-07-06T20:18:54.403 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:54.403 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":85,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:17:58.736720+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:18:54.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.403+0000 7f96793ff700 1 -- 172.21.15.69:0/3477858806 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f965c0a1220 msgr2=0x7f965c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:54.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.403+0000 7f96793ff700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f965c0a1220 0x7f965c0a36d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9668009c60 tx=0x7f9668009260 comp rx=0 tx=0).stop 2023-07-06T20:18:54.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.403+0000 7f96793ff700 1 -- 172.21.15.69:0/3477858806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96740ffc30 msgr2=0x7f96740743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:54.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.403+0000 7f96793ff700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96740ffc30 0x7f96740743f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f966400e970 tx=0x7f966400ec80 comp rx=0 tx=0).stop 2023-07-06T20:18:54.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.403+0000 7f96793ff700 1 -- 172.21.15.69:0/3477858806 shutdown_connections 2023-07-06T20:18:54.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.403+0000 7f96793ff700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f965c0a1220 0x7f965c0a36d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.403+0000 7f96793ff700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96741017b0 0x7f9674072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.403+0000 7f96793ff700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96740ffc30 0x7f96740743f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.403+0000 7f96793ff700 1 --2- 172.21.15.69:0/3477858806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9674100e20 0x7f9674072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.403+0000 7f96793ff700 1 -- 172.21.15.69:0/3477858806 >> 172.21.15.69:0/3477858806 conn(0x7f96740fb210 msgr2=0x7f96740fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:54.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.403+0000 7f96793ff700 1 -- 172.21.15.69:0/3477858806 shutdown_connections 2023-07-06T20:18:54.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.403+0000 7f96793ff700 1 -- 172.21.15.69:0/3477858806 wait complete. 2023-07-06T20:18:54.424 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:14.022799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:17:26.523183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:17:27.543386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:18:54.425 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:18:54.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.621+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4194937386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9db80ffc30 msgr2=0x7f9db8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:54.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.621+0000 7f9dbe7ff700 1 --2- 172.21.15.69:0/4194937386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9db80ffc30 0x7f9db8100040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009990 tx=0x7f9da8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.621+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4194937386 shutdown_connections 2023-07-06T20:18:54.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.621+0000 7f9dbe7ff700 1 --2- 172.21.15.69:0/4194937386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9db81017b0 0x7f9db8108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.621+0000 7f9dbe7ff700 1 --2- 172.21.15.69:0/4194937386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9db80ffc30 0x7f9db8100040 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.621+0000 7f9dbe7ff700 1 --2- 172.21.15.69:0/4194937386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db8100e20 0x7f9db8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.621+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4194937386 >> 172.21.15.69:0/4194937386 conn(0x7f9db80fb250 msgr2=0x7f9db80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:54.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.621+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4194937386 shutdown_connections 2023-07-06T20:18:54.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.621+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4194937386 wait complete. 2023-07-06T20:18:54.626 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.622+0000 7f9dbe7ff700 1 Processor -- start 2023-07-06T20:18:54.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.622+0000 7f9dbe7ff700 1 -- start start 2023-07-06T20:18:54.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbe7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db80ffc30 0x7f9db8197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:54.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbe7ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9db8100e20 0x7f9db8198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:54.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbe7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9db81017b0 0x7f9db819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:54.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbe7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db819f740 con 0x7f9db81017b0 2023-07-06T20:18:54.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbe7ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db819f880 con 0x7f9db8100e20 2023-07-06T20:18:54.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbe7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db819fa10 con 0x7f9db80ffc30 2023-07-06T20:18:54.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbcd9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9db81017b0 0x7f9db819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:54.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbcd9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9db81017b0 0x7f9db819f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56872/0 (socket says 172.21.15.69:56872) 2023-07-06T20:18:54.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbcd9c700 1 -- 172.21.15.69:0/4015159854 learned_addr learned my addr 172.21.15.69:0/4015159854 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:54.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9db7fff700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db80ffc30 0x7f9db8197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:54.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9db77fe700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9db8100e20 0x7f9db8198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:54.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbcd9c700 1 -- 172.21.15.69:0/4015159854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db80ffc30 msgr2=0x7f9db8197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:54.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbcd9c700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db80ffc30 0x7f9db8197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbcd9c700 1 -- 172.21.15.69:0/4015159854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9db8100e20 msgr2=0x7f9db8198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:54.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbcd9c700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9db8100e20 0x7f9db8198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.623+0000 7f9dbcd9c700 1 -- 172.21.15.69:0/4015159854 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da80095f0 con 0x7f9db81017b0 2023-07-06T20:18:54.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.624+0000 7f9dbcd9c700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9db81017b0 0x7f9db819f170 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9db000ef30 tx=0x7f9db000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:54.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.624+0000 7f9db57fa700 1 -- 172.21.15.69:0/4015159854 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0018070 con 0x7f9db81017b0 2023-07-06T20:18:54.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.624+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4015159854 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db819fc90 con 0x7f9db81017b0 2023-07-06T20:18:54.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.624+0000 7f9db57fa700 1 -- 172.21.15.69:0/4015159854 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9db000f040 con 0x7f9db81017b0 2023-07-06T20:18:54.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.624+0000 7f9db57fa700 1 -- 172.21.15.69:0/4015159854 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db000cd00 con 0x7f9db81017b0 2023-07-06T20:18:54.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.624+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4015159854 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db81a01a0 con 0x7f9db81017b0 2023-07-06T20:18:54.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.625+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4015159854 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db810b450 con 0x7f9db81017b0 2023-07-06T20:18:54.630 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.627+0000 7f9db57fa700 1 -- 172.21.15.69:0/4015159854 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9db0003a00 con 0x7f9db81017b0 2023-07-06T20:18:54.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.629+0000 7f9db57fa700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9da00a11e0 0x7f9da00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:54.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.629+0000 7f9db57fa700 1 -- 172.21.15.69:0/4015159854 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9db00cc640 con 0x7f9db81017b0 2023-07-06T20:18:54.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.629+0000 7f9db7fff700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9da00a11e0 0x7f9da00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:54.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.629+0000 7f9db7fff700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9da00a11e0 0x7f9da00a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9da800b2e0 tx=0x7f9da8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:54.634 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.632+0000 7f9db57fa700 1 -- 172.21.15.69:0/4015159854 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9db0095c50 con 0x7f9db81017b0 2023-07-06T20:18:54.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:54 smithi069 ceph-mon[126399]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:54.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:54 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3477858806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:54.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.830+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4015159854 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9db81a0750 con 0x7f9db81017b0 2023-07-06T20:18:54.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.831+0000 7f9db57fa700 1 -- 172.21.15.69:0/4015159854 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9db0099270 con 0x7f9db81017b0 2023-07-06T20:18:54.833 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2023-07-06T20:18:54.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.833+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4015159854 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9da00a11e0 msgr2=0x7f9da00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:54.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.833+0000 7f9dbe7ff700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9da00a11e0 0x7f9da00a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9da800b2e0 tx=0x7f9da8005fb0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.833+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4015159854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9db81017b0 msgr2=0x7f9db819f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:54.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.833+0000 7f9dbe7ff700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9db81017b0 0x7f9db819f170 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9db000ef30 tx=0x7f9db000c3a0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.833+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4015159854 shutdown_connections 2023-07-06T20:18:54.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.833+0000 7f9dbe7ff700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9da00a11e0 0x7f9da00a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.833+0000 7f9dbe7ff700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9db81017b0 0x7f9db819f170 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.833+0000 7f9dbe7ff700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9db80ffc30 0x7f9db8197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.833+0000 7f9dbe7ff700 1 --2- 172.21.15.69:0/4015159854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9db8100e20 0x7f9db8198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.833+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4015159854 >> 172.21.15.69:0/4015159854 conn(0x7f9db80fb250 msgr2=0x7f9db80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.834+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4015159854 shutdown_connections 2023-07-06T20:18:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:54.834+0000 7f9dbe7ff700 1 -- 172.21.15.69:0/4015159854 wait complete. 2023-07-06T20:18:54.855 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:18:54.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:54 smithi098 ceph-mon[128968]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:54.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:54 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3477858806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:55.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.052+0000 7fcc933ff700 1 -- 172.21.15.69:0/701637992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc8c100e10 msgr2=0x7fcc8c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.052+0000 7fcc933ff700 1 --2- 172.21.15.69:0/701637992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc8c100e10 0x7fcc8c101260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009990 tx=0x7fcc7c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.052+0000 7fcc933ff700 1 -- 172.21.15.69:0/701637992 shutdown_connections 2023-07-06T20:18:55.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.052+0000 7fcc933ff700 1 --2- 172.21.15.69:0/701637992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc8c1017a0 0x7fcc8c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.052+0000 7fcc933ff700 1 --2- 172.21.15.69:0/701637992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc8c0ffc20 0x7fcc8c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.052+0000 7fcc933ff700 1 --2- 172.21.15.69:0/701637992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc8c100e10 0x7fcc8c101260 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.052+0000 7fcc933ff700 1 -- 172.21.15.69:0/701637992 >> 172.21.15.69:0/701637992 conn(0x7fcc8c0fb260 msgr2=0x7fcc8c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:55.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.052+0000 7fcc933ff700 1 -- 172.21.15.69:0/701637992 shutdown_connections 2023-07-06T20:18:55.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.052+0000 7fcc933ff700 1 -- 172.21.15.69:0/701637992 wait complete. 2023-07-06T20:18:55.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.053+0000 7fcc933ff700 1 Processor -- start 2023-07-06T20:18:55.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.053+0000 7fcc933ff700 1 -- start start 2023-07-06T20:18:55.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.053+0000 7fcc933ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc8c0ffc20 0x7fcc8c0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:55.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.053+0000 7fcc933ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc8c100e10 0x7fcc8c072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:55.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc933ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc8c1017a0 0x7fcc8c072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:55.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc933ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c073710 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc933ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c073880 con 0x7fcc8c100e10 2023-07-06T20:18:55.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc933ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c19f9d0 con 0x7fcc8c1017a0 2023-07-06T20:18:55.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc9119b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc8c0ffc20 0x7fcc8c0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:55.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc9119b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc8c0ffc20 0x7fcc8c0743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56890/0 (socket says 172.21.15.69:56890) 2023-07-06T20:18:55.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc9119b700 1 -- 172.21.15.69:0/1069931124 learned_addr learned my addr 172.21.15.69:0/1069931124 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:55.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc9099a700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc8c100e10 0x7fcc8c072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:55.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc9199c700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc8c1017a0 0x7fcc8c072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:55.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc9119b700 1 -- 172.21.15.69:0/1069931124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc8c1017a0 msgr2=0x7fcc8c072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc9119b700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc8c1017a0 0x7fcc8c072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc9119b700 1 -- 172.21.15.69:0/1069931124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc8c100e10 msgr2=0x7fcc8c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc9119b700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc8c100e10 0x7fcc8c072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc9119b700 1 -- 172.21.15.69:0/1069931124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c0095f0 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc9199c700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc8c1017a0 0x7fcc8c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:55.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.054+0000 7fcc9119b700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc8c0ffc20 0x7fcc8c0743f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800d980 tx=0x7fcc8800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:55.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.055+0000 7fcc827fc700 1 -- 172.21.15.69:0/1069931124 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8800b950 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.055+0000 7fcc933ff700 1 -- 172.21.15.69:0/1069931124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8c19fcb0 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.055+0000 7fcc827fc700 1 -- 172.21.15.69:0/1069931124 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc88007620 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.055+0000 7fcc827fc700 1 -- 172.21.15.69:0/1069931124 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8800f3c0 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.055+0000 7fcc933ff700 1 -- 172.21.15.69:0/1069931124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8c1a01c0 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.056+0000 7fcc933ff700 1 -- 172.21.15.69:0/1069931124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8c10b440 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.058+0000 7fcc827fc700 1 -- 172.21.15.69:0/1069931124 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fcc88010640 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.060+0000 7fcc827fc700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcc780a11e0 0x7fcc780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:55.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.060+0000 7fcc827fc700 1 -- 172.21.15.69:0/1069931124 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fcc880cc9f0 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.060+0000 7fcc9099a700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcc780a11e0 0x7fcc780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:55.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.061+0000 7fcc9099a700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcc780a11e0 0x7fcc780a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00b5c0 tx=0x7fcc7c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:55.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.064+0000 7fcc827fc700 1 -- 172.21.15.69:0/1069931124 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcc88095f70 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:54 smithi156 ceph-mon[129280]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:55.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:54 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3477858806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:55.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.262+0000 7fcc933ff700 1 -- 172.21.15.69:0/1069931124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc8c106b20 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.262+0000 7fcc827fc700 1 -- 172.21.15.69:0/1069931124 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc88099590 con 0x7fcc8c0ffc20 2023-07-06T20:18:55.265 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2023-07-06T20:18:55.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.264+0000 7fcc933ff700 1 -- 172.21.15.69:0/1069931124 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcc780a11e0 msgr2=0x7fcc780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.264+0000 7fcc933ff700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcc780a11e0 0x7fcc780a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00b5c0 tx=0x7fcc7c005fd0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.264+0000 7fcc933ff700 1 -- 172.21.15.69:0/1069931124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc8c0ffc20 msgr2=0x7fcc8c0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.264+0000 7fcc933ff700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc8c0ffc20 0x7fcc8c0743f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800d980 tx=0x7fcc8800dc90 comp rx=0 tx=0).stop 2023-07-06T20:18:55.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.264+0000 7fcc933ff700 1 -- 172.21.15.69:0/1069931124 shutdown_connections 2023-07-06T20:18:55.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.264+0000 7fcc933ff700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcc780a11e0 0x7fcc780a3690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.264+0000 7fcc933ff700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc8c1017a0 0x7fcc8c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.264+0000 7fcc933ff700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc8c0ffc20 0x7fcc8c0743f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.264+0000 7fcc933ff700 1 --2- 172.21.15.69:0/1069931124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc8c100e10 0x7fcc8c072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.264+0000 7fcc933ff700 1 -- 172.21.15.69:0/1069931124 >> 172.21.15.69:0/1069931124 conn(0x7fcc8c0fb260 msgr2=0x7fcc8c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:55.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.265+0000 7fcc933ff700 1 -- 172.21.15.69:0/1069931124 shutdown_connections 2023-07-06T20:18:55.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.265+0000 7fcc933ff700 1 -- 172.21.15.69:0/1069931124 wait complete. 2023-07-06T20:18:55.285 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:18:55.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.480+0000 7fb00a1ff700 1 -- 172.21.15.69:0/144605618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0040ffc10 msgr2=0x7fb004100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.480+0000 7fb00a1ff700 1 --2- 172.21.15.69:0/144605618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0040ffc10 0x7fb004100020 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faff4009990 tx=0x7faff4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.480+0000 7fb00a1ff700 1 -- 172.21.15.69:0/144605618 shutdown_connections 2023-07-06T20:18:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.480+0000 7fb00a1ff700 1 --2- 172.21.15.69:0/144605618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb004101790 0x7fb004108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.480+0000 7fb00a1ff700 1 --2- 172.21.15.69:0/144605618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0040ffc10 0x7fb004100020 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.480+0000 7fb00a1ff700 1 --2- 172.21.15.69:0/144605618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb004100e00 0x7fb004101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.480+0000 7fb00a1ff700 1 -- 172.21.15.69:0/144605618 >> 172.21.15.69:0/144605618 conn(0x7fb0040fb210 msgr2=0x7fb0040fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.480+0000 7fb00a1ff700 1 -- 172.21.15.69:0/144605618 shutdown_connections 2023-07-06T20:18:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.480+0000 7fb00a1ff700 1 -- 172.21.15.69:0/144605618 wait complete. 2023-07-06T20:18:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.481+0000 7fb00a1ff700 1 Processor -- start 2023-07-06T20:18:55.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.481+0000 7fb00a1ff700 1 -- start start 2023-07-06T20:18:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.481+0000 7fb00a1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0040ffc10 0x7fb004197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb00a1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb004100e00 0x7fb004198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb00a1ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb004101790 0x7fb00419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb00a1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00419f740 con 0x7fb0040ffc10 2023-07-06T20:18:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb00a1ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00419f880 con 0x7fb004101790 2023-07-06T20:18:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb00a1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00419fa10 con 0x7fb004100e00 2023-07-06T20:18:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb0037fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0040ffc10 0x7fb004197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb0037fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0040ffc10 0x7fb004197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56898/0 (socket says 172.21.15.69:56898) 2023-07-06T20:18:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb0037fe700 1 -- 172.21.15.69:0/1770592919 learned_addr learned my addr 172.21.15.69:0/1770592919 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb003fff700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb004101790 0x7fb00419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb002ffd700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb004100e00 0x7fb004198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb0037fe700 1 -- 172.21.15.69:0/1770592919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb004100e00 msgr2=0x7fb004198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb0037fe700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb004100e00 0x7fb004198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb0037fe700 1 -- 172.21.15.69:0/1770592919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb004101790 msgr2=0x7fb00419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb0037fe700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb004101790 0x7fb00419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb0037fe700 1 -- 172.21.15.69:0/1770592919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff40095f0 con 0x7fb0040ffc10 2023-07-06T20:18:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb003fff700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb004101790 0x7fb00419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.482+0000 7fb0037fe700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0040ffc10 0x7fb004197bc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faff400b620 tx=0x7faff40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.483+0000 7fb000ff9700 1 -- 172.21.15.69:0/1770592919 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff401d070 con 0x7fb0040ffc10 2023-07-06T20:18:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.483+0000 7fb000ff9700 1 -- 172.21.15.69:0/1770592919 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faff4021410 con 0x7fb0040ffc10 2023-07-06T20:18:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.483+0000 7fb00a1ff700 1 -- 172.21.15.69:0/1770592919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00419fc90 con 0x7fb0040ffc10 2023-07-06T20:18:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.483+0000 7fb000ff9700 1 -- 172.21.15.69:0/1770592919 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4017650 con 0x7fb0040ffc10 2023-07-06T20:18:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.483+0000 7fb00a1ff700 1 -- 172.21.15.69:0/1770592919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0041a00b0 con 0x7fb0040ffc10 2023-07-06T20:18:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.484+0000 7fb00a1ff700 1 -- 172.21.15.69:0/1770592919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00410b430 con 0x7fb0040ffc10 2023-07-06T20:18:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.486+0000 7fb000ff9700 1 -- 172.21.15.69:0/1770592919 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7faff4021580 con 0x7fb0040ffc10 2023-07-06T20:18:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.487+0000 7fb000ff9700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fafec0a11e0 0x7fafec0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.487+0000 7fb000ff9700 1 -- 172.21.15.69:0/1770592919 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7faff40ce070 con 0x7fb0040ffc10 2023-07-06T20:18:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.487+0000 7fb002ffd700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fafec0a11e0 0x7fafec0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:55.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.488+0000 7fb002ffd700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fafec0a11e0 0x7fafec0a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faff8005950 tx=0x7faff80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.491+0000 7fb000ff9700 1 -- 172.21.15.69:0/1770592919 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7faff40976e0 con 0x7fb0040ffc10 2023-07-06T20:18:55.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.693+0000 7fb00a1ff700 1 -- 172.21.15.69:0/1770592919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0041a06c0 con 0x7fb0040ffc10 2023-07-06T20:18:55.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.693+0000 7fb000ff9700 1 -- 172.21.15.69:0/1770592919 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7faff409ad00 con 0x7fb0040ffc10 2023-07-06T20:18:55.696 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2023-07-06T20:18:55.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.696+0000 7fb00a1ff700 1 -- 172.21.15.69:0/1770592919 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fafec0a11e0 msgr2=0x7fafec0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.696+0000 7fb00a1ff700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fafec0a11e0 0x7fafec0a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faff8005950 tx=0x7faff80058e0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.696+0000 7fb00a1ff700 1 -- 172.21.15.69:0/1770592919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0040ffc10 msgr2=0x7fb004197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.696+0000 7fb00a1ff700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0040ffc10 0x7fb004197bc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faff400b620 tx=0x7faff40057f0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.696+0000 7fb00a1ff700 1 -- 172.21.15.69:0/1770592919 shutdown_connections 2023-07-06T20:18:55.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.696+0000 7fb00a1ff700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fafec0a11e0 0x7fafec0a3690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.696+0000 7fb00a1ff700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb004101790 0x7fb00419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.696+0000 7fb00a1ff700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0040ffc10 0x7fb004197bc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.696+0000 7fb00a1ff700 1 --2- 172.21.15.69:0/1770592919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb004100e00 0x7fb004198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.696+0000 7fb00a1ff700 1 -- 172.21.15.69:0/1770592919 >> 172.21.15.69:0/1770592919 conn(0x7fb0040fb210 msgr2=0x7fb0040fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:55.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.696+0000 7fb00a1ff700 1 -- 172.21.15.69:0/1770592919 shutdown_connections 2023-07-06T20:18:55.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.696+0000 7fb00a1ff700 1 -- 172.21.15.69:0/1770592919 wait complete. 2023-07-06T20:18:55.720 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:18:55.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:55 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/4015159854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:18:55.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:55 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1069931124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:18:55.915 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.913+0000 7f0e677ff700 1 -- 172.21.15.69:0/1599753548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e601017b0 msgr2=0x7f0e60108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.915 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.913+0000 7f0e677ff700 1 --2- 172.21.15.69:0/1599753548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e601017b0 0x7f0e60108070 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800b0a0 tx=0x7f0e5800b3b0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.915 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.913+0000 7f0e677ff700 1 -- 172.21.15.69:0/1599753548 shutdown_connections 2023-07-06T20:18:55.916 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.913+0000 7f0e677ff700 1 --2- 172.21.15.69:0/1599753548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e601017b0 0x7f0e60108070 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.916 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.913+0000 7f0e677ff700 1 --2- 172.21.15.69:0/1599753548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e600ffc30 0x7f0e60100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.916 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.913+0000 7f0e677ff700 1 --2- 172.21.15.69:0/1599753548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e60100e20 0x7f0e60101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.916 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.913+0000 7f0e677ff700 1 -- 172.21.15.69:0/1599753548 >> 172.21.15.69:0/1599753548 conn(0x7f0e600fb210 msgr2=0x7f0e600fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:55.916 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.913+0000 7f0e677ff700 1 -- 172.21.15.69:0/1599753548 shutdown_connections 2023-07-06T20:18:55.916 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.913+0000 7f0e677ff700 1 -- 172.21.15.69:0/1599753548 wait complete. 2023-07-06T20:18:55.918 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.914+0000 7f0e677ff700 1 Processor -- start 2023-07-06T20:18:55.919 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.915+0000 7f0e677ff700 1 -- start start 2023-07-06T20:18:55.919 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.915+0000 7f0e677ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e600ffc30 0x7f0e60195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:55.919 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.915+0000 7f0e677ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e60100e20 0x7f0e60195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:55.919 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.915+0000 7f0e677ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e601017b0 0x7f0e6019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:55.919 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.915+0000 7f0e677ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6019d5f0 con 0x7f0e601017b0 2023-07-06T20:18:55.919 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.915+0000 7f0e677ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6019d730 con 0x7f0e60100e20 2023-07-06T20:18:55.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.915+0000 7f0e677ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6019d8c0 con 0x7f0e600ffc30 2023-07-06T20:18:55.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.915+0000 7f0e6559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e600ffc30 0x7f0e60195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:55.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.915+0000 7f0e65d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e601017b0 0x7f0e6019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:55.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.915+0000 7f0e6559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e600ffc30 0x7f0e60195a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:48078/0 (socket says 172.21.15.69:48078) 2023-07-06T20:18:55.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.915+0000 7f0e6559b700 1 -- 172.21.15.69:0/3456016898 learned_addr learned my addr 172.21.15.69:0/3456016898 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:55.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.915+0000 7f0e64d9a700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e60100e20 0x7f0e60195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:55.920 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.916+0000 7f0e65d9c700 1 -- 172.21.15.69:0/3456016898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e600ffc30 msgr2=0x7f0e60195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.921 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.916+0000 7f0e65d9c700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e600ffc30 0x7f0e60195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.921 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.916+0000 7f0e65d9c700 1 -- 172.21.15.69:0/3456016898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e60100e20 msgr2=0x7f0e60195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:55.921 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.916+0000 7f0e65d9c700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e60100e20 0x7f0e60195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:55.921 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.916+0000 7f0e65d9c700 1 -- 172.21.15.69:0/3456016898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e58009d00 con 0x7f0e601017b0 2023-07-06T20:18:55.921 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.916+0000 7f0e65d9c700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e601017b0 0x7f0e6019d020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800b070 tx=0x7f0e5800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:55.922 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.916+0000 7f0e567fc700 1 -- 172.21.15.69:0/3456016898 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800e040 con 0x7f0e601017b0 2023-07-06T20:18:55.922 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.916+0000 7f0e567fc700 1 -- 172.21.15.69:0/3456016898 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e58009480 con 0x7f0e601017b0 2023-07-06T20:18:55.922 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.916+0000 7f0e677ff700 1 -- 172.21.15.69:0/3456016898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6019db40 con 0x7f0e601017b0 2023-07-06T20:18:55.922 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.916+0000 7f0e567fc700 1 -- 172.21.15.69:0/3456016898 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e580052a0 con 0x7f0e601017b0 2023-07-06T20:18:55.923 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.916+0000 7f0e677ff700 1 -- 172.21.15.69:0/3456016898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6019df10 con 0x7f0e601017b0 2023-07-06T20:18:55.924 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.917+0000 7f0e677ff700 1 -- 172.21.15.69:0/3456016898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6010b460 con 0x7f0e601017b0 2023-07-06T20:18:55.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.919+0000 7f0e567fc700 1 -- 172.21.15.69:0/3456016898 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0e5801d480 con 0x7f0e601017b0 2023-07-06T20:18:55.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.920+0000 7f0e567fc700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0e4c0a1220 0x7f0e4c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:55.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.921+0000 7f0e567fc700 1 -- 172.21.15.69:0/3456016898 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f0e580cdd50 con 0x7f0e601017b0 2023-07-06T20:18:55.926 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.921+0000 7f0e6559b700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0e4c0a1220 0x7f0e4c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:55.926 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.921+0000 7f0e6559b700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0e4c0a1220 0x7f0e4c0a36d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005950 tx=0x7f0e5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:55.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:55.924+0000 7f0e567fc700 1 -- 172.21.15.69:0/3456016898 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0e58097360 con 0x7f0e601017b0 2023-07-06T20:18:56.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:55 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/4015159854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:18:56.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:55 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1069931124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:18:56.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:55 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/4015159854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:18:56.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:55 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1069931124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:18:56.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.171+0000 7f0e677ff700 1 -- 172.21.15.69:0/3456016898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e60106a40 con 0x7f0e601017b0 2023-07-06T20:18:56.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.171+0000 7f0e567fc700 1 -- 172.21.15.69:0/3456016898 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10795 (secure 0 0 0) 0x7f0e58011390 con 0x7f0e601017b0 2023-07-06T20:18:56.174 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:56.174 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-06T20:17:28.144582+0000","modified":"2023-07-06T20:17:33.557511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24539,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/3235525224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3235525224},{"type":"v1","addr":"172.21.15.98:6839","nonce":3235525224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:18:56.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.174+0000 7f0e677ff700 1 -- 172.21.15.69:0/3456016898 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0e4c0a1220 msgr2=0x7f0e4c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:56.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.174+0000 7f0e677ff700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0e4c0a1220 0x7f0e4c0a36d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005950 tx=0x7f0e5c009260 comp rx=0 tx=0).stop 2023-07-06T20:18:56.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.174+0000 7f0e677ff700 1 -- 172.21.15.69:0/3456016898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e601017b0 msgr2=0x7f0e6019d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:56.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.174+0000 7f0e677ff700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e601017b0 0x7f0e6019d020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800b070 tx=0x7f0e5800bc80 comp rx=0 tx=0).stop 2023-07-06T20:18:56.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.174+0000 7f0e677ff700 1 -- 172.21.15.69:0/3456016898 shutdown_connections 2023-07-06T20:18:56.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.174+0000 7f0e677ff700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0e4c0a1220 0x7f0e4c0a36d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.174+0000 7f0e677ff700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e601017b0 0x7f0e6019d020 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.174+0000 7f0e677ff700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e600ffc30 0x7f0e60195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.174+0000 7f0e677ff700 1 --2- 172.21.15.69:0/3456016898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e60100e20 0x7f0e60195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.174+0000 7f0e677ff700 1 -- 172.21.15.69:0/3456016898 >> 172.21.15.69:0/3456016898 conn(0x7f0e600fb210 msgr2=0x7f0e600fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:56.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.174+0000 7f0e677ff700 1 -- 172.21.15.69:0/3456016898 shutdown_connections 2023-07-06T20:18:56.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.174+0000 7f0e677ff700 1 -- 172.21.15.69:0/3456016898 wait complete. 2023-07-06T20:18:56.178 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2023-07-06T20:18:56.195 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:18:56.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.389+0000 7f6c893ff700 1 -- 172.21.15.69:0/1076488112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c84100e20 msgr2=0x7f6c84101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:56.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.389+0000 7f6c893ff700 1 --2- 172.21.15.69:0/1076488112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c84100e20 0x7f6c84101270 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6c780098f0 tx=0x7f6c78009c00 comp rx=0 tx=0).stop 2023-07-06T20:18:56.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.389+0000 7f6c893ff700 1 -- 172.21.15.69:0/1076488112 shutdown_connections 2023-07-06T20:18:56.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.389+0000 7f6c893ff700 1 --2- 172.21.15.69:0/1076488112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c841017b0 0x7f6c84108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.389+0000 7f6c893ff700 1 --2- 172.21.15.69:0/1076488112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c840ffc30 0x7f6c84100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.389+0000 7f6c893ff700 1 --2- 172.21.15.69:0/1076488112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c84100e20 0x7f6c84101270 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.389+0000 7f6c893ff700 1 -- 172.21.15.69:0/1076488112 >> 172.21.15.69:0/1076488112 conn(0x7f6c840fb210 msgr2=0x7f6c840fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:56.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.389+0000 7f6c893ff700 1 -- 172.21.15.69:0/1076488112 shutdown_connections 2023-07-06T20:18:56.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.389+0000 7f6c893ff700 1 -- 172.21.15.69:0/1076488112 wait complete. 2023-07-06T20:18:56.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.390+0000 7f6c893ff700 1 Processor -- start 2023-07-06T20:18:56.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c893ff700 1 -- start start 2023-07-06T20:18:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c893ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c840ffc30 0x7f6c84195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c893ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c84100e20 0x7f6c84195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:56.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c893ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c841017b0 0x7f6c8419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:56.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c893ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8419d500 con 0x7f6c841017b0 2023-07-06T20:18:56.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c893ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8419d640 con 0x7f6c84100e20 2023-07-06T20:18:56.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c893ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8419d7d0 con 0x7f6c840ffc30 2023-07-06T20:18:56.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c837fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c841017b0 0x7f6c8419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:56.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c827fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c84100e20 0x7f6c84195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:56.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c827fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c84100e20 0x7f6c84195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:42610/0 (socket says 172.21.15.69:42610) 2023-07-06T20:18:56.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c837fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c841017b0 0x7f6c8419cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56928/0 (socket says 172.21.15.69:56928) 2023-07-06T20:18:56.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c827fc700 1 -- 172.21.15.69:0/323904523 learned_addr learned my addr 172.21.15.69:0/323904523 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:56.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.391+0000 7f6c82ffd700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c840ffc30 0x7f6c84195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:56.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.392+0000 7f6c827fc700 1 -- 172.21.15.69:0/323904523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c840ffc30 msgr2=0x7f6c84195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:56.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.392+0000 7f6c827fc700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c840ffc30 0x7f6c84195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.392+0000 7f6c827fc700 1 -- 172.21.15.69:0/323904523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c841017b0 msgr2=0x7f6c8419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:56.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.392+0000 7f6c827fc700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c841017b0 0x7f6c8419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.392+0000 7f6c827fc700 1 -- 172.21.15.69:0/323904523 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c78009520 con 0x7f6c84100e20 2023-07-06T20:18:56.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.392+0000 7f6c837fe700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c841017b0 0x7f6c8419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:56.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.392+0000 7f6c827fc700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c84100e20 0x7f6c84195ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005e90 tx=0x7f6c7800b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:56.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.392+0000 7f6c6bfff700 1 -- 172.21.15.69:0/323904523 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7801c070 con 0x7f6c84100e20 2023-07-06T20:18:56.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.392+0000 7f6c893ff700 1 -- 172.21.15.69:0/323904523 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8419da50 con 0x7f6c84100e20 2023-07-06T20:18:56.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.392+0000 7f6c6bfff700 1 -- 172.21.15.69:0/323904523 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c7800f7f0 con 0x7f6c84100e20 2023-07-06T20:18:56.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.392+0000 7f6c6bfff700 1 -- 172.21.15.69:0/323904523 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c780049c0 con 0x7f6c84100e20 2023-07-06T20:18:56.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.393+0000 7f6c893ff700 1 -- 172.21.15.69:0/323904523 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8419de70 con 0x7f6c84100e20 2023-07-06T20:18:56.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.393+0000 7f6c893ff700 1 -- 172.21.15.69:0/323904523 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c841071c0 con 0x7f6c84100e20 2023-07-06T20:18:56.403 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.395+0000 7f6c6bfff700 1 -- 172.21.15.69:0/323904523 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6c78020070 con 0x7f6c84100e20 2023-07-06T20:18:56.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.397+0000 7f6c6bfff700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6c6c0a1220 0x7f6c6c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:56.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.397+0000 7f6c6bfff700 1 -- 172.21.15.69:0/323904523 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f6c780cdf50 con 0x7f6c84100e20 2023-07-06T20:18:56.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.397+0000 7f6c82ffd700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6c6c0a1220 0x7f6c6c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:56.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.397+0000 7f6c82ffd700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6c6c0a1220 0x7f6c6c0a36d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6c84100c80 tx=0x7f6c74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:56.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.399+0000 7f6c6bfff700 1 -- 172.21.15.69:0/323904523 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6c78097560 con 0x7f6c84100e20 2023-07-06T20:18:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.602+0000 7f6c893ff700 1 -- 172.21.15.69:0/323904523 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c8402cb30 con 0x7f6c84100e20 2023-07-06T20:18:56.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.604+0000 7f6c6bfff700 1 -- 172.21.15.69:0/323904523 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22564 (secure 0 0 0) 0x7f6c7809ab80 con 0x7f6c84100e20 2023-07-06T20:18:56.607 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:56.608 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":85,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:17:58.736720+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:18:56.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.607+0000 7f6c893ff700 1 -- 172.21.15.69:0/323904523 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6c6c0a1220 msgr2=0x7f6c6c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:56.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.607+0000 7f6c893ff700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6c6c0a1220 0x7f6c6c0a36d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6c84100c80 tx=0x7f6c74009260 comp rx=0 tx=0).stop 2023-07-06T20:18:56.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.607+0000 7f6c893ff700 1 -- 172.21.15.69:0/323904523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c84100e20 msgr2=0x7f6c84195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:56.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.607+0000 7f6c893ff700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c84100e20 0x7f6c84195ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005e90 tx=0x7f6c7800b730 comp rx=0 tx=0).stop 2023-07-06T20:18:56.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.607+0000 7f6c893ff700 1 -- 172.21.15.69:0/323904523 shutdown_connections 2023-07-06T20:18:56.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.607+0000 7f6c893ff700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6c6c0a1220 0x7f6c6c0a36d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.607+0000 7f6c893ff700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c841017b0 0x7f6c8419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.607+0000 7f6c893ff700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c840ffc30 0x7f6c84195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.607+0000 7f6c893ff700 1 --2- 172.21.15.69:0/323904523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c84100e20 0x7f6c84195ec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.607+0000 7f6c893ff700 1 -- 172.21.15.69:0/323904523 >> 172.21.15.69:0/323904523 conn(0x7f6c840fb210 msgr2=0x7f6c84104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:56.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.607+0000 7f6c893ff700 1 -- 172.21.15.69:0/323904523 shutdown_connections 2023-07-06T20:18:56.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.607+0000 7f6c893ff700 1 -- 172.21.15.69:0/323904523 wait complete. 2023-07-06T20:18:56.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:14.022799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:17:26.523183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:17:27.543386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:18:56.630 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:18:56.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.826+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/1025104440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cb81017a0 msgr2=0x7f3cb8108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:56.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.826+0000 7f3cbf1ff700 1 --2- 172.21.15.69:0/1025104440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cb81017a0 0x7f3cb8108060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000b0a0 tx=0x7f3cb000b3b0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.826+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/1025104440 shutdown_connections 2023-07-06T20:18:56.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.826+0000 7f3cbf1ff700 1 --2- 172.21.15.69:0/1025104440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cb81017a0 0x7f3cb8108060 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.826+0000 7f3cbf1ff700 1 --2- 172.21.15.69:0/1025104440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cb80ffc20 0x7f3cb8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.826+0000 7f3cbf1ff700 1 --2- 172.21.15.69:0/1025104440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cb8100e10 0x7f3cb8101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.826+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/1025104440 >> 172.21.15.69:0/1025104440 conn(0x7f3cb80fb260 msgr2=0x7f3cb80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:56.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.826+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/1025104440 shutdown_connections 2023-07-06T20:18:56.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.826+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/1025104440 wait complete. 2023-07-06T20:18:56.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.827+0000 7f3cbf1ff700 1 Processor -- start 2023-07-06T20:18:56.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.827+0000 7f3cbf1ff700 1 -- start start 2023-07-06T20:18:56.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.827+0000 7f3cbf1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cb80ffc20 0x7f3cb80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:56.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.827+0000 7f3cbf1ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cb8100e10 0x7f3cb8072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:56.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.827+0000 7f3cbf1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cb81017a0 0x7f3cb8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:56.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.827+0000 7f3cbf1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb8073710 con 0x7f3cb81017a0 2023-07-06T20:18:56.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.827+0000 7f3cbf1ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb8073880 con 0x7f3cb8100e10 2023-07-06T20:18:56.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.827+0000 7f3cbf1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb819faf0 con 0x7f3cb80ffc20 2023-07-06T20:18:56.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cbd79c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cb81017a0 0x7f3cb8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:56.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cbd79c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cb81017a0 0x7f3cb8072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56954/0 (socket says 172.21.15.69:56954) 2023-07-06T20:18:56.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cbd79c700 1 -- 172.21.15.69:0/292330893 learned_addr learned my addr 172.21.15.69:0/292330893 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:56.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cbcf9b700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cb80ffc20 0x7f3cb80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:56.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cb7fff700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cb8100e10 0x7f3cb8072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:56.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cbd79c700 1 -- 172.21.15.69:0/292330893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cb80ffc20 msgr2=0x7f3cb80743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:56.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cbd79c700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cb80ffc20 0x7f3cb80743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cbd79c700 1 -- 172.21.15.69:0/292330893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cb8100e10 msgr2=0x7f3cb8072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:56.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cbd79c700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cb8100e10 0x7f3cb8072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:56.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cbd79c700 1 -- 172.21.15.69:0/292330893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb0009d00 con 0x7f3cb81017a0 2023-07-06T20:18:56.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cbd79c700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cb81017a0 0x7f3cb8072f70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000d010 tx=0x7f3cb000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:56.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cb5ffb700 1 -- 172.21.15.69:0/292330893 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb000e040 con 0x7f3cb81017a0 2023-07-06T20:18:56.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.828+0000 7f3cb5ffb700 1 -- 172.21.15.69:0/292330893 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cb000bdb0 con 0x7f3cb81017a0 2023-07-06T20:18:56.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.829+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/292330893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb819fd70 con 0x7f3cb81017a0 2023-07-06T20:18:56.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.829+0000 7f3cb5ffb700 1 -- 172.21.15.69:0/292330893 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0005260 con 0x7f3cb81017a0 2023-07-06T20:18:56.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.829+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/292330893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb81a0190 con 0x7f3cb81017a0 2023-07-06T20:18:56.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.830+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/292330893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb8106ab0 con 0x7f3cb81017a0 2023-07-06T20:18:56.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.832+0000 7f3cb5ffb700 1 -- 172.21.15.69:0/292330893 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3cb001d480 con 0x7f3cb81017a0 2023-07-06T20:18:56.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.833+0000 7f3cb5ffb700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3ca80a1260 0x7f3ca80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:56.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.833+0000 7f3cb5ffb700 1 -- 172.21.15.69:0/292330893 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f3cb00ce070 con 0x7f3cb81017a0 2023-07-06T20:18:56.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.833+0000 7f3cbcf9b700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3ca80a1260 0x7f3ca80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:56.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.834+0000 7f3cbcf9b700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3ca80a1260 0x7f3ca80a3710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8100c70 tx=0x7f3cac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:56.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:56.835+0000 7f3cb5ffb700 1 -- 172.21.15.69:0/292330893 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3cb00976e0 con 0x7f3cb81017a0 2023-07-06T20:18:56.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:56 smithi098 ceph-mon[128968]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-06T20:18:56.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:56 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1770592919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:18:56.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:56 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3456016898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:56.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:56 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/323904523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:56 smithi069 ceph-mon[126399]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-06T20:18:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:56 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1770592919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:18:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:56 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3456016898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:56 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/323904523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:57.035 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.033+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/292330893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3cb810b700 con 0x7f3cb81017a0 2023-07-06T20:18:57.036 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.034+0000 7f3cb5ffb700 1 -- 172.21.15.69:0/292330893 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3cb009ad00 con 0x7f3cb81017a0 2023-07-06T20:18:57.036 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2023-07-06T20:18:57.038 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.036+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/292330893 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3ca80a1260 msgr2=0x7f3ca80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:57.038 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.036+0000 7f3cbf1ff700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3ca80a1260 0x7f3ca80a3710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8100c70 tx=0x7f3cac009260 comp rx=0 tx=0).stop 2023-07-06T20:18:57.038 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.036+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/292330893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cb81017a0 msgr2=0x7f3cb8072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:57.039 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.036+0000 7f3cbf1ff700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cb81017a0 0x7f3cb8072f70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000d010 tx=0x7f3cb000bc80 comp rx=0 tx=0).stop 2023-07-06T20:18:57.039 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.036+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/292330893 shutdown_connections 2023-07-06T20:18:57.039 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.036+0000 7f3cbf1ff700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3ca80a1260 0x7f3ca80a3710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.039 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.036+0000 7f3cbf1ff700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cb81017a0 0x7f3cb8072f70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.039 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.036+0000 7f3cbf1ff700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cb80ffc20 0x7f3cb80743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.039 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.036+0000 7f3cbf1ff700 1 --2- 172.21.15.69:0/292330893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cb8100e10 0x7f3cb8072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.039 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.036+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/292330893 >> 172.21.15.69:0/292330893 conn(0x7f3cb80fb260 msgr2=0x7f3cb80fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:57.040 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.036+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/292330893 shutdown_connections 2023-07-06T20:18:57.040 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.036+0000 7f3cbf1ff700 1 -- 172.21.15.69:0/292330893 wait complete. 2023-07-06T20:18:57.057 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:18:57.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:56 smithi156 ceph-mon[129280]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-06T20:18:57.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:56 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1770592919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:18:57.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:56 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3456016898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:57.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:56 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/323904523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:18:57.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.253+0000 7fb9621ff700 1 -- 172.21.15.69:0/2872832815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb95c0ffc40 msgr2=0x7fb95c100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:57.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.253+0000 7fb9621ff700 1 --2- 172.21.15.69:0/2872832815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb95c0ffc40 0x7fb95c100050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009990 tx=0x7fb94c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.254+0000 7fb9621ff700 1 -- 172.21.15.69:0/2872832815 shutdown_connections 2023-07-06T20:18:57.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.254+0000 7fb9621ff700 1 --2- 172.21.15.69:0/2872832815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb95c1017c0 0x7fb95c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.254+0000 7fb9621ff700 1 --2- 172.21.15.69:0/2872832815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb95c0ffc40 0x7fb95c100050 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.254+0000 7fb9621ff700 1 --2- 172.21.15.69:0/2872832815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c100e30 0x7fb95c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.254+0000 7fb9621ff700 1 -- 172.21.15.69:0/2872832815 >> 172.21.15.69:0/2872832815 conn(0x7fb95c0fb260 msgr2=0x7fb95c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.254+0000 7fb9621ff700 1 -- 172.21.15.69:0/2872832815 shutdown_connections 2023-07-06T20:18:57.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.254+0000 7fb9621ff700 1 -- 172.21.15.69:0/2872832815 wait complete. 2023-07-06T20:18:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.255+0000 7fb9621ff700 1 Processor -- start 2023-07-06T20:18:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.255+0000 7fb9621ff700 1 -- start start 2023-07-06T20:18:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.255+0000 7fb9621ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb95c0ffc40 0x7fb95c0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.255+0000 7fb9621ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb95c100e30 0x7fb95c072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.255+0000 7fb9621ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c1017c0 0x7fb95c072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:57.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.255+0000 7fb9621ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c073710 con 0x7fb95c0ffc40 2023-07-06T20:18:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.255+0000 7fb9621ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c073880 con 0x7fb95c1017c0 2023-07-06T20:18:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.255+0000 7fb9621ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c19f9f0 con 0x7fb95c100e30 2023-07-06T20:18:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.255+0000 7fb95b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb95c0ffc40 0x7fb95c0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.255+0000 7fb95b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb95c0ffc40 0x7fb95c0743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56980/0 (socket says 172.21.15.69:56980) 2023-07-06T20:18:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.255+0000 7fb95b7fe700 1 -- 172.21.15.69:0/1868954208 learned_addr learned my addr 172.21.15.69:0/1868954208 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb95bfff700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c1017c0 0x7fb95c072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:57.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb95affd700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb95c100e30 0x7fb95c072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb95b7fe700 1 -- 172.21.15.69:0/1868954208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb95c100e30 msgr2=0x7fb95c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb95b7fe700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb95c100e30 0x7fb95c072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb95b7fe700 1 -- 172.21.15.69:0/1868954208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c1017c0 msgr2=0x7fb95c072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb95b7fe700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c1017c0 0x7fb95c072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb95b7fe700 1 -- 172.21.15.69:0/1868954208 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb950009520 con 0x7fb95c0ffc40 2023-07-06T20:18:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb95bfff700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c1017c0 0x7fb95c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:57.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb95b7fe700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb95c0ffc40 0x7fb95c0743f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb94c004730 tx=0x7fb94c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:57.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb958ff9700 1 -- 172.21.15.69:0/1868954208 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c01d070 con 0x7fb95c0ffc40 2023-07-06T20:18:57.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb958ff9700 1 -- 172.21.15.69:0/1868954208 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb94c004cd0 con 0x7fb95c0ffc40 2023-07-06T20:18:57.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb9621ff700 1 -- 172.21.15.69:0/1868954208 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c0095f0 con 0x7fb95c0ffc40 2023-07-06T20:18:57.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb958ff9700 1 -- 172.21.15.69:0/1868954208 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c005530 con 0x7fb95c0ffc40 2023-07-06T20:18:57.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.256+0000 7fb9621ff700 1 -- 172.21.15.69:0/1868954208 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c19ff00 con 0x7fb95c0ffc40 2023-07-06T20:18:57.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.257+0000 7fb9621ff700 1 -- 172.21.15.69:0/1868954208 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95c106b70 con 0x7fb95c0ffc40 2023-07-06T20:18:57.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.259+0000 7fb958ff9700 1 -- 172.21.15.69:0/1868954208 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb94c00fbc0 con 0x7fb95c0ffc40 2023-07-06T20:18:57.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.261+0000 7fb958ff9700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb9440a1260 0x7fb9440a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:57.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.261+0000 7fb958ff9700 1 -- 172.21.15.69:0/1868954208 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb94c0cddd0 con 0x7fb95c0ffc40 2023-07-06T20:18:57.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.262+0000 7fb95affd700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb9440a1260 0x7fb9440a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:57.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.262+0000 7fb95affd700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb9440a1260 0x7fb9440a3710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb950009d80 tx=0x7fb950009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:57.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.265+0000 7fb958ff9700 1 -- 172.21.15.69:0/1868954208 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb94c05a770 con 0x7fb95c0ffc40 2023-07-06T20:18:57.463 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.461+0000 7fb9621ff700 1 -- 172.21.15.69:0/1868954208 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb95c02cd00 con 0x7fb95c0ffc40 2023-07-06T20:18:57.464 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.462+0000 7fb958ff9700 1 -- 172.21.15.69:0/1868954208 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb94c0973e0 con 0x7fb95c0ffc40 2023-07-06T20:18:57.464 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2023-07-06T20:18:57.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.464+0000 7fb9621ff700 1 -- 172.21.15.69:0/1868954208 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb9440a1260 msgr2=0x7fb9440a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:57.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.464+0000 7fb9621ff700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb9440a1260 0x7fb9440a3710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb950009d80 tx=0x7fb950009260 comp rx=0 tx=0).stop 2023-07-06T20:18:57.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.464+0000 7fb9621ff700 1 -- 172.21.15.69:0/1868954208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb95c0ffc40 msgr2=0x7fb95c0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:57.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.464+0000 7fb9621ff700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb95c0ffc40 0x7fb95c0743f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb94c004730 tx=0x7fb94c009510 comp rx=0 tx=0).stop 2023-07-06T20:18:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.464+0000 7fb9621ff700 1 -- 172.21.15.69:0/1868954208 shutdown_connections 2023-07-06T20:18:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.464+0000 7fb9621ff700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb9440a1260 0x7fb9440a3710 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.464+0000 7fb9621ff700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c1017c0 0x7fb95c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.464+0000 7fb9621ff700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb95c0ffc40 0x7fb95c0743f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.464+0000 7fb9621ff700 1 --2- 172.21.15.69:0/1868954208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb95c100e30 0x7fb95c072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.464+0000 7fb9621ff700 1 -- 172.21.15.69:0/1868954208 >> 172.21.15.69:0/1868954208 conn(0x7fb95c0fb260 msgr2=0x7fb95c0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.464+0000 7fb9621ff700 1 -- 172.21.15.69:0/1868954208 shutdown_connections 2023-07-06T20:18:57.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.464+0000 7fb9621ff700 1 -- 172.21.15.69:0/1868954208 wait complete. 2023-07-06T20:18:57.485 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:18:57.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.712+0000 7fed06dff700 1 -- 172.21.15.69:0/886419083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed000ffc20 msgr2=0x7fed00100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:57.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.712+0000 7fed06dff700 1 --2- 172.21.15.69:0/886419083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed000ffc20 0x7fed00100030 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009990 tx=0x7fecf4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.712+0000 7fed06dff700 1 -- 172.21.15.69:0/886419083 shutdown_connections 2023-07-06T20:18:57.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.712+0000 7fed06dff700 1 --2- 172.21.15.69:0/886419083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed001017a0 0x7fed00108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.712+0000 7fed06dff700 1 --2- 172.21.15.69:0/886419083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed000ffc20 0x7fed00100030 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.712+0000 7fed06dff700 1 --2- 172.21.15.69:0/886419083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed00100e10 0x7fed00101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.712+0000 7fed06dff700 1 -- 172.21.15.69:0/886419083 >> 172.21.15.69:0/886419083 conn(0x7fed000fb260 msgr2=0x7fed000fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:57.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.712+0000 7fed06dff700 1 -- 172.21.15.69:0/886419083 shutdown_connections 2023-07-06T20:18:57.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.712+0000 7fed06dff700 1 -- 172.21.15.69:0/886419083 wait complete. 2023-07-06T20:18:57.718 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.713+0000 7fed06dff700 1 Processor -- start 2023-07-06T20:18:57.718 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.713+0000 7fed06dff700 1 -- start start 2023-07-06T20:18:57.718 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.713+0000 7fed06dff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed000ffc20 0x7fed00197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:57.718 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.713+0000 7fed06dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed00100e10 0x7fed00198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:57.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fed06dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed001017a0 0x7fed0019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:57.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fed06dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0019f750 con 0x7fed00100e10 2023-07-06T20:18:57.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fed06dff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0019f890 con 0x7fed000ffc20 2023-07-06T20:18:57.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fed06dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0019fa20 con 0x7fed001017a0 2023-07-06T20:18:57.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fecfffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed00100e10 0x7fed00198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:57.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fecfffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed00100e10 0x7fed00198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56996/0 (socket says 172.21.15.69:56996) 2023-07-06T20:18:57.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fecfffff700 1 -- 172.21.15.69:0/641733097 learned_addr learned my addr 172.21.15.69:0/641733097 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:57.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fed04b9b700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed000ffc20 0x7fed00197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:57.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fed0539c700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed001017a0 0x7fed0019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:57.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fecfffff700 1 -- 172.21.15.69:0/641733097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed001017a0 msgr2=0x7fed0019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:57.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fecfffff700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed001017a0 0x7fed0019f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fecfffff700 1 -- 172.21.15.69:0/641733097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed000ffc20 msgr2=0x7fed00197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:57.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fecfffff700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed000ffc20 0x7fed00197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fecfffff700 1 -- 172.21.15.69:0/641733097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf40095f0 con 0x7fed00100e10 2023-07-06T20:18:57.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fed0539c700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed001017a0 0x7fed0019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:57.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.714+0000 7fecfffff700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed00100e10 0x7fed00198110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fecec00e9a0 tx=0x7fecec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:57.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.715+0000 7fecfdffb700 1 -- 172.21.15.69:0/641733097 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec009790 con 0x7fed00100e10 2023-07-06T20:18:57.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.715+0000 7fecfdffb700 1 -- 172.21.15.69:0/641733097 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecec00cbd0 con 0x7fed00100e10 2023-07-06T20:18:57.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.715+0000 7fecfdffb700 1 -- 172.21.15.69:0/641733097 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec018680 con 0x7fed00100e10 2023-07-06T20:18:57.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.715+0000 7fed06dff700 1 -- 172.21.15.69:0/641733097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0019fd00 con 0x7fed00100e10 2023-07-06T20:18:57.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.715+0000 7fed06dff700 1 -- 172.21.15.69:0/641733097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed001a01c0 con 0x7fed00100e10 2023-07-06T20:18:57.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.716+0000 7fed06dff700 1 -- 172.21.15.69:0/641733097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed00192300 con 0x7fed00100e10 2023-07-06T20:18:57.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.718+0000 7fecfdffb700 1 -- 172.21.15.69:0/641733097 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fecec00cd40 con 0x7fed00100e10 2023-07-06T20:18:57.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.719+0000 7fecfdffb700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fecf00a1260 0x7fecf00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:57.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.719+0000 7fecfdffb700 1 -- 172.21.15.69:0/641733097 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fecec014070 con 0x7fed00100e10 2023-07-06T20:18:57.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.719+0000 7fed04b9b700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fecf00a1260 0x7fecf00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:57.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.720+0000 7fed04b9b700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fecf00a1260 0x7fecf00a3710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fecf400b2e0 tx=0x7fecf4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:57.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.723+0000 7fecfdffb700 1 -- 172.21.15.69:0/641733097 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fecec051770 con 0x7fed00100e10 2023-07-06T20:18:57.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:57 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/292330893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:18:57.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:57 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1868954208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:18:57.923 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.921+0000 7fed06dff700 1 -- 172.21.15.69:0/641733097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed0002cb00 con 0x7fed00100e10 2023-07-06T20:18:57.923 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.921+0000 7fecfdffb700 1 -- 172.21.15.69:0/641733097 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fecec0962b0 con 0x7fed00100e10 2023-07-06T20:18:57.924 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2023-07-06T20:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.924+0000 7fed06dff700 1 -- 172.21.15.69:0/641733097 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fecf00a1260 msgr2=0x7fecf00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.924+0000 7fed06dff700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fecf00a1260 0x7fecf00a3710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fecf400b2e0 tx=0x7fecf4005fb0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.924+0000 7fed06dff700 1 -- 172.21.15.69:0/641733097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed00100e10 msgr2=0x7fed00198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.924+0000 7fed06dff700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed00100e10 0x7fed00198110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fecec00e9a0 tx=0x7fecec00ecb0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.924+0000 7fed06dff700 1 -- 172.21.15.69:0/641733097 shutdown_connections 2023-07-06T20:18:57.926 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.924+0000 7fed06dff700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fecf00a1260 0x7fecf00a3710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.924+0000 7fed06dff700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed001017a0 0x7fed0019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.924+0000 7fed06dff700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed000ffc20 0x7fed00197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.924+0000 7fed06dff700 1 --2- 172.21.15.69:0/641733097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed00100e10 0x7fed00198110 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:57.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.924+0000 7fed06dff700 1 -- 172.21.15.69:0/641733097 >> 172.21.15.69:0/641733097 conn(0x7fed000fb260 msgr2=0x7fed001066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:57.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.924+0000 7fed06dff700 1 -- 172.21.15.69:0/641733097 shutdown_connections 2023-07-06T20:18:57.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:57.924+0000 7fed06dff700 1 -- 172.21.15.69:0/641733097 wait complete. 2023-07-06T20:18:57.945 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:18:58.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:57 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/292330893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:18:58.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:57 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1868954208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:18:58.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:57 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/292330893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:18:58.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:57 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1868954208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:18:58.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.138+0000 7f5feb5ff700 1 -- 172.21.15.69:0/796131910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe40ffc30 msgr2=0x7f5fe4100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:58.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.138+0000 7f5feb5ff700 1 --2- 172.21.15.69:0/796131910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe40ffc30 0x7f5fe4100040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009990 tx=0x7f5fe0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.138+0000 7f5feb5ff700 1 -- 172.21.15.69:0/796131910 shutdown_connections 2023-07-06T20:18:58.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.138+0000 7f5feb5ff700 1 --2- 172.21.15.69:0/796131910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fe41017b0 0x7f5fe4108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.138+0000 7f5feb5ff700 1 --2- 172.21.15.69:0/796131910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe40ffc30 0x7f5fe4100040 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.138+0000 7f5feb5ff700 1 --2- 172.21.15.69:0/796131910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe4100e20 0x7f5fe4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.138+0000 7f5feb5ff700 1 -- 172.21.15.69:0/796131910 >> 172.21.15.69:0/796131910 conn(0x7f5fe40fb210 msgr2=0x7f5fe40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:58.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.138+0000 7f5feb5ff700 1 -- 172.21.15.69:0/796131910 shutdown_connections 2023-07-06T20:18:58.142 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.138+0000 7f5feb5ff700 1 -- 172.21.15.69:0/796131910 wait complete. 2023-07-06T20:18:58.143 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.139+0000 7f5feb5ff700 1 Processor -- start 2023-07-06T20:18:58.143 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.139+0000 7f5feb5ff700 1 -- start start 2023-07-06T20:18:58.143 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5feb5ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe40ffc30 0x7f5fe4195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:58.144 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5feb5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe4100e20 0x7f5fe4195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:58.144 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5feb5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fe41017b0 0x7f5fe419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:58.144 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5feb5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe419d4f0 con 0x7f5fe4100e20 2023-07-06T20:18:58.144 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5feb5ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe419d630 con 0x7f5fe40ffc30 2023-07-06T20:18:58.144 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5feb5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe419d7c0 con 0x7f5fe41017b0 2023-07-06T20:18:58.144 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5fe8b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe4100e20 0x7f5fe4195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:58.144 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5fe8b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe4100e20 0x7f5fe4195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57010/0 (socket says 172.21.15.69:57010) 2023-07-06T20:18:58.145 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5fe8b9a700 1 -- 172.21.15.69:0/3408637634 learned_addr learned my addr 172.21.15.69:0/3408637634 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:58.145 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5fe939b700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe40ffc30 0x7f5fe4195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:58.145 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5fe9b9c700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fe41017b0 0x7f5fe419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:58.145 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5fe8b9a700 1 -- 172.21.15.69:0/3408637634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fe41017b0 msgr2=0x7f5fe419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:58.145 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5fe8b9a700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fe41017b0 0x7f5fe419cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.145 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5fe8b9a700 1 -- 172.21.15.69:0/3408637634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe40ffc30 msgr2=0x7f5fe4195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:58.145 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5fe8b9a700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe40ffc30 0x7f5fe4195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.146 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5fe8b9a700 1 -- 172.21.15.69:0/3408637634 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe00095f0 con 0x7f5fe4100e20 2023-07-06T20:18:58.146 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.140+0000 7f5fe9b9c700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fe41017b0 0x7f5fe419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:58.146 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.141+0000 7f5fe8b9a700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe4100e20 0x7f5fe4195eb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400e9a0 tx=0x7f5fd400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:58.146 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.141+0000 7f5fda7fc700 1 -- 172.21.15.69:0/3408637634 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4009790 con 0x7f5fe4100e20 2023-07-06T20:18:58.146 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.141+0000 7f5fda7fc700 1 -- 172.21.15.69:0/3408637634 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fd400cbd0 con 0x7f5fe4100e20 2023-07-06T20:18:58.146 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.141+0000 7f5feb5ff700 1 -- 172.21.15.69:0/3408637634 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe419daa0 con 0x7f5fe4100e20 2023-07-06T20:18:58.146 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.141+0000 7f5fda7fc700 1 -- 172.21.15.69:0/3408637634 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4018680 con 0x7f5fe4100e20 2023-07-06T20:18:58.146 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.141+0000 7f5feb5ff700 1 -- 172.21.15.69:0/3408637634 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe419e000 con 0x7f5fe4100e20 2023-07-06T20:18:58.147 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.142+0000 7f5feb5ff700 1 -- 172.21.15.69:0/3408637634 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe410b450 con 0x7f5fe4100e20 2023-07-06T20:18:58.147 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.144+0000 7f5fda7fc700 1 -- 172.21.15.69:0/3408637634 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5fd400cd40 con 0x7f5fe4100e20 2023-07-06T20:18:58.147 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.145+0000 7f5fda7fc700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5fd00a12a0 0x7f5fd00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:58.148 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.145+0000 7f5fda7fc700 1 -- 172.21.15.69:0/3408637634 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f5fd4014070 con 0x7f5fe4100e20 2023-07-06T20:18:58.148 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.145+0000 7f5fe939b700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5fd00a12a0 0x7f5fd00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:58.148 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.146+0000 7f5fe939b700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5fd00a12a0 0x7f5fd00a3750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000b2e0 tx=0x7f5fe0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:58.151 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.149+0000 7f5fda7fc700 1 -- 172.21.15.69:0/3408637634 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5fd4051770 con 0x7f5fe4100e20 2023-07-06T20:18:58.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.392+0000 7f5feb5ff700 1 -- 172.21.15.69:0/3408637634 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fe410b640 con 0x7f5fe4100e20 2023-07-06T20:18:58.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.392+0000 7f5fda7fc700 1 -- 172.21.15.69:0/3408637634 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10795 (secure 0 0 0) 0x7f5fd40962b0 con 0x7f5fe4100e20 2023-07-06T20:18:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:58.395 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-06T20:17:28.144582+0000","modified":"2023-07-06T20:17:33.557511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24539,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/3235525224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3235525224},{"type":"v1","addr":"172.21.15.98:6839","nonce":3235525224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:18:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.395+0000 7f5feb5ff700 1 -- 172.21.15.69:0/3408637634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5fd00a12a0 msgr2=0x7f5fd00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.395+0000 7f5feb5ff700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5fd00a12a0 0x7f5fd00a3750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000b2e0 tx=0x7f5fe0005fb0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.395+0000 7f5feb5ff700 1 -- 172.21.15.69:0/3408637634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe4100e20 msgr2=0x7f5fe4195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.395+0000 7f5feb5ff700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe4100e20 0x7f5fe4195eb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400e9a0 tx=0x7f5fd400ecb0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.395+0000 7f5feb5ff700 1 -- 172.21.15.69:0/3408637634 shutdown_connections 2023-07-06T20:18:58.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.395+0000 7f5feb5ff700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5fd00a12a0 0x7f5fd00a3750 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.395+0000 7f5feb5ff700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fe41017b0 0x7f5fe419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.395+0000 7f5feb5ff700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fe40ffc30 0x7f5fe4195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.395+0000 7f5feb5ff700 1 --2- 172.21.15.69:0/3408637634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe4100e20 0x7f5fe4195eb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.395+0000 7f5feb5ff700 1 -- 172.21.15.69:0/3408637634 >> 172.21.15.69:0/3408637634 conn(0x7f5fe40fb210 msgr2=0x7f5fe40fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:58.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.395+0000 7f5feb5ff700 1 -- 172.21.15.69:0/3408637634 shutdown_connections 2023-07-06T20:18:58.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.395+0000 7f5feb5ff700 1 -- 172.21.15.69:0/3408637634 wait complete. 2023-07-06T20:18:58.399 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2023-07-06T20:18:58.416 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:18:58.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.614+0000 7f6b84dff700 1 -- 172.21.15.69:0/1168883292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b800ffc30 msgr2=0x7f6b80100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:58.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.614+0000 7f6b84dff700 1 --2- 172.21.15.69:0/1168883292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b800ffc30 0x7f6b80100040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009990 tx=0x7f6b70009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.614+0000 7f6b84dff700 1 -- 172.21.15.69:0/1168883292 shutdown_connections 2023-07-06T20:18:58.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.614+0000 7f6b84dff700 1 --2- 172.21.15.69:0/1168883292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b801017b0 0x7f6b80108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.614+0000 7f6b84dff700 1 --2- 172.21.15.69:0/1168883292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b800ffc30 0x7f6b80100040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.614+0000 7f6b84dff700 1 --2- 172.21.15.69:0/1168883292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b80100e20 0x7f6b80101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.614+0000 7f6b84dff700 1 -- 172.21.15.69:0/1168883292 >> 172.21.15.69:0/1168883292 conn(0x7f6b800fb210 msgr2=0x7f6b800fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:58.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.614+0000 7f6b84dff700 1 -- 172.21.15.69:0/1168883292 shutdown_connections 2023-07-06T20:18:58.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.614+0000 7f6b84dff700 1 -- 172.21.15.69:0/1168883292 wait complete. 2023-07-06T20:18:58.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.616+0000 7f6b84dff700 1 Processor -- start 2023-07-06T20:18:58.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.616+0000 7f6b84dff700 1 -- start start 2023-07-06T20:18:58.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.616+0000 7f6b84dff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b800ffc30 0x7f6b80195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:58.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.616+0000 7f6b84dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b80100e20 0x7f6b80195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:58.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.616+0000 7f6b84dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b801017b0 0x7f6b8019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:58.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.616+0000 7f6b84dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8019d4f0 con 0x7f6b801017b0 2023-07-06T20:18:58.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b84dff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8019d630 con 0x7f6b800ffc30 2023-07-06T20:18:58.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b84dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8019d7c0 con 0x7f6b80100e20 2023-07-06T20:18:58.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b7e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b800ffc30 0x7f6b80195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:58.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b7ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b801017b0 0x7f6b8019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:58.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b7e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b800ffc30 0x7f6b80195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:42708/0 (socket says 172.21.15.69:42708) 2023-07-06T20:18:58.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b7e59c700 1 -- 172.21.15.69:0/3716287623 learned_addr learned my addr 172.21.15.69:0/3716287623 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:58.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b7dd9b700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b80100e20 0x7f6b80195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:58.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b7e59c700 1 -- 172.21.15.69:0/3716287623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b80100e20 msgr2=0x7f6b80195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:58.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b7e59c700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b80100e20 0x7f6b80195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b7e59c700 1 -- 172.21.15.69:0/3716287623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b801017b0 msgr2=0x7f6b8019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:58.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b7e59c700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b801017b0 0x7f6b8019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b7e59c700 1 -- 172.21.15.69:0/3716287623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b700095f0 con 0x7f6b800ffc30 2023-07-06T20:18:58.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.617+0000 7f6b7e59c700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b800ffc30 0x7f6b80195970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6b70004730 tx=0x7f6b70004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:58.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.618+0000 7f6b6f7fe700 1 -- 172.21.15.69:0/3716287623 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7001d070 con 0x7f6b800ffc30 2023-07-06T20:18:58.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.618+0000 7f6b84dff700 1 -- 172.21.15.69:0/3716287623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8019da40 con 0x7f6b800ffc30 2023-07-06T20:18:58.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.618+0000 7f6b6f7fe700 1 -- 172.21.15.69:0/3716287623 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b70021410 con 0x7f6b800ffc30 2023-07-06T20:18:58.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.618+0000 7f6b84dff700 1 -- 172.21.15.69:0/3716287623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8019dec0 con 0x7f6b800ffc30 2023-07-06T20:18:58.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.618+0000 7f6b6f7fe700 1 -- 172.21.15.69:0/3716287623 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70017650 con 0x7f6b800ffc30 2023-07-06T20:18:58.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.619+0000 7f6b84dff700 1 -- 172.21.15.69:0/3716287623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8010b450 con 0x7f6b800ffc30 2023-07-06T20:18:58.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.621+0000 7f6b6f7fe700 1 -- 172.21.15.69:0/3716287623 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6b70021580 con 0x7f6b800ffc30 2023-07-06T20:18:58.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.622+0000 7f6b6f7fe700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6b680a11a0 0x7f6b680a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:58.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.622+0000 7f6b6f7fe700 1 -- 172.21.15.69:0/3716287623 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f6b700cdc80 con 0x7f6b800ffc30 2023-07-06T20:18:58.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.622+0000 7f6b7dd9b700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6b680a11a0 0x7f6b680a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:58.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.623+0000 7f6b7dd9b700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6b680a11a0 0x7f6b680a3650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009c60 tx=0x7f6b74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:58.626 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.624+0000 7f6b6f7fe700 1 -- 172.21.15.69:0/3716287623 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6b700973f0 con 0x7f6b800ffc30 2023-07-06T20:18:58.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:58 smithi069 ceph-mon[126399]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:58.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:58 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2023-07-06T20:18:58.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:58 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/641733097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:18:58.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:58 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3408637634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:58.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:58 smithi069 ceph-mon[126399]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-06T20:18:58.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.829+0000 7f6b84dff700 1 -- 172.21.15.69:0/3716287623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b80106ad0 con 0x7f6b800ffc30 2023-07-06T20:18:58.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.831+0000 7f6b6f7fe700 1 -- 172.21.15.69:0/3716287623 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22612 (secure 0 0 0) 0x7f6b7009aa10 con 0x7f6b800ffc30 2023-07-06T20:18:58.834 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:18:58.834 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":86,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:18:58.629769+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:18:58.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.834+0000 7f6b84dff700 1 -- 172.21.15.69:0/3716287623 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6b680a11a0 msgr2=0x7f6b680a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:58.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.834+0000 7f6b84dff700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6b680a11a0 0x7f6b680a3650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009c60 tx=0x7f6b74009260 comp rx=0 tx=0).stop 2023-07-06T20:18:58.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.834+0000 7f6b84dff700 1 -- 172.21.15.69:0/3716287623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b800ffc30 msgr2=0x7f6b80195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:58.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.834+0000 7f6b84dff700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b800ffc30 0x7f6b80195970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6b70004730 tx=0x7f6b70004760 comp rx=0 tx=0).stop 2023-07-06T20:18:58.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.834+0000 7f6b84dff700 1 -- 172.21.15.69:0/3716287623 shutdown_connections 2023-07-06T20:18:58.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.834+0000 7f6b84dff700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6b680a11a0 0x7f6b680a3650 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.834+0000 7f6b84dff700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b801017b0 0x7f6b8019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.834+0000 7f6b84dff700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b800ffc30 0x7f6b80195970 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.834+0000 7f6b84dff700 1 --2- 172.21.15.69:0/3716287623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b80100e20 0x7f6b80195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:58.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.834+0000 7f6b84dff700 1 -- 172.21.15.69:0/3716287623 >> 172.21.15.69:0/3716287623 conn(0x7f6b800fb210 msgr2=0x7f6b800fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:58.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.834+0000 7f6b84dff700 1 -- 172.21.15.69:0/3716287623 shutdown_connections 2023-07-06T20:18:58.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:58.834+0000 7f6b84dff700 1 -- 172.21.15.69:0/3716287623 wait complete. 2023-07-06T20:18:58.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-06T20:17:28.144582+0000', 'modified': '2023-07-06T20:17:33.557511+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24584, 'mds_1': 24539, 'mds_2': 24512}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/2525319359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 2525319359}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 2525319359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24512': {'gid': 24512, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.98:6835/827333150', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 827333150}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 827333150}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24517': {'gid': 24517, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6837/595692705', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 595692705}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 595692705}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24535': {'gid': 24535, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6839/4200610083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 4200610083}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 4200610083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.98:6837/4010267682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 4010267682}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 4010267682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24584': {'gid': 24584, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6839/3235525224', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 3235525224}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 3235525224}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T20:18:58.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-06T20:18:58.856 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-07-06T20:18:59.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.051+0000 7f71a15ff700 1 -- 172.21.15.69:0/3439597831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f719c100e10 msgr2=0x7f719c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.051+0000 7f71a15ff700 1 --2- 172.21.15.69:0/3439597831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f719c100e10 0x7f719c101260 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7190009990 tx=0x7f7190009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.051+0000 7f71a15ff700 1 -- 172.21.15.69:0/3439597831 shutdown_connections 2023-07-06T20:18:59.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.051+0000 7f71a15ff700 1 --2- 172.21.15.69:0/3439597831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c1017a0 0x7f719c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.051+0000 7f71a15ff700 1 --2- 172.21.15.69:0/3439597831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f719c0ffc20 0x7f719c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.051+0000 7f71a15ff700 1 --2- 172.21.15.69:0/3439597831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f719c100e10 0x7f719c101260 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.051+0000 7f71a15ff700 1 -- 172.21.15.69:0/3439597831 >> 172.21.15.69:0/3439597831 conn(0x7f719c0fb260 msgr2=0x7f719c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.051+0000 7f71a15ff700 1 -- 172.21.15.69:0/3439597831 shutdown_connections 2023-07-06T20:18:59.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.051+0000 7f71a15ff700 1 -- 172.21.15.69:0/3439597831 wait complete. 2023-07-06T20:18:59.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.052+0000 7f71a15ff700 1 Processor -- start 2023-07-06T20:18:59.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.052+0000 7f71a15ff700 1 -- start start 2023-07-06T20:18:59.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.052+0000 7f71a15ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f719c0ffc20 0x7f719c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.052+0000 7f71a15ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f719c100e10 0x7f719c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.052+0000 7f71a15ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c1017a0 0x7f719c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.052+0000 7f71a15ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c19f710 con 0x7f719c100e10 2023-07-06T20:18:59.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.052+0000 7f71a15ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c19f850 con 0x7f719c0ffc20 2023-07-06T20:18:59.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.052+0000 7f71a15ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c19f9e0 con 0x7f719c1017a0 2023-07-06T20:18:59.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f719a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f719c100e10 0x7f719c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f719a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f719c100e10 0x7f719c1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57056/0 (socket says 172.21.15.69:57056) 2023-07-06T20:18:59.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f719a7fc700 1 -- 172.21.15.69:0/530060343 learned_addr learned my addr 172.21.15.69:0/530060343 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f719affd700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f719c0ffc20 0x7f719c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f719b7fe700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c1017a0 0x7f719c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f719a7fc700 1 -- 172.21.15.69:0/530060343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c1017a0 msgr2=0x7f719c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f719a7fc700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c1017a0 0x7f719c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.058 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f719a7fc700 1 -- 172.21.15.69:0/530060343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f719c0ffc20 msgr2=0x7f719c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f719a7fc700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f719c0ffc20 0x7f719c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f719a7fc700 1 -- 172.21.15.69:0/530060343 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71900095f0 con 0x7f719c100e10 2023-07-06T20:18:59.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f719b7fe700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c1017a0 0x7f719c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:59.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f719a7fc700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f719c100e10 0x7f719c1980d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7190004760 tx=0x7f7190004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f7183fff700 1 -- 172.21.15.69:0/530060343 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719001d070 con 0x7f719c100e10 2023-07-06T20:18:59.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f7183fff700 1 -- 172.21.15.69:0/530060343 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7190004830 con 0x7f719c100e10 2023-07-06T20:18:59.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f7183fff700 1 -- 172.21.15.69:0/530060343 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7190005480 con 0x7f719c100e10 2023-07-06T20:18:59.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.053+0000 7f71a15ff700 1 -- 172.21.15.69:0/530060343 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c19fc60 con 0x7f719c100e10 2023-07-06T20:18:59.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.054+0000 7f71a15ff700 1 -- 172.21.15.69:0/530060343 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c1a0020 con 0x7f719c100e10 2023-07-06T20:18:59.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.056+0000 7f7183fff700 1 -- 172.21.15.69:0/530060343 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f719000fad0 con 0x7f719c100e10 2023-07-06T20:18:59.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.058+0000 7f71b4ef4b80 1 Processor -- start 2023-07-06T20:18:59.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f71b4ef4b80 1 -- start start 2023-07-06T20:18:59.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f71b4ef4b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e0096652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f71b4ef4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009665db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f7183fff700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71840a11d0 0x7f71840a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f71b4ef4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 0x55e0096667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f71b4ef4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e009569c60 con 0x55e009665820 2023-07-06T20:18:59.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f71b4ef4b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0095bc9f0 con 0x55e009664ed0 2023-07-06T20:18:59.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f71b4ef4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0095b5e80 con 0x55e009666340 2023-07-06T20:18:59.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f7183fff700 1 -- 172.21.15.69:0/530060343 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f71900cdad0 con 0x7f719c100e10 2023-07-06T20:18:59.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f719b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e0096652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f719b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e0096652e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:42730/0 (socket says 172.21.15.69:42730) 2023-07-06T20:18:59.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f719affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009665db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f719b7fe700 1 -- 172.21.15.69:0/4293058700 learned_addr learned my addr 172.21.15.69:0/4293058700 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.059+0000 7f719a7fc700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71840a11d0 0x7f71840a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.060+0000 7f719b7fe700 1 --2- 172.21.15.69:0/4293058700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 0x55e0096667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.060+0000 7f719affd700 1 -- 172.21.15.69:0/4293058700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 msgr2=0x55e0096667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.060+0000 7f719affd700 1 --2- 172.21.15.69:0/4293058700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 0x55e0096667e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.060+0000 7f719affd700 1 -- 172.21.15.69:0/4293058700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 msgr2=0x55e0096652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.060+0000 7f719affd700 1 --2- 172.21.15.69:0/4293058700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e0096652e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.060+0000 7f719affd700 1 -- 172.21.15.69:0/4293058700 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0093e3d90 con 0x55e009665820 2023-07-06T20:18:59.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.060+0000 7f719a7fc700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71840a11d0 0x7f71840a3680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7190004800 tx=0x7f7190009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.060+0000 7f719affd700 1 --2- 172.21.15.69:0/4293058700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009665db0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f718c009c80 tx=0x7f718c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b209fbf912d604ca server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.060+0000 7f71817fa700 1 -- 172.21.15.69:0/4293058700 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c011cb0 con 0x55e009665820 2023-07-06T20:18:59.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.060+0000 7f71817fa700 1 -- 172.21.15.69:0/4293058700 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f718c011e10 con 0x55e009665820 2023-07-06T20:18:59.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.060+0000 7f71817fa700 1 -- 172.21.15.69:0/4293058700 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c005280 con 0x55e009665820 2023-07-06T20:18:59.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.061+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/4293058700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 msgr2=0x55e009665db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.061+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/4293058700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009665db0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f718c009c80 tx=0x7f718c009f90 comp rx=0 tx=0).stop 2023-07-06T20:18:59.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.061+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/4293058700 shutdown_connections 2023-07-06T20:18:59.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.061+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/4293058700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 0x55e0096667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.061+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/4293058700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e0096652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.061+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/4293058700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009665db0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.061+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/4293058700 >> 172.21.15.69:0/4293058700 conn(0x55e00965a5f0 msgr2=0x55e00965f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.061+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/4293058700 shutdown_connections 2023-07-06T20:18:59.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.061+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/4293058700 wait complete. 2023-07-06T20:18:59.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.062+0000 7f71b4ef4b80 1 -- start start 2023-07-06T20:18:59.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.062+0000 7f71b4ef4b80 1 Processor -- start 2023-07-06T20:18:59.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.062+0000 7f71b4ef4b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e00966e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.062+0000 7f71b4ef4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e00966ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.062+0000 7f71b4ef4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 0x55e009675db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.062+0000 7f71b4ef4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e009555a70 con 0x55e009665820 2023-07-06T20:18:59.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.062+0000 7f71b4ef4b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e009555770 con 0x55e009664ed0 2023-07-06T20:18:59.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.062+0000 7f71b4ef4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0094398d0 con 0x55e009666340 2023-07-06T20:18:59.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.062+0000 7f71b4ef4b80 10 client.0 _ll_register_callbacks cb 0x55e009553710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:18:59.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f719affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e00966ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f719affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e00966ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57076/0 (socket says 172.21.15.69:57076) 2023-07-06T20:18:59.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f719affd700 1 -- 172.21.15.69:0/1941393833 learned_addr learned my addr 172.21.15.69:0/1941393833 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f719b7fe700 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e00966e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f719b7fe700 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 0x55e009675db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f719affd700 1 -- 172.21.15.69:0/1941393833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 msgr2=0x55e009675db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f719affd700 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 0x55e009675db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f719affd700 1 -- 172.21.15.69:0/1941393833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 msgr2=0x55e00966e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f719affd700 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e00966e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f719affd700 1 -- 172.21.15.69:0/1941393833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718c0098e0 con 0x55e009665820 2023-07-06T20:18:59.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f719b7fe700 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e00966e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:59.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f719affd700 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e00966ed70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f718c011720 tx=0x7f718c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934c0789fbd82e86 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.063+0000 7f717eff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:18:59.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f717eff5700 1 -- 172.21.15.69:0/1941393833 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c004020 con 0x55e009665820 2023-07-06T20:18:59.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f717eff5700 1 -- 172.21.15.69:0/1941393833 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f718c004180 con 0x55e009665820 2023-07-06T20:18:59.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/1941393833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e00950f4a0 con 0x55e009665820 2023-07-06T20:18:59.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f717eff5700 1 -- 172.21.15.69:0/1941393833 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f718c004340 con 0x55e009665820 2023-07-06T20:18:59.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f71b4ef4b80 10 client.15090 fetch_fsmap learned FSMap version 23 2023-07-06T20:18:59.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/1941393833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e0093ccc10 con 0x55e009665820 2023-07-06T20:18:59.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f717eff5700 1 -- 172.21.15.69:0/1941393833 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e00950f4a0 con 0x55e009665820 2023-07-06T20:18:59.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f717eff5700 1 -- 172.21.15.69:0/1941393833 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f718c003bc0 con 0x55e009665820 2023-07-06T20:18:59.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f71b4ef4b80 10 client.15090 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:18:59.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f71b4ef4b80 10 client.15090 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-06T20:18:59.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f71b4ef4b80 20 client.15090 populate_metadata read hostname 'smithi069' 2023-07-06T20:18:59.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x55e0096799a0 0x55e009678e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f71b4ef4b80 4 client.15090 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:18:59.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.064+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/1941393833 --> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e009633d10 con 0x55e0096799a0 2023-07-06T20:18:59.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.065+0000 7f719b7fe700 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x55e0096799a0 0x55e009678e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.065+0000 7f719b7fe700 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x55e0096799a0 0x55e009678e00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99204f3acbe480ab server_cookie=5e93c8e4f686a8ce in_seq=0 out_seq=0 2023-07-06T20:18:59.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.065+0000 7f717eff5700 10 client.15090 ms_handle_connect on v2:172.21.15.98:6834/827333150 2023-07-06T20:18:59.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.066+0000 7f717eff5700 1 -- 172.21.15.69:0/1941393833 <== mds.2 v2:172.21.15.98:6834/827333150 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e009633d10 con 0x55e0096799a0 2023-07-06T20:18:59.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.066+0000 7f717eff5700 10 client.15090 handle_command_reply: tid=0 2023-07-06T20:18:59.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.066+0000 7f71b4ef4b80 1 client.15090 shutdown 2023-07-06T20:18:59.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.066+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/1941393833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 msgr2=0x55e00966ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.066+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e00966ed70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f718c011720 tx=0x7f718c0047e0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.066+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/1941393833 shutdown_connections 2023-07-06T20:18:59.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.066+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 0x55e009675db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.066+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x55e0096799a0 0x55e009678e00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.066+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e00966e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.066+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/1941393833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e00966ed70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.066+0000 7f717eff5700 0 client.15090 ms_handle_reset on v2:172.21.15.98:6834/827333150 2023-07-06T20:18:59.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.066+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/1941393833 >> 172.21.15.69:0/1941393833 conn(0x55e00965a5f0 msgr2=0x55e00965f060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.067+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/1941393833 shutdown_connections 2023-07-06T20:18:59.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.067+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/1941393833 wait complete. 2023-07-06T20:18:59.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.067+0000 7f71b4ef4b80 20 client.15090 trim_cache size 0 max 16384 2023-07-06T20:18:59.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f71b4ef4b80 1 Processor -- start 2023-07-06T20:18:59.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f71b4ef4b80 1 -- start start 2023-07-06T20:18:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f71b4ef4b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e00966d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f71b4ef4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009675d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f71b4ef4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 0x55e00965f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f71b4ef4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00950f4a0 con 0x55e009665820 2023-07-06T20:18:59.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f71b4ef4b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00966fa20 con 0x55e009664ed0 2023-07-06T20:18:59.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f71b4ef4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e009584790 con 0x55e009666340 2023-07-06T20:18:59.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f719affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009675d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f719affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009675d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57090/0 (socket says 172.21.15.69:57090) 2023-07-06T20:18:59.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f719affd700 1 -- 172.21.15.69:0/863671279 learned_addr learned my addr 172.21.15.69:0/863671279 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f719b7fe700 1 --2- 172.21.15.69:0/863671279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e00966d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.085+0000 7f719b7fe700 1 --2- 172.21.15.69:0/863671279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 0x55e00965f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.086+0000 7f719affd700 1 -- 172.21.15.69:0/863671279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 msgr2=0x55e00965f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.086+0000 7f719affd700 1 --2- 172.21.15.69:0/863671279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 0x55e00965f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.086+0000 7f719affd700 1 -- 172.21.15.69:0/863671279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 msgr2=0x55e00966d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.086+0000 7f719affd700 1 --2- 172.21.15.69:0/863671279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e00966d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.086+0000 7f719affd700 1 -- 172.21.15.69:0/863671279 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e00940a200 con 0x55e009665820 2023-07-06T20:18:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.086+0000 7f719b7fe700 1 --2- 172.21.15.69:0/863671279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e00966d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.086+0000 7f719affd700 1 --2- 172.21.15.69:0/863671279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009675d70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f718c0194e0 tx=0x7f718c004850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0459e77b43a4ac1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.086+0000 7f7181ffb700 1 -- 172.21.15.69:0/863671279 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c004d00 con 0x55e009665820 2023-07-06T20:18:59.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.086+0000 7f7181ffb700 1 -- 172.21.15.69:0/863671279 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f718c0044e0 con 0x55e009665820 2023-07-06T20:18:59.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.086+0000 7f7181ffb700 1 -- 172.21.15.69:0/863671279 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c0055d0 con 0x55e009665820 2023-07-06T20:18:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.087+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/863671279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 msgr2=0x55e009675d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.087+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/863671279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009675d70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f718c0194e0 tx=0x7f718c004850 comp rx=0 tx=0).stop 2023-07-06T20:18:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.087+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/863671279 shutdown_connections 2023-07-06T20:18:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.087+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/863671279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009666340 0x55e00965f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.087+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/863671279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009664ed0 0x55e00966d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.087+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/863671279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009675d70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.087+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/863671279 >> 172.21.15.69:0/863671279 conn(0x55e00965a5f0 msgr2=0x55e009682710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.087+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/863671279 shutdown_connections 2023-07-06T20:18:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.087+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/863671279 wait complete. 2023-07-06T20:18:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.087+0000 7f71b4ef4b80 1 -- start start 2023-07-06T20:18:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.087+0000 7f71b4ef4b80 1 Processor -- start 2023-07-06T20:18:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.088+0000 7f71b4ef4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009664ed0 0x55e009670f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.088+0000 7f71b4ef4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009671480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.088+0000 7f71b4ef4b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009666340 0x55e0096cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.088+0000 7f71b4ef4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e009675ec0 con 0x55e009665820 2023-07-06T20:18:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.088+0000 7f71b4ef4b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e009676030 con 0x55e009666340 2023-07-06T20:18:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.088+0000 7f71b4ef4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0096761f0 con 0x55e009664ed0 2023-07-06T20:18:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.088+0000 7f71b4ef4b80 10 client.0 _ll_register_callbacks cb 0x7f7194012440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:18:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.088+0000 7f719affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009671480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.088+0000 7f719affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009671480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57106/0 (socket says 172.21.15.69:57106) 2023-07-06T20:18:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.088+0000 7f719affd700 1 -- 172.21.15.69:0/3185444558 learned_addr learned my addr 172.21.15.69:0/3185444558 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.088+0000 7f719b7fe700 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009666340 0x55e0096cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.088+0000 7f719b7fe700 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009664ed0 0x55e009670f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f719affd700 1 -- 172.21.15.69:0/3185444558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009664ed0 msgr2=0x55e009670f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f719affd700 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009664ed0 0x55e009670f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f719affd700 1 -- 172.21.15.69:0/3185444558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009666340 msgr2=0x55e0096cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f719affd700 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009666340 0x55e0096cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f719affd700 1 -- 172.21.15.69:0/3185444558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0093ccc10 con 0x55e009665820 2023-07-06T20:18:59.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f719b7fe700 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009664ed0 0x55e009670f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:59.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f719affd700 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009671480 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f718c000c00 tx=0x7f718c0049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928af20ef60a0732 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f717eff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:18:59.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f717eff5700 1 -- 172.21.15.69:0/3185444558 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c004020 con 0x55e009665820 2023-07-06T20:18:59.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/3185444558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f718c011e10 con 0x55e009665820 2023-07-06T20:18:59.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f717eff5700 1 -- 172.21.15.69:0/3185444558 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f718c004180 con 0x55e009665820 2023-07-06T20:18:59.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f717eff5700 1 -- 172.21.15.69:0/3185444558 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f718c004360 con 0x55e009665820 2023-07-06T20:18:59.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f71b4ef4b80 10 client.15096 fetch_fsmap learned FSMap version 23 2023-07-06T20:18:59.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.089+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/3185444558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e0093e2260 con 0x55e009665820 2023-07-06T20:18:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.090+0000 7f717eff5700 1 -- 172.21.15.69:0/3185444558 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f718c011e10 con 0x55e009665820 2023-07-06T20:18:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.090+0000 7f717eff5700 1 -- 172.21.15.69:0/3185444558 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f718c00f770 con 0x55e009665820 2023-07-06T20:18:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.090+0000 7f71b4ef4b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:18:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.090+0000 7f71b4ef4b80 10 client.15096 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-06T20:18:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.090+0000 7f71b4ef4b80 20 client.15096 populate_metadata read hostname 'smithi069' 2023-07-06T20:18:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.090+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x55e0096d2460 0x55e0096cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.090+0000 7f71b4ef4b80 4 client.15096 mds_command: new command op to 24512 tid=0[{"prefix": "status"}] 2023-07-06T20:18:59.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.090+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/3185444558 --> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e0095aee60 con 0x55e0096d2460 2023-07-06T20:18:59.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.090+0000 7f719b7fe700 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x55e0096d2460 0x55e0096cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f719b7fe700 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x55e0096d2460 0x55e0096cee00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dab4077cd7e291f5 server_cookie=fad80d75915ad7e7 in_seq=0 out_seq=0 2023-07-06T20:18:59.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f717eff5700 10 client.15096 ms_handle_connect on v2:172.21.15.98:6834/827333150 2023-07-06T20:18:59.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f717eff5700 1 -- 172.21.15.69:0/3185444558 <== mds.2 v2:172.21.15.98:6834/827333150 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55e0095aee60 con 0x55e0096d2460 2023-07-06T20:18:59.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f717eff5700 10 client.15096 handle_command_reply: tid=0 2023-07-06T20:18:59.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f71b4ef4b80 1 client.15096 shutdown 2023-07-06T20:18:59.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/3185444558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 msgr2=0x55e009671480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009671480 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f718c000c00 tx=0x7f718c0049a0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/3185444558 shutdown_connections 2023-07-06T20:18:59.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x55e0096d2460 0x55e0096cee00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e009666340 0x55e0096cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e009664ed0 0x55e009670f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f717eff5700 0 client.15096 ms_handle_reset on v2:172.21.15.98:6834/827333150 2023-07-06T20:18:59.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f71b4ef4b80 1 --2- 172.21.15.69:0/3185444558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e009665820 0x55e009671480 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/3185444558 >> 172.21.15.69:0/3185444558 conn(0x55e00965a5f0 msgr2=0x55e00966d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/3185444558 shutdown_connections 2023-07-06T20:18:59.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.091+0000 7f71b4ef4b80 1 -- 172.21.15.69:0/3185444558 wait complete. 2023-07-06T20:18:59.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71b4ef4b80 20 client.15096 trim_cache size 0 max 16384 2023-07-06T20:18:59.102 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:18:59.102 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "e3e8ec0c-1c38-11ee-9b2f-001a4aab830c", 2023-07-06T20:18:59.103 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2023-07-06T20:18:59.103 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24512, 2023-07-06T20:18:59.103 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2023-07-06T20:18:59.103 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2023-07-06T20:18:59.103 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2023-07-06T20:18:59.103 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 86.538783367999997, 2023-07-06T20:18:59.103 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 23, 2023-07-06T20:18:59.103 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 86, 2023-07-06T20:18:59.104 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 83, 2023-07-06T20:18:59.104 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 114.290371624 2023-07-06T20:18:59.104 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:18:59.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71a15ff700 1 -- 172.21.15.69:0/530060343 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71840a11d0 msgr2=0x7f71840a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71a15ff700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71840a11d0 0x7f71840a3680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7190004800 tx=0x7f7190009310 comp rx=0 tx=0).stop 2023-07-06T20:18:59.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71a15ff700 1 -- 172.21.15.69:0/530060343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f719c100e10 msgr2=0x7f719c1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71a15ff700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f719c100e10 0x7f719c1980d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7190004760 tx=0x7f7190004790 comp rx=0 tx=0).stop 2023-07-06T20:18:59.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71a15ff700 1 -- 172.21.15.69:0/530060343 shutdown_connections 2023-07-06T20:18:59.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71a15ff700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71840a11d0 0x7f71840a3680 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71a15ff700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f719c1017a0 0x7f719c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71a15ff700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f719c0ffc20 0x7f719c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71a15ff700 1 --2- 172.21.15.69:0/530060343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f719c100e10 0x7f719c1980d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71a15ff700 1 -- 172.21.15.69:0/530060343 >> 172.21.15.69:0/530060343 conn(0x7f719c0fb260 msgr2=0x7f719c0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71a15ff700 1 -- 172.21.15.69:0/530060343 shutdown_connections 2023-07-06T20:18:59.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.092+0000 7f71a15ff700 1 -- 172.21.15.69:0/530060343 wait complete. 2023-07-06T20:18:59.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:58 smithi156 ceph-mon[129280]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:59.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:58 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2023-07-06T20:18:59.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:58 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/641733097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:18:59.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:58 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3408637634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:59.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:58 smithi156 ceph-mon[129280]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-06T20:18:59.112 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-06T20:18:59.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:58 smithi098 ceph-mon[128968]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-06T20:18:59.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:58 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2023-07-06T20:18:59.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:58 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/641733097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:18:59.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:58 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3408637634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:18:59.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:58 smithi098 ceph-mon[128968]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-06T20:18:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.307+0000 7fd4d55ff700 1 -- 172.21.15.69:0/3499324148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d00730a0 msgr2=0x7fd4d0073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.307+0000 7fd4d55ff700 1 --2- 172.21.15.69:0/3499324148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d00730a0 0x7fd4d0073510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009990 tx=0x7fd4c4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.307+0000 7fd4d55ff700 1 -- 172.21.15.69:0/3499324148 shutdown_connections 2023-07-06T20:18:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.307+0000 7fd4d55ff700 1 --2- 172.21.15.69:0/3499324148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d0107ae0 0x7fd4d0109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.307+0000 7fd4d55ff700 1 --2- 172.21.15.69:0/3499324148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0074680 0x7fd4d0072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.307+0000 7fd4d55ff700 1 --2- 172.21.15.69:0/3499324148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d00730a0 0x7fd4d0073510 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.307+0000 7fd4d55ff700 1 -- 172.21.15.69:0/3499324148 >> 172.21.15.69:0/3499324148 conn(0x7fd4d00fb260 msgr2=0x7fd4d00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.307+0000 7fd4d55ff700 1 -- 172.21.15.69:0/3499324148 shutdown_connections 2023-07-06T20:18:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.307+0000 7fd4d55ff700 1 -- 172.21.15.69:0/3499324148 wait complete. 2023-07-06T20:18:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.309+0000 7fd4d55ff700 1 Processor -- start 2023-07-06T20:18:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.309+0000 7fd4d55ff700 1 -- start start 2023-07-06T20:18:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.309+0000 7fd4d55ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d00730a0 0x7fd4d0197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.309+0000 7fd4d55ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0074680 0x7fd4d0198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.309+0000 7fd4d55ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d0107ae0 0x7fd4d019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.309+0000 7fd4d55ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d019f760 con 0x7fd4d00730a0 2023-07-06T20:18:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.309+0000 7fd4d55ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d019f8a0 con 0x7fd4d0074680 2023-07-06T20:18:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.309+0000 7fd4d55ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d019fa30 con 0x7fd4d0107ae0 2023-07-06T20:18:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4ceffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d00730a0 0x7fd4d0197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4ceffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d00730a0 0x7fd4d0197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57120/0 (socket says 172.21.15.69:57120) 2023-07-06T20:18:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4ceffd700 1 -- 172.21.15.69:0/2726354102 learned_addr learned my addr 172.21.15.69:0/2726354102 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4ce7fc700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0074680 0x7fd4d0198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4cf7fe700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d0107ae0 0x7fd4d019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4ceffd700 1 -- 172.21.15.69:0/2726354102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d0107ae0 msgr2=0x7fd4d019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d0107ae0 0x7fd4d019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4ceffd700 1 -- 172.21.15.69:0/2726354102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0074680 msgr2=0x7fd4d0198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0074680 0x7fd4d0198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4ceffd700 1 -- 172.21.15.69:0/2726354102 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c40095f0 con 0x7fd4d00730a0 2023-07-06T20:18:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d00730a0 0x7fd4d0197be0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000e970 tx=0x7fd4c000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4b7fff700 1 -- 172.21.15.69:0/2726354102 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0009790 con 0x7fd4d00730a0 2023-07-06T20:18:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4d55ff700 1 -- 172.21.15.69:0/2726354102 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d019fd10 con 0x7fd4d00730a0 2023-07-06T20:18:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4b7fff700 1 -- 172.21.15.69:0/2726354102 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4c0004cf0 con 0x7fd4d00730a0 2023-07-06T20:18:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.310+0000 7fd4b7fff700 1 -- 172.21.15.69:0/2726354102 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0005680 con 0x7fd4d00730a0 2023-07-06T20:18:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.311+0000 7fd4d55ff700 1 -- 172.21.15.69:0/2726354102 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d01a01c0 con 0x7fd4d00730a0 2023-07-06T20:18:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.313+0000 7fd4b7fff700 1 -- 172.21.15.69:0/2726354102 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd4c0004750 con 0x7fd4d00730a0 2023-07-06T20:18:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.315+0000 7fd4e8e53b80 1 Processor -- start 2023-07-06T20:18:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.315+0000 7fd4b7fff700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd4b80a1150 0x7fd4b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.315+0000 7fd4e8e53b80 1 -- start start 2023-07-06T20:18:59.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.315+0000 7fd4b7fff700 1 -- 172.21.15.69:0/2726354102 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd4c0014070 con 0x7fd4d00730a0 2023-07-06T20:18:59.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.315+0000 7fd4e8e53b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef459d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.315+0000 7fd4cf7fe700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd4b80a1150 0x7fd4b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.315+0000 7fd4e8e53b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459d820 0x564ef459ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.315+0000 7fd4e8e53b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef459e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.315+0000 7fd4e8e53b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef44a1c60 con 0x564ef459e340 2023-07-06T20:18:59.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.315+0000 7fd4e8e53b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef44f49f0 con 0x564ef459ced0 2023-07-06T20:18:59.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.315+0000 7fd4e8e53b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef44ede80 con 0x564ef459d820 2023-07-06T20:18:59.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef459e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef459e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57122/0 (socket says 172.21.15.69:57122) 2023-07-06T20:18:59.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4cf7fe700 1 -- 172.21.15.69:0/1658924123 learned_addr learned my addr 172.21.15.69:0/1658924123 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/1658924123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef459d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4ce7fc700 1 --2- 172.21.15.69:0/1658924123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459d820 0x564ef459ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4ceffd700 1 -- 172.21.15.69:0/1658924123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459d820 msgr2=0x564ef459ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/1658924123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459d820 0x564ef459ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4ceffd700 1 -- 172.21.15.69:0/1658924123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 msgr2=0x564ef459e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.320 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/1658924123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef459e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4ceffd700 1 -- 172.21.15.69:0/1658924123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ef431bd90 con 0x564ef459ced0 2023-07-06T20:18:59.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4cf7fe700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd4b80a1150 0x7fd4b80a3600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8007750 tx=0x7fd4c8007b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/1658924123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef459d2e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0005950 tx=0x7fd4c000b2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3990d6f0c1dc3303 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.316+0000 7fd4b57fa700 1 -- 172.21.15.69:0/1658924123 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c009e070 con 0x564ef459ced0 2023-07-06T20:18:59.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.317+0000 7fd4b57fa700 1 -- 172.21.15.69:0/1658924123 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4c00a2490 con 0x564ef459ced0 2023-07-06T20:18:59.321 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.317+0000 7fd4b57fa700 1 -- 172.21.15.69:0/1658924123 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c009a2a0 con 0x564ef459ced0 2023-07-06T20:18:59.323 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.317+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1658924123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 msgr2=0x564ef459d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.317+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/1658924123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef459d2e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0005950 tx=0x7fd4c000b2c0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.317+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1658924123 shutdown_connections 2023-07-06T20:18:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.317+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/1658924123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef459e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.317+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/1658924123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef459d2e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.317+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/1658924123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459d820 0x564ef459ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.317+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1658924123 >> 172.21.15.69:0/1658924123 conn(0x564ef45925f0 msgr2=0x564ef45979a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.317+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1658924123 shutdown_connections 2023-07-06T20:18:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.317+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1658924123 wait complete. 2023-07-06T20:18:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.318+0000 7fd4e8e53b80 1 -- start start 2023-07-06T20:18:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.318+0000 7fd4e8e53b80 1 Processor -- start 2023-07-06T20:18:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.318+0000 7fd4e8e53b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef45a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.318+0000 7fd4e8e53b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459d820 0x564ef45a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.318+0000 7fd4e8e53b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef45ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.318+0000 7fd4e8e53b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef448da70 con 0x564ef459e340 2023-07-06T20:18:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.318+0000 7fd4e8e53b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef448d770 con 0x564ef459ced0 2023-07-06T20:18:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.318+0000 7fd4e8e53b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef43718d0 con 0x564ef459d820 2023-07-06T20:18:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4e8e53b80 10 client.0 _ll_register_callbacks cb 0x564ef448b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:18:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef45ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4ceffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef45a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef45ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57130/0 (socket says 172.21.15.69:57130) 2023-07-06T20:18:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4cf7fe700 1 -- 172.21.15.69:0/1933754723 learned_addr learned my addr 172.21.15.69:0/1933754723 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4ce7fc700 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459d820 0x564ef45a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4cf7fe700 1 -- 172.21.15.69:0/1933754723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459d820 msgr2=0x564ef45a6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4cf7fe700 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459d820 0x564ef45a6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4cf7fe700 1 -- 172.21.15.69:0/1933754723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 msgr2=0x564ef45a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4cf7fe700 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef45a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4cf7fe700 1 -- 172.21.15.69:0/1933754723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d01a01c0 con 0x564ef459e340 2023-07-06T20:18:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef45a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4cf7fe700 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef45ade40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd4c801aba0 tx=0x7fd4c8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e603c25c6c678c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.319+0000 7fd4b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:18:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.320+0000 7fd4b2ff5700 1 -- 172.21.15.69:0/1933754723 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c800e050 con 0x564ef459e340 2023-07-06T20:18:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.320+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1933754723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ef44474a0 con 0x564ef459e340 2023-07-06T20:18:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.320+0000 7fd4b2ff5700 1 -- 172.21.15.69:0/1933754723 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd4c8017d50 con 0x564ef459e340 2023-07-06T20:18:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.320+0000 7fd4b2ff5700 1 -- 172.21.15.69:0/1933754723 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4c8024470 con 0x564ef459e340 2023-07-06T20:18:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.320+0000 7fd4e8e53b80 10 client.15108 fetch_fsmap learned FSMap version 23 2023-07-06T20:18:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.320+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1933754723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564ef4304c10 con 0x564ef459e340 2023-07-06T20:18:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.320+0000 7fd4b2ff5700 1 -- 172.21.15.69:0/1933754723 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4c8033020 con 0x564ef459e340 2023-07-06T20:18:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.320+0000 7fd4b2ff5700 1 -- 172.21.15.69:0/1933754723 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd4c802d500 con 0x564ef459e340 2023-07-06T20:18:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.320+0000 7fd4e8e53b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:18:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.320+0000 7fd4e8e53b80 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-06T20:18:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.320+0000 7fd4e8e53b80 20 client.15108 populate_metadata read hostname 'smithi069' 2023-07-06T20:18:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.320+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x564ef45b1b90 0x564ef45b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.321+0000 7fd4e8e53b80 4 client.15108 mds_command: new command op to 24539 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:18:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.321+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1933754723 --> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ef456bd10 con 0x564ef45b1b90 2023-07-06T20:18:59.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.321+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x564ef45b1b90 0x564ef45b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.321+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x564ef45b1b90 0x564ef45b0f40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e78c5409bdaaf4b server_cookie=b1bb1e2ea08b4870 in_seq=0 out_seq=0 2023-07-06T20:18:59.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.321+0000 7fd4b2ff5700 10 client.15108 ms_handle_connect on v2:172.21.15.98:6836/4010267682 2023-07-06T20:18:59.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.322+0000 7fd4b2ff5700 1 -- 172.21.15.69:0/1933754723 <== mds.1 v2:172.21.15.98:6836/4010267682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564ef456bd10 con 0x564ef45b1b90 2023-07-06T20:18:59.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.322+0000 7fd4b2ff5700 10 client.15108 handle_command_reply: tid=0 2023-07-06T20:18:59.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.322+0000 7fd4e8e53b80 1 client.15108 shutdown 2023-07-06T20:18:59.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.322+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1933754723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 msgr2=0x564ef45ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.322+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef45ade40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd4c801aba0 tx=0x7fd4c8004b40 comp rx=0 tx=0).stop 2023-07-06T20:18:59.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.322+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1933754723 shutdown_connections 2023-07-06T20:18:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.322+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x564ef45b1b90 0x564ef45b0f40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.322+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef45ade40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.322+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef45a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.322+0000 7fd4b2ff5700 0 client.15108 ms_handle_reset on v2:172.21.15.98:6836/4010267682 2023-07-06T20:18:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.322+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/1933754723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459d820 0x564ef45a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.322+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1933754723 >> 172.21.15.69:0/1933754723 conn(0x564ef45925f0 msgr2=0x564ef4597060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.323+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1933754723 shutdown_connections 2023-07-06T20:18:59.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.323+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/1933754723 wait complete. 2023-07-06T20:18:59.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.323+0000 7fd4e8e53b80 20 client.15108 trim_cache size 0 max 16384 2023-07-06T20:18:59.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.340+0000 7fd4e8e53b80 1 Processor -- start 2023-07-06T20:18:59.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.340+0000 7fd4e8e53b80 1 -- start start 2023-07-06T20:18:59.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4e8e53b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459ced0 0x564ef459e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4e8e53b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459d820 0x564ef45add60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4e8e53b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef45a4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4e8e53b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef45a6490 con 0x564ef459e340 2023-07-06T20:18:59.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4e8e53b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef45987c0 con 0x564ef459d820 2023-07-06T20:18:59.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4e8e53b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef456bd10 con 0x564ef459ced0 2023-07-06T20:18:59.345 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef45a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.345 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4cf7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef45a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57142/0 (socket says 172.21.15.69:57142) 2023-07-06T20:18:59.345 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4cf7fe700 1 -- 172.21.15.69:0/4214498955 learned_addr learned my addr 172.21.15.69:0/4214498955 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.345 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4ce7fc700 1 --2- 172.21.15.69:0/4214498955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459d820 0x564ef45add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/4214498955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459ced0 0x564ef459e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4cf7fe700 1 -- 172.21.15.69:0/4214498955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459ced0 msgr2=0x564ef459e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4cf7fe700 1 --2- 172.21.15.69:0/4214498955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459ced0 0x564ef459e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4cf7fe700 1 -- 172.21.15.69:0/4214498955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459d820 msgr2=0x564ef45add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4cf7fe700 1 --2- 172.21.15.69:0/4214498955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459d820 0x564ef45add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.341+0000 7fd4cf7fe700 1 -- 172.21.15.69:0/4214498955 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ef4342200 con 0x564ef459e340 2023-07-06T20:18:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.342+0000 7fd4cf7fe700 1 --2- 172.21.15.69:0/4214498955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef45a4830 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009fd0 tx=0x7fd4c8007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd525fd061085be server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.342+0000 7fd4b5ffb700 1 -- 172.21.15.69:0/4214498955 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c800e040 con 0x564ef459e340 2023-07-06T20:18:59.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.342+0000 7fd4b5ffb700 1 -- 172.21.15.69:0/4214498955 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4c8033030 con 0x564ef459e340 2023-07-06T20:18:59.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.342+0000 7fd4b5ffb700 1 -- 172.21.15.69:0/4214498955 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80246b0 con 0x564ef459e340 2023-07-06T20:18:59.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.342+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/4214498955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 msgr2=0x564ef45a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.342+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/4214498955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef45a4830 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009fd0 tx=0x7fd4c8007f10 comp rx=0 tx=0).stop 2023-07-06T20:18:59.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.342+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/4214498955 shutdown_connections 2023-07-06T20:18:59.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.342+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/4214498955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459e340 0x564ef45a4830 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.342+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/4214498955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459ced0 0x564ef459e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.342+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/4214498955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459d820 0x564ef45add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.342+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/4214498955 >> 172.21.15.69:0/4214498955 conn(0x564ef45925f0 msgr2=0x564ef45a54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.343+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/4214498955 shutdown_connections 2023-07-06T20:18:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.343+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/4214498955 wait complete. 2023-07-06T20:18:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.343+0000 7fd4e8e53b80 1 -- start start 2023-07-06T20:18:59.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.343+0000 7fd4e8e53b80 1 Processor -- start 2023-07-06T20:18:59.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.343+0000 7fd4e8e53b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef45aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4e8e53b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459d820 0x564ef45ab520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4e8e53b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459e340 0x564ef4605a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4e8e53b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef45adf50 con 0x564ef459d820 2023-07-06T20:18:59.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4e8e53b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef45ae0c0 con 0x564ef459ced0 2023-07-06T20:18:59.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4e8e53b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ef45ae280 con 0x564ef459e340 2023-07-06T20:18:59.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4e8e53b80 10 client.0 _ll_register_callbacks cb 0x564ef448b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:18:59.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4ce7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459d820 0x564ef45ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4ce7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459d820 0x564ef45ab520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57144/0 (socket says 172.21.15.69:57144) 2023-07-06T20:18:59.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4ce7fc700 1 -- 172.21.15.69:0/3256851968 learned_addr learned my addr 172.21.15.69:0/3256851968 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef45aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4cf7fe700 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459e340 0x564ef4605a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4ce7fc700 1 -- 172.21.15.69:0/3256851968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459e340 msgr2=0x564ef4605a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4ce7fc700 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459e340 0x564ef4605a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4ce7fc700 1 -- 172.21.15.69:0/3256851968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 msgr2=0x564ef45aafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4ce7fc700 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef45aafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4ce7fc700 1 -- 172.21.15.69:0/3256851968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ef4304c10 con 0x564ef459d820 2023-07-06T20:18:59.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4ce7fc700 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459d820 0x564ef45ab520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00743a0 tx=0x7fd4c4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10c3a500f6321a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.344+0000 7fd4b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:18:59.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4b2ff5700 1 -- 172.21.15.69:0/3256851968 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4005680 con 0x564ef459d820 2023-07-06T20:18:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/3256851968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4c00a2490 con 0x564ef459d820 2023-07-06T20:18:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4b2ff5700 1 -- 172.21.15.69:0/3256851968 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd4c4004500 con 0x564ef459d820 2023-07-06T20:18:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4b2ff5700 1 -- 172.21.15.69:0/3256851968 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4c40046c0 con 0x564ef459d820 2023-07-06T20:18:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4e8e53b80 10 client.15114 fetch_fsmap learned FSMap version 23 2023-07-06T20:18:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/3256851968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564ef431a260 con 0x564ef459d820 2023-07-06T20:18:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4b2ff5700 1 -- 172.21.15.69:0/3256851968 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4c4034020 con 0x564ef459d820 2023-07-06T20:18:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4b2ff5700 1 -- 172.21.15.69:0/3256851968 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd4c400f9c0 con 0x564ef459d820 2023-07-06T20:18:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4e8e53b80 10 client.15114 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:18:59.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4e8e53b80 10 client.15114 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-06T20:18:59.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4e8e53b80 20 client.15114 populate_metadata read hostname 'smithi069' 2023-07-06T20:18:59.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x564ef460aba0 0x564ef46074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4e8e53b80 4 client.15114 mds_command: new command op to 24539 tid=0[{"prefix": "status"}] 2023-07-06T20:18:59.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.345+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/3256851968 --> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564ef44e6e60 con 0x564ef460aba0 2023-07-06T20:18:59.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.346+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x564ef460aba0 0x564ef46074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.346+0000 7fd4b2ff5700 10 client.15114 ms_handle_connect on v2:172.21.15.98:6836/4010267682 2023-07-06T20:18:59.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.346+0000 7fd4ceffd700 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x564ef460aba0 0x564ef46074f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e96a49e0dca295a server_cookie=13f65f9f582698bc in_seq=0 out_seq=0 2023-07-06T20:18:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4b2ff5700 1 -- 172.21.15.69:0/3256851968 <== mds.1 v2:172.21.15.98:6836/4010267682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564ef44e6e60 con 0x564ef460aba0 2023-07-06T20:18:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4b2ff5700 10 client.15114 handle_command_reply: tid=0 2023-07-06T20:18:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4e8e53b80 1 client.15114 shutdown 2023-07-06T20:18:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/3256851968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459d820 msgr2=0x564ef45ab520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459d820 0x564ef45ab520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00743a0 tx=0x7fd4c4005040 comp rx=0 tx=0).stop 2023-07-06T20:18:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/3256851968 shutdown_connections 2023-07-06T20:18:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564ef459e340 0x564ef4605a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ef459ced0 0x564ef45aafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x564ef460aba0 0x564ef46074f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4e8e53b80 1 --2- 172.21.15.69:0/3256851968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564ef459d820 0x564ef45ab520 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4b2ff5700 0 client.15114 ms_handle_reset on v2:172.21.15.98:6836/4010267682 2023-07-06T20:18:59.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/3256851968 >> 172.21.15.69:0/3256851968 conn(0x564ef45925f0 msgr2=0x564ef45a4830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/3256851968 shutdown_connections 2023-07-06T20:18:59.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4e8e53b80 1 -- 172.21.15.69:0/3256851968 wait complete. 2023-07-06T20:18:59.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.347+0000 7fd4e8e53b80 20 client.15114 trim_cache size 0 max 16384 2023-07-06T20:18:59.358 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:18:59.358 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "e3e8ec0c-1c38-11ee-9b2f-001a4aab830c", 2023-07-06T20:18:59.358 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2023-07-06T20:18:59.358 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24539, 2023-07-06T20:18:59.358 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2023-07-06T20:18:59.358 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2023-07-06T20:18:59.359 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2023-07-06T20:18:59.359 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 87.737740185000007, 2023-07-06T20:18:59.359 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 23, 2023-07-06T20:18:59.359 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 86, 2023-07-06T20:18:59.359 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 83, 2023-07-06T20:18:59.359 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 107.704240434 2023-07-06T20:18:59.359 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:18:59.360 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.348+0000 7fd4d55ff700 1 -- 172.21.15.69:0/2726354102 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd4b80a1150 msgr2=0x7fd4b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.360 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.348+0000 7fd4d55ff700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd4b80a1150 0x7fd4b80a3600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8007750 tx=0x7fd4c8007b10 comp rx=0 tx=0).stop 2023-07-06T20:18:59.360 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.348+0000 7fd4d55ff700 1 -- 172.21.15.69:0/2726354102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d00730a0 msgr2=0x7fd4d0197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.360 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.348+0000 7fd4d55ff700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d00730a0 0x7fd4d0197be0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000e970 tx=0x7fd4c000ed30 comp rx=0 tx=0).stop 2023-07-06T20:18:59.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.348+0000 7fd4d55ff700 1 -- 172.21.15.69:0/2726354102 shutdown_connections 2023-07-06T20:18:59.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.348+0000 7fd4d55ff700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd4b80a1150 0x7fd4b80a3600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.348+0000 7fd4d55ff700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d0107ae0 0x7fd4d019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.348+0000 7fd4d55ff700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4d00730a0 0x7fd4d0197be0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.348+0000 7fd4d55ff700 1 --2- 172.21.15.69:0/2726354102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0074680 0x7fd4d0198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.348+0000 7fd4d55ff700 1 -- 172.21.15.69:0/2726354102 >> 172.21.15.69:0/2726354102 conn(0x7fd4d00fb260 msgr2=0x7fd4d00fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.348+0000 7fd4d55ff700 1 -- 172.21.15.69:0/2726354102 shutdown_connections 2023-07-06T20:18:59.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.348+0000 7fd4d55ff700 1 -- 172.21.15.69:0/2726354102 wait complete. 2023-07-06T20:18:59.368 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-06T20:18:59.568 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.565+0000 7f9f58bff700 1 -- 172.21.15.69:0/3969418437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f54074620 msgr2=0x7f9f54072a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.568 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.565+0000 7f9f58bff700 1 --2- 172.21.15.69:0/3969418437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f54074620 0x7f9f54072a70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009990 tx=0x7f9f44009ca0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.568 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.566+0000 7f9f58bff700 1 -- 172.21.15.69:0/3969418437 shutdown_connections 2023-07-06T20:18:59.568 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.566+0000 7f9f58bff700 1 --2- 172.21.15.69:0/3969418437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54107ab0 0x7f9f54109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.566+0000 7f9f58bff700 1 --2- 172.21.15.69:0/3969418437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f54074620 0x7f9f54072a70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.566+0000 7f9f58bff700 1 --2- 172.21.15.69:0/3969418437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54073040 0x7f9f540734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.566+0000 7f9f58bff700 1 -- 172.21.15.69:0/3969418437 >> 172.21.15.69:0/3969418437 conn(0x7f9f540fb260 msgr2=0x7f9f540fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.566+0000 7f9f58bff700 1 -- 172.21.15.69:0/3969418437 shutdown_connections 2023-07-06T20:18:59.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.566+0000 7f9f58bff700 1 -- 172.21.15.69:0/3969418437 wait complete. 2023-07-06T20:18:59.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.567+0000 7f9f58bff700 1 Processor -- start 2023-07-06T20:18:59.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.567+0000 7f9f58bff700 1 -- start start 2023-07-06T20:18:59.571 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.567+0000 7f9f58bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54073040 0x7f9f54197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.567+0000 7f9f58bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54074620 0x7f9f541982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.567+0000 7f9f58bff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f54107ab0 0x7f9f5419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.567+0000 7f9f58bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419f8e0 con 0x7f9f54074620 2023-07-06T20:18:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.567+0000 7f9f58bff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419fa50 con 0x7f9f54107ab0 2023-07-06T20:18:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.567+0000 7f9f58bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419fc10 con 0x7f9f54073040 2023-07-06T20:18:59.572 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.567+0000 7f9f51d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54074620 0x7f9f541982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.567+0000 7f9f51d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54074620 0x7f9f541982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57166/0 (socket says 172.21.15.69:57166) 2023-07-06T20:18:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.567+0000 7f9f51d9b700 1 -- 172.21.15.69:0/3472088603 learned_addr learned my addr 172.21.15.69:0/3472088603 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f52d9d700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f54107ab0 0x7f9f5419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f5259c700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54073040 0x7f9f54197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f51d9b700 1 -- 172.21.15.69:0/3472088603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54073040 msgr2=0x7f9f54197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54073040 0x7f9f54197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f51d9b700 1 -- 172.21.15.69:0/3472088603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f54107ab0 msgr2=0x7f9f5419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f54107ab0 0x7f9f5419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f51d9b700 1 -- 172.21.15.69:0/3472088603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f440095f0 con 0x7f9f54074620 2023-07-06T20:18:59.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f5259c700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54073040 0x7f9f54197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:59.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f52d9d700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f54107ab0 0x7f9f5419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:59.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54074620 0x7f9f541982e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800e9a0 tx=0x7f9f4800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f437fe700 1 -- 172.21.15.69:0/3472088603 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48009790 con 0x7f9f54074620 2023-07-06T20:18:59.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f437fe700 1 -- 172.21.15.69:0/3472088603 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f4800cbd0 con 0x7f9f54074620 2023-07-06T20:18:59.575 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f58bff700 1 -- 172.21.15.69:0/3472088603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5419fef0 con 0x7f9f54074620 2023-07-06T20:18:59.575 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f437fe700 1 -- 172.21.15.69:0/3472088603 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48018680 con 0x7f9f54074620 2023-07-06T20:18:59.575 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.568+0000 7f9f58bff700 1 -- 172.21.15.69:0/3472088603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f541a0310 con 0x7f9f54074620 2023-07-06T20:18:59.576 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.572+0000 7f9f437fe700 1 -- 172.21.15.69:0/3472088603 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9f4800cd40 con 0x7f9f54074620 2023-07-06T20:18:59.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.573+0000 7f9f6c52bb80 1 Processor -- start 2023-07-06T20:18:59.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.573+0000 7f9f437fe700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9f3c0a1150 0x7f9f3c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.573+0000 7f9f6c52bb80 1 -- start start 2023-07-06T20:18:59.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.573+0000 7f9f437fe700 1 -- 172.21.15.69:0/3472088603 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f9f48014070 con 0x7f9f54074620 2023-07-06T20:18:59.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.573+0000 7f9f52d9d700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9f3c0a1150 0x7f9f3c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.573+0000 7f9f6c52bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa569fed0 0x55eaa56a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.573+0000 7f9f6c52bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a0820 0x55eaa56a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.573+0000 7f9f6c52bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a1340 0x55eaa56a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.573+0000 7f9f6c52bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa55a4c60 con 0x55eaa56a0820 2023-07-06T20:18:59.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.573+0000 7f9f6c52bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa55f79f0 con 0x55eaa569fed0 2023-07-06T20:18:59.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.573+0000 7f9f6c52bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa55f0e80 con 0x55eaa56a1340 2023-07-06T20:18:59.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f51d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a0820 0x55eaa56a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f5259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa569fed0 0x55eaa56a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f5259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa569fed0 0x55eaa56a02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:42826/0 (socket says 172.21.15.69:42826) 2023-07-06T20:18:59.579 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f5259c700 1 -- 172.21.15.69:0/2122633485 learned_addr learned my addr 172.21.15.69:0/2122633485 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.579 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f52d9d700 1 --2- 172.21.15.69:0/2122633485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a1340 0x55eaa56a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.579 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f52d9d700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9f3c0a1150 0x7f9f3c0a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009d90 tx=0x7f9f4c0077e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.579 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f51d9b700 1 -- 172.21.15.69:0/2122633485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a1340 msgr2=0x55eaa56a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.579 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/2122633485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a1340 0x55eaa56a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.579 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f51d9b700 1 -- 172.21.15.69:0/2122633485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa569fed0 msgr2=0x55eaa56a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.579 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/2122633485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa569fed0 0x55eaa56a02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f51d9b700 1 -- 172.21.15.69:0/2122633485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaa541ed90 con 0x55eaa56a0820 2023-07-06T20:18:59.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/2122633485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a0820 0x55eaa56a0db0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800e970 tx=0x7f9f4809a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2c4d503c5feddd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.574+0000 7f9f40ff9700 1 -- 172.21.15.69:0/2122633485 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480a24f0 con 0x55eaa56a0820 2023-07-06T20:18:59.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.575+0000 7f9f40ff9700 1 -- 172.21.15.69:0/2122633485 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f48098440 con 0x55eaa56a0820 2023-07-06T20:18:59.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.575+0000 7f9f40ff9700 1 -- 172.21.15.69:0/2122633485 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48099000 con 0x55eaa56a0820 2023-07-06T20:18:59.583 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.575+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2122633485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a0820 msgr2=0x55eaa56a0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.583 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.575+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/2122633485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a0820 0x55eaa56a0db0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800e970 tx=0x7f9f4809a0a0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.583 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.575+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2122633485 shutdown_connections 2023-07-06T20:18:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.575+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/2122633485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a1340 0x55eaa56a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.575+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/2122633485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa569fed0 0x55eaa56a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.575+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/2122633485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a0820 0x55eaa56a0db0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.575+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2122633485 >> 172.21.15.69:0/2122633485 conn(0x55eaa56955f0 msgr2=0x55eaa569a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.575+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2122633485 shutdown_connections 2023-07-06T20:18:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.575+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2122633485 wait complete. 2023-07-06T20:18:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.576+0000 7f9f6c52bb80 1 -- start start 2023-07-06T20:18:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.576+0000 7f9f6c52bb80 1 Processor -- start 2023-07-06T20:18:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.577+0000 7f9f6c52bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa569fed0 0x55eaa56a9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.577+0000 7f9f6c52bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a0820 0x55eaa56a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.577+0000 7f9f6c52bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 0x55eaa56b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.577+0000 7f9f6c52bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa5590a70 con 0x55eaa56a1340 2023-07-06T20:18:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.577+0000 7f9f6c52bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa5590770 con 0x55eaa569fed0 2023-07-06T20:18:59.586 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.577+0000 7f9f6c52bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa54748d0 con 0x55eaa56a0820 2023-07-06T20:18:59.586 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.577+0000 7f9f6c52bb80 10 client.0 _ll_register_callbacks cb 0x55eaa558e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:18:59.586 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.577+0000 7f9f52d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 0x55eaa56b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.586 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.577+0000 7f9f52d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 0x55eaa56b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57178/0 (socket says 172.21.15.69:57178) 2023-07-06T20:18:59.586 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.577+0000 7f9f52d9d700 1 -- 172.21.15.69:0/2107917397 learned_addr learned my addr 172.21.15.69:0/2107917397 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.586 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.577+0000 7f9f5259c700 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa569fed0 0x55eaa56a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.586 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.577+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a0820 0x55eaa56a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f52d9d700 1 -- 172.21.15.69:0/2107917397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a0820 msgr2=0x55eaa56a9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f52d9d700 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a0820 0x55eaa56a9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f52d9d700 1 -- 172.21.15.69:0/2107917397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa569fed0 msgr2=0x55eaa56a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f52d9d700 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa569fed0 0x55eaa56a9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f52d9d700 1 -- 172.21.15.69:0/2107917397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f541a0310 con 0x55eaa56a1340 2023-07-06T20:18:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f52d9d700 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 0x55eaa56b0e00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c019de0 tx=0x7f9f4c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2778e6ee4750f67 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:18:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f39ffb700 1 -- 172.21.15.69:0/2107917397 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c019e10 con 0x55eaa56a1340 2023-07-06T20:18:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f39ffb700 1 -- 172.21.15.69:0/2107917397 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f9f4c01e040 con 0x55eaa56a1340 2023-07-06T20:18:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2107917397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eaa554a4a0 con 0x55eaa56a1340 2023-07-06T20:18:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f39ffb700 1 -- 172.21.15.69:0/2107917397 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4c016d50 con 0x55eaa56a1340 2023-07-06T20:18:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f6c52bb80 10 client.15126 fetch_fsmap learned FSMap version 23 2023-07-06T20:18:59.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.578+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2107917397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55eaa5407c10 con 0x55eaa56a1340 2023-07-06T20:18:59.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.579+0000 7f9f39ffb700 1 -- 172.21.15.69:0/2107917397 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f4c031020 con 0x55eaa56a1340 2023-07-06T20:18:59.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.579+0000 7f9f39ffb700 1 -- 172.21.15.69:0/2107917397 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9f4c0184b0 con 0x55eaa56a1340 2023-07-06T20:18:59.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.579+0000 7f9f6c52bb80 10 client.15126 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:18:59.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.579+0000 7f9f6c52bb80 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-06T20:18:59.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.579+0000 7f9f6c52bb80 20 client.15126 populate_metadata read hostname 'smithi069' 2023-07-06T20:18:59.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.579+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55eaa56b4c00 0x55eaa56b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.579+0000 7f9f6c52bb80 4 client.15126 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:18:59.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.579+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2107917397 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eaa566ed10 con 0x55eaa56b4c00 2023-07-06T20:18:59.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.579+0000 7f9f5259c700 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55eaa56b4c00 0x55eaa56b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.580+0000 7f9f5259c700 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55eaa56b4c00 0x55eaa56b3fb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac44979a656aa00e server_cookie=ffabde075a4fdd0f in_seq=0 out_seq=0 2023-07-06T20:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.580+0000 7f9f39ffb700 10 client.15126 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.580+0000 7f9f39ffb700 1 -- 172.21.15.69:0/2107917397 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eaa566ed10 con 0x55eaa56b4c00 2023-07-06T20:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.580+0000 7f9f39ffb700 10 client.15126 handle_command_reply: tid=0 2023-07-06T20:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.580+0000 7f9f6c52bb80 1 client.15126 shutdown 2023-07-06T20:18:59.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.581+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2107917397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 msgr2=0x55eaa56b0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.581+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 0x55eaa56b0e00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c019de0 tx=0x7f9f4c004b40 comp rx=0 tx=0).stop 2023-07-06T20:18:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.581+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2107917397 shutdown_connections 2023-07-06T20:18:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.581+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55eaa56b4c00 0x55eaa56b3fb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.581+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 0x55eaa56b0e00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.581+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa569fed0 0x55eaa56a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.581+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/2107917397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a0820 0x55eaa56a9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.581+0000 7f9f39ffb700 0 client.15126 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:18:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.581+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2107917397 >> 172.21.15.69:0/2107917397 conn(0x55eaa56955f0 msgr2=0x55eaa569a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.581+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2107917397 shutdown_connections 2023-07-06T20:18:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.581+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/2107917397 wait complete. 2023-07-06T20:18:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.581+0000 7f9f6c52bb80 20 client.15126 trim_cache size 0 max 16384 2023-07-06T20:18:59.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.599+0000 7f9f6c52bb80 1 Processor -- start 2023-07-06T20:18:59.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.599+0000 7f9f6c52bb80 1 -- start start 2023-07-06T20:18:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.599+0000 7f9f6c52bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa569fed0 0x55eaa56a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.599+0000 7f9f6c52bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a0820 0x55eaa56b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.599+0000 7f9f6c52bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 0x55eaa569a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.599+0000 7f9f6c52bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa5480de0 con 0x55eaa56a1340 2023-07-06T20:18:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.599+0000 7f9f6c52bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa55af480 con 0x55eaa56a0820 2023-07-06T20:18:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.599+0000 7f9f6c52bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa558cd50 con 0x55eaa569fed0 2023-07-06T20:18:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.600+0000 7f9f52d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 0x55eaa569a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.600+0000 7f9f51d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a0820 0x55eaa56b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.600+0000 7f9f51d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a0820 0x55eaa56b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:42848/0 (socket says 172.21.15.69:42848) 2023-07-06T20:18:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.600+0000 7f9f51d9b700 1 -- 172.21.15.69:0/734925302 learned_addr learned my addr 172.21.15.69:0/734925302 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.600+0000 7f9f5259c700 1 --2- 172.21.15.69:0/734925302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa569fed0 0x55eaa56a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.600+0000 7f9f51d9b700 1 -- 172.21.15.69:0/734925302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa569fed0 msgr2=0x55eaa56a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.600+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/734925302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa569fed0 0x55eaa56a77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.600+0000 7f9f51d9b700 1 -- 172.21.15.69:0/734925302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 msgr2=0x55eaa569a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.600+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/734925302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 0x55eaa569a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.600+0000 7f9f51d9b700 1 -- 172.21.15.69:0/734925302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaa56a1c30 con 0x55eaa56a0820 2023-07-06T20:18:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.600+0000 7f9f52d9d700 1 --2- 172.21.15.69:0/734925302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 0x55eaa569a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:18:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.600+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/734925302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a0820 0x55eaa56b0d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f480ca7e0 tx=0x7f9f48099be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=955af76cb8eab141 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.601+0000 7f9f417fa700 1 -- 172.21.15.69:0/734925302 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480a24f0 con 0x55eaa56a0820 2023-07-06T20:18:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.601+0000 7f9f417fa700 1 -- 172.21.15.69:0/734925302 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f480b8270 con 0x55eaa56a0820 2023-07-06T20:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.601+0000 7f9f417fa700 1 -- 172.21.15.69:0/734925302 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480997b0 con 0x55eaa56a0820 2023-07-06T20:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.601+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/734925302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a0820 msgr2=0x55eaa56b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.601+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/734925302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a0820 0x55eaa56b0d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f480ca7e0 tx=0x7f9f48099be0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.601+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/734925302 shutdown_connections 2023-07-06T20:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.601+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/734925302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa56a1340 0x55eaa569a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.601+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/734925302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa569fed0 0x55eaa56a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.601+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/734925302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a0820 0x55eaa56b0d20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.601+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/734925302 >> 172.21.15.69:0/734925302 conn(0x55eaa56955f0 msgr2=0x55eaa56a84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.601+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/734925302 shutdown_connections 2023-07-06T20:18:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.601+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/734925302 wait complete. 2023-07-06T20:18:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.602+0000 7f9f6c52bb80 1 -- start start 2023-07-06T20:18:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.602+0000 7f9f6c52bb80 1 Processor -- start 2023-07-06T20:18:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.602+0000 7f9f6c52bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa569fed0 0x55eaa56adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.602+0000 7f9f6c52bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a0820 0x55eaa56ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.602+0000 7f9f6c52bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a1340 0x55eaa56af310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.602+0000 7f9f6c52bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa56af850 con 0x55eaa569fed0 2023-07-06T20:18:59.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.602+0000 7f9f6c52bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa56af9c0 con 0x55eaa56a1340 2023-07-06T20:18:59.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.602+0000 7f9f6c52bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa56afb80 con 0x55eaa56a0820 2023-07-06T20:18:59.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.602+0000 7f9f6c52bb80 10 client.0 _ll_register_callbacks cb 0x55eaa558e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:18:59.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.602+0000 7f9f5259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa569fed0 0x55eaa56adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f5259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa569fed0 0x55eaa56adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57192/0 (socket says 172.21.15.69:57192) 2023-07-06T20:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f5259c700 1 -- 172.21.15.69:0/61241053 learned_addr learned my addr 172.21.15.69:0/61241053 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a0820 0x55eaa56ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f52d9d700 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a1340 0x55eaa56af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f5259c700 1 -- 172.21.15.69:0/61241053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a0820 msgr2=0x55eaa56ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f5259c700 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a0820 0x55eaa56ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f5259c700 1 -- 172.21.15.69:0/61241053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a1340 msgr2=0x55eaa56af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:18:59.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f5259c700 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a1340 0x55eaa56af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f5259c700 1 -- 172.21.15.69:0/61241053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaa5407c10 con 0x55eaa569fed0 2023-07-06T20:18:59.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f52d9d700 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a1340 0x55eaa56af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:18:59.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f5259c700 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa569fed0 0x55eaa56adfa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f440095c0 tx=0x7f9f4400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62efcd4c4f60e9b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:18:59.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:18:59.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f39ffb700 1 -- 172.21.15.69:0/61241053 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4402a070 con 0x55eaa569fed0 2023-07-06T20:18:59.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f39ffb700 1 -- 172.21.15.69:0/61241053 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f9f440053c0 con 0x55eaa569fed0 2023-07-06T20:18:59.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.603+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/61241053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f48098440 con 0x55eaa569fed0 2023-07-06T20:18:59.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.604+0000 7f9f39ffb700 1 -- 172.21.15.69:0/61241053 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f44005580 con 0x55eaa569fed0 2023-07-06T20:18:59.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.604+0000 7f9f6c52bb80 10 client.15132 fetch_fsmap learned FSMap version 23 2023-07-06T20:18:59.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.604+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/61241053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55eaa541ed90 con 0x55eaa569fed0 2023-07-06T20:18:59.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.604+0000 7f9f39ffb700 1 -- 172.21.15.69:0/61241053 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f48098440 con 0x55eaa569fed0 2023-07-06T20:18:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.604+0000 7f9f39ffb700 1 -- 172.21.15.69:0/61241053 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9f4400f4d0 con 0x55eaa569fed0 2023-07-06T20:18:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.604+0000 7f9f6c52bb80 10 client.15132 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:18:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.604+0000 7f9f6c52bb80 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-06T20:18:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.604+0000 7f9f6c52bb80 20 client.15132 populate_metadata read hostname 'smithi069' 2023-07-06T20:18:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.604+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55eaa570e0f0 0x55eaa570aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:18:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.604+0000 7f9f6c52bb80 4 client.15132 mds_command: new command op to 24584 tid=0[{"prefix": "status"}] 2023-07-06T20:18:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.604+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/61241053 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55eaa55e9e60 con 0x55eaa570e0f0 2023-07-06T20:18:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.604+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55eaa570e0f0 0x55eaa570aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:18:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f51d9b700 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55eaa570e0f0 0x55eaa570aa90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e11e0289edfcd14b server_cookie=ce5abc8194b42617 in_seq=0 out_seq=0 2023-07-06T20:18:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f39ffb700 10 client.15132 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:18:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f39ffb700 1 -- 172.21.15.69:0/61241053 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55eaa55e9e60 con 0x55eaa570e0f0 2023-07-06T20:18:59.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f39ffb700 10 client.15132 handle_command_reply: tid=0 2023-07-06T20:18:59.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f6c52bb80 1 client.15132 shutdown 2023-07-06T20:18:59.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/61241053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa569fed0 msgr2=0x55eaa56adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa569fed0 0x55eaa56adfa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f440095c0 tx=0x7f9f4400bf50 comp rx=0 tx=0).stop 2023-07-06T20:18:59.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/61241053 shutdown_connections 2023-07-06T20:18:59.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55eaa570e0f0 0x55eaa570aa90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eaa56a1340 0x55eaa56af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eaa569fed0 0x55eaa56adfa0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f39ffb700 0 client.15132 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:18:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f6c52bb80 1 --2- 172.21.15.69:0/61241053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa56a0820 0x55eaa56ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.605+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/61241053 >> 172.21.15.69:0/61241053 conn(0x55eaa56955f0 msgr2=0x55eaa56a77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/61241053 shutdown_connections 2023-07-06T20:18:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f6c52bb80 1 -- 172.21.15.69:0/61241053 wait complete. 2023-07-06T20:18:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f6c52bb80 20 client.15132 trim_cache size 0 max 16384 2023-07-06T20:18:59.619 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:18:59.619 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "e3e8ec0c-1c38-11ee-9b2f-001a4aab830c", 2023-07-06T20:18:59.619 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2023-07-06T20:18:59.619 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24584, 2023-07-06T20:18:59.619 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2023-07-06T20:18:59.620 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2023-07-06T20:18:59.620 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2023-07-06T20:18:59.620 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 91.020150865000005, 2023-07-06T20:18:59.620 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 23, 2023-07-06T20:18:59.620 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 86, 2023-07-06T20:18:59.620 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 83, 2023-07-06T20:18:59.620 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 97.232062963999994 2023-07-06T20:18:59.621 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:18:59.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f58bff700 1 -- 172.21.15.69:0/3472088603 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9f3c0a1150 msgr2=0x7f9f3c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f58bff700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9f3c0a1150 0x7f9f3c0a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009d90 tx=0x7f9f4c0077e0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f58bff700 1 -- 172.21.15.69:0/3472088603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54074620 msgr2=0x7f9f541982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:18:59.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f58bff700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54074620 0x7f9f541982e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800e9a0 tx=0x7f9f4800ed60 comp rx=0 tx=0).stop 2023-07-06T20:18:59.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f58bff700 1 -- 172.21.15.69:0/3472088603 shutdown_connections 2023-07-06T20:18:59.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f58bff700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9f3c0a1150 0x7f9f3c0a3600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f58bff700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f54107ab0 0x7f9f5419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f58bff700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54073040 0x7f9f54197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f58bff700 1 --2- 172.21.15.69:0/3472088603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54074620 0x7f9f541982e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:18:59.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.606+0000 7f9f58bff700 1 -- 172.21.15.69:0/3472088603 >> 172.21.15.69:0/3472088603 conn(0x7f9f540fb260 msgr2=0x7f9f54109d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:18:59.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.607+0000 7f9f58bff700 1 -- 172.21.15.69:0/3472088603 shutdown_connections 2023-07-06T20:18:59.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:18:59.607+0000 7f9f58bff700 1 -- 172.21.15.69:0/3472088603 wait complete. 2023-07-06T20:18:59.627 INFO:teuthology.run_tasks:Running task kclient... 2023-07-06T20:18:59.651 INFO:tasks.kclient:Mounting kernel clients... 2023-07-06T20:18:59.651 INFO:tasks.kclient:config is {'client.0': {}} 2023-07-06T20:18:59.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:18:59.652 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2023-07-06T20:18:59.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:18:59.696 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2023-07-06T20:18:59.805 INFO:teuthology.orchestra.run.smithi069.stderr:Cannot find device "ceph-brx" 2023-07-06T20:18:59.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:18:59.809 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2023-07-06T20:18:59.809 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T20:18:59.809 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-07-06T20:18:59.809 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-07-06T20:18:59.810 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph fs ls 2023-07-06T20:19:00.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:59 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2023-07-06T20:19:00.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:59 smithi069 ceph-mon[126399]: osdmap e86: 12 total, 12 up, 12 in 2023-07-06T20:19:00.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:18:59 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3716287623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:00.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:59 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2023-07-06T20:19:00.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:59 smithi156 ceph-mon[129280]: osdmap e86: 12 total, 12 up, 12 in 2023-07-06T20:19:00.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:18:59 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3716287623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:00.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:59 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2023-07-06T20:19:00.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:59 smithi098 ceph-mon[128968]: osdmap e86: 12 total, 12 up, 12 in 2023-07-06T20:19:00.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:18:59 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3716287623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:00.525 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:19:00.962 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:00 smithi069 ceph-mon[126399]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-06T20:19:00.963 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:00 smithi069 ceph-mon[126399]: osdmap e87: 12 total, 12 up, 12 in 2023-07-06T20:19:01.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:00 smithi156 ceph-mon[129280]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-06T20:19:01.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:00 smithi156 ceph-mon[129280]: osdmap e87: 12 total, 12 up, 12 in 2023-07-06T20:19:01.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:00 smithi098 ceph-mon[128968]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-06T20:19:01.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:00 smithi098 ceph-mon[128968]: osdmap e87: 12 total, 12 up, 12 in 2023-07-06T20:19:02.016 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.014+0000 7f083a31d700 1 -- 172.21.15.69:0/267185738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0834104fd0 msgr2=0x7f083410cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:02.016 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.014+0000 7f083a31d700 1 --2- 172.21.15.69:0/267185738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0834104fd0 0x7f083410cb50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f082c00b0d0 tx=0x7f082c00b3e0 comp rx=0 tx=0).stop 2023-07-06T20:19:02.017 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.014+0000 7f083a31d700 1 -- 172.21.15.69:0/267185738 shutdown_connections 2023-07-06T20:19:02.017 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.014+0000 7f083a31d700 1 --2- 172.21.15.69:0/267185738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0834104fd0 0x7f083410cb50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:02.017 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.014+0000 7f083a31d700 1 --2- 172.21.15.69:0/267185738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08340fff30 0x7f0834100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:02.017 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.014+0000 7f083a31d700 1 --2- 172.21.15.69:0/267185738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08341008d0 0x7f08341048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:02.017 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.014+0000 7f083a31d700 1 -- 172.21.15.69:0/267185738 >> 172.21.15.69:0/267185738 conn(0x7f08340fb790 msgr2=0x7f08340fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:02.017 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.014+0000 7f083a31d700 1 -- 172.21.15.69:0/267185738 shutdown_connections 2023-07-06T20:19:02.017 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.014+0000 7f083a31d700 1 -- 172.21.15.69:0/267185738 wait complete. 2023-07-06T20:19:02.018 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.015+0000 7f083a31d700 1 Processor -- start 2023-07-06T20:19:02.019 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.015+0000 7f083a31d700 1 -- start start 2023-07-06T20:19:02.019 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.015+0000 7f083a31d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08340fff30 0x7f0834195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:02.019 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.015+0000 7f083a31d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08341008d0 0x7f08341963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:02.019 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.015+0000 7f083a31d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0834104fd0 0x7f083419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:02.019 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.015+0000 7f083a31d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083419ae50 con 0x7f08340fff30 2023-07-06T20:19:02.019 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.015+0000 7f083a31d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083419afc0 con 0x7f0834104fd0 2023-07-06T20:19:02.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.015+0000 7f083a31d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083419b180 con 0x7f08341008d0 2023-07-06T20:19:02.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.015+0000 7f0833fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08340fff30 0x7f0834195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:02.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f0833fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08340fff30 0x7f0834195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57208/0 (socket says 172.21.15.69:57208) 2023-07-06T20:19:02.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f0833fff700 1 -- 172.21.15.69:0/1866981745 learned_addr learned my addr 172.21.15.69:0/1866981745 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:02.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f08388ba700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0834104fd0 0x7f083419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:02.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f08337fe700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08341008d0 0x7f08341963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:02.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f0833fff700 1 -- 172.21.15.69:0/1866981745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08341008d0 msgr2=0x7f08341963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:02.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f0833fff700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08341008d0 0x7f08341963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:02.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f0833fff700 1 -- 172.21.15.69:0/1866981745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0834104fd0 msgr2=0x7f083419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:02.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f0833fff700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0834104fd0 0x7f083419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:02.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f0833fff700 1 -- 172.21.15.69:0/1866981745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c009d00 con 0x7f08340fff30 2023-07-06T20:19:02.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f0833fff700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08340fff30 0x7f0834195e90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f082400e9a0 tx=0x7f082400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:02.021 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f08317fa700 1 -- 172.21.15.69:0/1866981745 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824009790 con 0x7f08340fff30 2023-07-06T20:19:02.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f08317fa700 1 -- 172.21.15.69:0/1866981745 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0824010430 con 0x7f08340fff30 2023-07-06T20:19:02.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f08317fa700 1 -- 172.21.15.69:0/1866981745 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824018680 con 0x7f08340fff30 2023-07-06T20:19:02.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.016+0000 7f083a31d700 1 -- 172.21.15.69:0/1866981745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083419b460 con 0x7f08340fff30 2023-07-06T20:19:02.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.017+0000 7f083a31d700 1 -- 172.21.15.69:0/1866981745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08341a2cc0 con 0x7f08340fff30 2023-07-06T20:19:02.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.018+0000 7f083a31d700 1 -- 172.21.15.69:0/1866981745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083404e340 con 0x7f08340fff30 2023-07-06T20:19:02.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.019+0000 7f08317fa700 1 -- 172.21.15.69:0/1866981745 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f08240105a0 con 0x7f08340fff30 2023-07-06T20:19:02.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.021+0000 7f08317fa700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f081c0a11d0 0x7f081c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:02.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.021+0000 7f08317fa700 1 -- 172.21.15.69:0/1866981745 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f0824014070 con 0x7f08340fff30 2023-07-06T20:19:02.024 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.021+0000 7f08337fe700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f081c0a11d0 0x7f081c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:02.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.022+0000 7f08337fe700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f081c0a11d0 0x7f081c0a3680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f08341974b0 tx=0x7f0828009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:02.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.025+0000 7f08317fa700 1 -- 172.21.15.69:0/1866981745 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0824051770 con 0x7f08340fff30 2023-07-06T20:19:02.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.261+0000 7f083a31d700 1 -- 172.21.15.69:0/1866981745 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f08341a2fb0 con 0x7f08340fff30 2023-07-06T20:19:02.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.262+0000 7f08317fa700 1 -- 172.21.15.69:0/1866981745 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f08240961d0 con 0x7f08340fff30 2023-07-06T20:19:02.264 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T20:19:02.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.264+0000 7f083a31d700 1 -- 172.21.15.69:0/1866981745 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f081c0a11d0 msgr2=0x7f081c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:02.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.264+0000 7f083a31d700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f081c0a11d0 0x7f081c0a3680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f08341974b0 tx=0x7f0828009260 comp rx=0 tx=0).stop 2023-07-06T20:19:02.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.264+0000 7f083a31d700 1 -- 172.21.15.69:0/1866981745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08340fff30 msgr2=0x7f0834195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:02.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.264+0000 7f083a31d700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08340fff30 0x7f0834195e90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f082400e9a0 tx=0x7f082400ecb0 comp rx=0 tx=0).stop 2023-07-06T20:19:02.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.264+0000 7f083a31d700 1 -- 172.21.15.69:0/1866981745 shutdown_connections 2023-07-06T20:19:02.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.264+0000 7f083a31d700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f081c0a11d0 0x7f081c0a3680 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:02.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.264+0000 7f083a31d700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0834104fd0 0x7f083419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:02.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.264+0000 7f083a31d700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08340fff30 0x7f0834195e90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:02.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.264+0000 7f083a31d700 1 --2- 172.21.15.69:0/1866981745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08341008d0 0x7f08341963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:02.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.264+0000 7f083a31d700 1 -- 172.21.15.69:0/1866981745 >> 172.21.15.69:0/1866981745 conn(0x7f08340fb790 msgr2=0x7f08340fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:02.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.264+0000 7f083a31d700 1 -- 172.21.15.69:0/1866981745 shutdown_connections 2023-07-06T20:19:02.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:02.264+0000 7f083a31d700 1 -- 172.21.15.69:0/1866981745 wait complete. 2023-07-06T20:19:02.700 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T20:19:02.700 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T20:19:02.700 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2023-07-06T20:19:02.700 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T20:19:02.701 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-07-06T20:19:02.701 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T20:19:02.701 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T20:19:02.701 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T20:19:02.701 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-07-06T20:19:02.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:02.702 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2023-07-06T20:19:02.749 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T20:19:02.750 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T20:19:02.750 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T20:19:02.750 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:19:02.751 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2023-07-06T20:19:02.751 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:19:02.751 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:19:02.752 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:19:02.752 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2023-07-06T20:19:02.752 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:19:02.752 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:19:02.752 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2023-07-06T20:19:02.752 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T20:19:02.753 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:19:02.753 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T20:19:02.753 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2023-07-06T20:19:02.753 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T20:19:02.753 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 41236sec preferred_lft 41236sec 2023-07-06T20:19:02.753 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::9c26:d5af:e804:49a4/64 scope link noprefixroute 2023-07-06T20:19:02.753 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2023-07-06T20:19:02.754 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-06T20:19:02.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:02.754 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-07-06T20:19:02.755 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-07-06T20:19:02.755 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2023-07-06T20:19:02.755 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2023-07-06T20:19:02.755 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2023-07-06T20:19:02.756 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-06T20:19:02.756 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-07-06T20:19:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:02 smithi069 ceph-mon[126399]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2023-07-06T20:19:03.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:02 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1866981745' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-06T20:19:03.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:02 smithi156 ceph-mon[129280]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2023-07-06T20:19:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:02 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1866981745' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-06T20:19:03.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:02 smithi098 ceph-mon[128968]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2023-07-06T20:19:03.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:02 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1866981745' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-06T20:19:03.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:03.227 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T20:19:03.297 INFO:teuthology.orchestra.run.smithi069.stdout:1 2023-07-06T20:19:03.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:03.301 DEBUG:teuthology.orchestra.run.smithi069:> route 2023-07-06T20:19:03.370 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2023-07-06T20:19:03.370 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T20:19:03.370 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T20:19:03.370 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T20:19:03.371 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T20:19:03.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:03.372 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-07-06T20:19:03.372 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-07-06T20:19:03.373 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T20:19:03.373 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T20:19:03.373 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T20:19:03.373 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-07-06T20:19:04.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:04.050 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2023-07-06T20:19:04.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:04.095 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2023-07-06T20:19:04.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:04.177 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-07-06T20:19:04.178 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-07-06T20:19:04.178 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--tmp-cephfs 2023-07-06T20:19:04.178 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-07-06T20:19:04.178 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-07-06T20:19:04.360 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-07-06T20:19:04.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:04.361 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-07-06T20:19:04.361 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-07-06T20:19:04.361 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-07-06T20:19:04.361 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-06T20:19:04.362 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-07-06T20:19:04.362 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-07-06T20:19:04.362 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-07-06T20:19:04.362 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-07-06T20:19:04.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:04.641 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2023-07-06T20:19:04.641 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2023-07-06T20:19:04.641 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2023-07-06T20:19:04.641 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2023-07-06T20:19:04.642 DEBUG:teuthology.orchestra.run.smithi069:> ') 2023-07-06T20:19:04.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:19:04.821 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /tmp/cephfs 2023-07-06T20:19:04.863 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/tmp/cephfs' 2023-07-06T20:19:04.864 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T20:19:04.864 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /tmp/cephfs 2023-07-06T20:19:04.946 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-07-06T20:19:04.946 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-07-06T20:19:04.946 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/439e7144-7b1d-4988-b061-fb50b19330d0 2023-07-06T20:19:04.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:04.947 DEBUG:teuthology.orchestra.run.smithi069:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/439e7144-7b1d-4988-b061-fb50b19330d0 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2023-07-06T20:19:05.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:04 smithi069 ceph-mon[126399]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 895 B/s wr, 5 op/s; 133 B/s, 0 objects/s recovering 2023-07-06T20:19:05.097 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-07-06T20:19:05.098 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-07-06T20:19:05.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:04 smithi156 ceph-mon[129280]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 895 B/s wr, 5 op/s; 133 B/s, 0 objects/s recovering 2023-07-06T20:19:05.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:04 smithi098 ceph-mon[128968]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 895 B/s wr, 5 op/s; 133 B/s, 0 objects/s recovering 2023-07-06T20:19:05.561 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-07-06T20:19:05.562 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-07-06T20:19:05.562 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: resolved to: "172.21.15.69:3300,172.21.15.98:3300,172.21.15.156:3300" 2023-07-06T20:19:05.562 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: trying mount with new device syntax: 0@e3e8ec0c-1c38-11ee-9b2f-001a4aab830c.cephfs=/volumes/_nogroup/sv_1/439e7144-7b1d-4988-b061-fb50b19330d0 2023-07-06T20:19:05.562 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.69:3300/172.21.15.98:3300/172.21.15.156:3300" will pass to kernel 2023-07-06T20:19:05.568 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T20:19:05.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:05.569 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /tmp/cephfs 2023-07-06T20:19:05.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:05.680 DEBUG:teuthology.orchestra.run.smithi069:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:19:05.776 INFO:teuthology.orchestra.run.smithi069.stdout:client25150 2023-07-06T20:19:05.776 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:19:05.777 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:19:05.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:05.820 DEBUG:teuthology.orchestra.run.smithi069:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:19:05.945 INFO:teuthology.orchestra.run.smithi069.stdout:client25150 2023-07-06T20:19:05.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:05.946 DEBUG:teuthology.orchestra.run.smithi069:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T20:19:06.031 INFO:teuthology.orchestra.run.smithi069.stdout:client25150 2023-07-06T20:19:06.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:19:06.032 DEBUG:teuthology.orchestra.run.smithi069:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c.client25150/status') 2023-07-06T20:19:06.139 INFO:teuthology.orchestra.run.smithi069.stdout:instance: client.25150 (3)172.21.15.69:0/260976338 2023-07-06T20:19:06.139 INFO:teuthology.orchestra.run.smithi069.stdout:blocklisted: false 2023-07-06T20:19:06.140 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records in 2023-07-06T20:19:06.140 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records out 2023-07-06T20:19:06.140 INFO:teuthology.orchestra.run.smithi069.stderr:70 bytes copied, 5.0846e-05 s, 1.4 MB/s 2023-07-06T20:19:06.142 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-06T20:19:06.153 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-06T20:19:06.178 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:19:06.375 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.373+0000 7f8dbdfff700 1 -- 172.21.15.69:0/1509215340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db80ffc30 msgr2=0x7f8db8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:06.375 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.373+0000 7f8dbdfff700 1 --2- 172.21.15.69:0/1509215340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db80ffc30 0x7f8db8100040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009990 tx=0x7f8da8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.373+0000 7f8dbdfff700 1 -- 172.21.15.69:0/1509215340 shutdown_connections 2023-07-06T20:19:06.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.373+0000 7f8dbdfff700 1 --2- 172.21.15.69:0/1509215340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8db81017b0 0x7f8db8108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.373+0000 7f8dbdfff700 1 --2- 172.21.15.69:0/1509215340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db80ffc30 0x7f8db8100040 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.373+0000 7f8dbdfff700 1 --2- 172.21.15.69:0/1509215340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8db8100e20 0x7f8db8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.373+0000 7f8dbdfff700 1 -- 172.21.15.69:0/1509215340 >> 172.21.15.69:0/1509215340 conn(0x7f8db80fb210 msgr2=0x7f8db80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:06.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.373+0000 7f8dbdfff700 1 -- 172.21.15.69:0/1509215340 shutdown_connections 2023-07-06T20:19:06.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.373+0000 7f8dbdfff700 1 -- 172.21.15.69:0/1509215340 wait complete. 2023-07-06T20:19:06.378 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.374+0000 7f8dbdfff700 1 Processor -- start 2023-07-06T20:19:06.378 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.374+0000 7f8dbdfff700 1 -- start start 2023-07-06T20:19:06.378 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8dbdfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8db80ffc30 0x7f8db8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:06.378 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8dbdfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db8100e20 0x7f8db8198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:06.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8dbdfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8db81017b0 0x7f8db819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:06.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8dbdfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db819f680 con 0x7f8db81017b0 2023-07-06T20:19:06.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8dbdfff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db819f7c0 con 0x7f8db80ffc30 2023-07-06T20:19:06.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8dbdfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db819f950 con 0x7f8db8100e20 2023-07-06T20:19:06.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8db6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db8100e20 0x7f8db8198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:06.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8db7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8db81017b0 0x7f8db819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:06.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8db6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db8100e20 0x7f8db8198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:59534/0 (socket says 172.21.15.69:59534) 2023-07-06T20:19:06.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8db6ffd700 1 -- 172.21.15.69:0/3573224053 learned_addr learned my addr 172.21.15.69:0/3573224053 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:06.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8db77fe700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8db80ffc30 0x7f8db8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:06.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8db7fff700 1 -- 172.21.15.69:0/3573224053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db8100e20 msgr2=0x7f8db8198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:06.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8db7fff700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db8100e20 0x7f8db8198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8db7fff700 1 -- 172.21.15.69:0/3573224053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8db80ffc30 msgr2=0x7f8db8197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:06.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8db7fff700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8db80ffc30 0x7f8db8197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8db7fff700 1 -- 172.21.15.69:0/3573224053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac009520 con 0x7f8db81017b0 2023-07-06T20:19:06.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.375+0000 7f8db6ffd700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db8100e20 0x7f8db8198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:06.381 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.376+0000 7f8db7fff700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8db81017b0 0x7f8db819f0b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8db000efc0 tx=0x7f8db000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:06.381 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.376+0000 7f8db4ff9700 1 -- 172.21.15.69:0/3573224053 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0018070 con 0x7f8db81017b0 2023-07-06T20:19:06.381 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.376+0000 7f8db4ff9700 1 -- 172.21.15.69:0/3573224053 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8db000f040 con 0x7f8db81017b0 2023-07-06T20:19:06.381 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.376+0000 7f8dbdfff700 1 -- 172.21.15.69:0/3573224053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da80095f0 con 0x7f8db81017b0 2023-07-06T20:19:06.381 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.376+0000 7f8db4ff9700 1 -- 172.21.15.69:0/3573224053 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db000cd40 con 0x7f8db81017b0 2023-07-06T20:19:06.381 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.376+0000 7f8dbdfff700 1 -- 172.21.15.69:0/3573224053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db819ff50 con 0x7f8db81017b0 2023-07-06T20:19:06.382 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.377+0000 7f8dbdfff700 1 -- 172.21.15.69:0/3573224053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db81922b0 con 0x7f8db81017b0 2023-07-06T20:19:06.382 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.379+0000 7f8db4ff9700 1 -- 172.21.15.69:0/3573224053 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8db0003a00 con 0x7f8db81017b0 2023-07-06T20:19:06.382 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.380+0000 7f8db4ff9700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f8da00a11a0 0x7f8da00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:06.383 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.381+0000 7f8db4ff9700 1 -- 172.21.15.69:0/3573224053 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f8db00cc510 con 0x7f8db81017b0 2023-07-06T20:19:06.383 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.381+0000 7f8db77fe700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f8da00a11a0 0x7f8da00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:06.383 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.381+0000 7f8db77fe700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f8da00a11a0 0x7f8da00a3650 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8da8000c00 tx=0x7f8da800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:06.386 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.384+0000 7f8db4ff9700 1 -- 172.21.15.69:0/3573224053 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8db0095c60 con 0x7f8db81017b0 2023-07-06T20:19:06.584 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.582+0000 7f8dbdfff700 1 -- 172.21.15.69:0/3573224053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8db802c8e0 con 0x7f8db81017b0 2023-07-06T20:19:06.585 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.583+0000 7f8db4ff9700 1 -- 172.21.15.69:0/3573224053 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7f8db0084090 con 0x7f8db81017b0 2023-07-06T20:19:06.586 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:06.586 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":87,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:18:59.638711+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:19:06.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.586+0000 7f8dbdfff700 1 -- 172.21.15.69:0/3573224053 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f8da00a11a0 msgr2=0x7f8da00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:06.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.586+0000 7f8dbdfff700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f8da00a11a0 0x7f8da00a3650 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8da8000c00 tx=0x7f8da800b260 comp rx=0 tx=0).stop 2023-07-06T20:19:06.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.586+0000 7f8dbdfff700 1 -- 172.21.15.69:0/3573224053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8db81017b0 msgr2=0x7f8db819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:06.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.586+0000 7f8dbdfff700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8db81017b0 0x7f8db819f0b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8db000efc0 tx=0x7f8db000c3a0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.586+0000 7f8dbdfff700 1 -- 172.21.15.69:0/3573224053 shutdown_connections 2023-07-06T20:19:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.586+0000 7f8dbdfff700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f8da00a11a0 0x7f8da00a3650 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.586+0000 7f8dbdfff700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8db81017b0 0x7f8db819f0b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.586+0000 7f8dbdfff700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8db80ffc30 0x7f8db8197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.586+0000 7f8dbdfff700 1 --2- 172.21.15.69:0/3573224053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8db8100e20 0x7f8db8198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.586+0000 7f8dbdfff700 1 -- 172.21.15.69:0/3573224053 >> 172.21.15.69:0/3573224053 conn(0x7f8db80fb210 msgr2=0x7f8db8106700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:06.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.586+0000 7f8dbdfff700 1 -- 172.21.15.69:0/3573224053 shutdown_connections 2023-07-06T20:19:06.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.586+0000 7f8dbdfff700 1 -- 172.21.15.69:0/3573224053 wait complete. 2023-07-06T20:19:06.607 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:14.022799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:17:26.523183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:17:27.543386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:19:06.608 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:19:06.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.805+0000 7f5643fff700 1 -- 172.21.15.69:0/3828665650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f563c101790 msgr2=0x7f563c108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:06.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.805+0000 7f5643fff700 1 --2- 172.21.15.69:0/3828665650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f563c101790 0x7f563c108050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f563400b0a0 tx=0x7f563400b3b0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.805+0000 7f5643fff700 1 -- 172.21.15.69:0/3828665650 shutdown_connections 2023-07-06T20:19:06.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.805+0000 7f5643fff700 1 --2- 172.21.15.69:0/3828665650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f563c101790 0x7f563c108050 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.805+0000 7f5643fff700 1 --2- 172.21.15.69:0/3828665650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c0ffc10 0x7f563c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.805+0000 7f5643fff700 1 --2- 172.21.15.69:0/3828665650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f563c100e00 0x7f563c101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.805+0000 7f5643fff700 1 -- 172.21.15.69:0/3828665650 >> 172.21.15.69:0/3828665650 conn(0x7f563c0fb210 msgr2=0x7f563c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:06.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.805+0000 7f5643fff700 1 -- 172.21.15.69:0/3828665650 shutdown_connections 2023-07-06T20:19:06.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.805+0000 7f5643fff700 1 -- 172.21.15.69:0/3828665650 wait complete. 2023-07-06T20:19:06.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.806+0000 7f5643fff700 1 Processor -- start 2023-07-06T20:19:06.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.806+0000 7f5643fff700 1 -- start start 2023-07-06T20:19:06.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5643fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f563c0ffc10 0x7f563c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:06.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5643fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c100e00 0x7f563c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:06.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5643fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f563c101790 0x7f563c19f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:06.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5643fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c19f810 con 0x7f563c0ffc10 2023-07-06T20:19:06.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5643fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c19f980 con 0x7f563c101790 2023-07-06T20:19:06.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5643fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c19fb40 con 0x7f563c100e00 2023-07-06T20:19:06.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5641d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f563c0ffc10 0x7f563c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:06.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5641d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f563c0ffc10 0x7f563c197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58252/0 (socket says 172.21.15.69:58252) 2023-07-06T20:19:06.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f564159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c100e00 0x7f563c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:06.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5641d9b700 1 -- 172.21.15.69:0/1146961725 learned_addr learned my addr 172.21.15.69:0/1146961725 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:06.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f564259c700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f563c101790 0x7f563c19f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:06.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5641d9b700 1 -- 172.21.15.69:0/1146961725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c100e00 msgr2=0x7f563c1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:06.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5641d9b700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c100e00 0x7f563c1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5641d9b700 1 -- 172.21.15.69:0/1146961725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f563c101790 msgr2=0x7f563c19f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:06.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5641d9b700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f563c101790 0x7f563c19f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:06.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f5641d9b700 1 -- 172.21.15.69:0/1146961725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c009520 con 0x7f563c0ffc10 2023-07-06T20:19:06.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.807+0000 7f564159a700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c100e00 0x7f563c1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:06.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.808+0000 7f5641d9b700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f563c0ffc10 0x7f563c197ca0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f563800e970 tx=0x7f563800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:06.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.808+0000 7f5632ffd700 1 -- 172.21.15.69:0/1146961725 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638009790 con 0x7f563c0ffc10 2023-07-06T20:19:06.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.808+0000 7f5632ffd700 1 -- 172.21.15.69:0/1146961725 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f563800c7e0 con 0x7f563c0ffc10 2023-07-06T20:19:06.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.808+0000 7f5643fff700 1 -- 172.21.15.69:0/1146961725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5634009d00 con 0x7f563c0ffc10 2023-07-06T20:19:06.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.808+0000 7f5632ffd700 1 -- 172.21.15.69:0/1146961725 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638003c70 con 0x7f563c0ffc10 2023-07-06T20:19:06.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.808+0000 7f5643fff700 1 -- 172.21.15.69:0/1146961725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c1a01a0 con 0x7f563c0ffc10 2023-07-06T20:19:06.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.809+0000 7f5643fff700 1 -- 172.21.15.69:0/1146961725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c10b440 con 0x7f563c0ffc10 2023-07-06T20:19:06.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.811+0000 7f5632ffd700 1 -- 172.21.15.69:0/1146961725 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f563800cc50 con 0x7f563c0ffc10 2023-07-06T20:19:06.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.813+0000 7f5632ffd700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f56280a1150 0x7f56280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:06.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.813+0000 7f5632ffd700 1 -- 172.21.15.69:0/1146961725 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f5638014070 con 0x7f563c0ffc10 2023-07-06T20:19:06.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.813+0000 7f564159a700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f56280a1150 0x7f56280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:06.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.814+0000 7f564159a700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f56280a1150 0x7f56280a3600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f562c009fd0 tx=0x7f562c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:06.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:06.817+0000 7f5632ffd700 1 -- 172.21.15.69:0/1146961725 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f56380961f0 con 0x7f563c0ffc10 2023-07-06T20:19:07.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:06 smithi069 ceph-mon[126399]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 895 B/s wr, 7 op/s; 133 B/s, 0 objects/s recovering 2023-07-06T20:19:07.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:06 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3573224053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:07.019 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.017+0000 7f5643fff700 1 -- 172.21.15.69:0/1146961725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f563c10b600 con 0x7f563c0ffc10 2023-07-06T20:19:07.020 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.018+0000 7f5632ffd700 1 -- 172.21.15.69:0/1146961725 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f563c10b600 con 0x7f563c0ffc10 2023-07-06T20:19:07.020 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2023-07-06T20:19:07.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.020+0000 7f5643fff700 1 -- 172.21.15.69:0/1146961725 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f56280a1150 msgr2=0x7f56280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:07.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.020+0000 7f5643fff700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f56280a1150 0x7f56280a3600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f562c009fd0 tx=0x7f562c009260 comp rx=0 tx=0).stop 2023-07-06T20:19:07.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.020+0000 7f5643fff700 1 -- 172.21.15.69:0/1146961725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f563c0ffc10 msgr2=0x7f563c197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:07.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.020+0000 7f5643fff700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f563c0ffc10 0x7f563c197ca0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f563800e970 tx=0x7f563800ec80 comp rx=0 tx=0).stop 2023-07-06T20:19:07.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.020+0000 7f5643fff700 1 -- 172.21.15.69:0/1146961725 shutdown_connections 2023-07-06T20:19:07.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.020+0000 7f5643fff700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f56280a1150 0x7f56280a3600 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.020+0000 7f5643fff700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f563c101790 0x7f563c19f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.020+0000 7f5643fff700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f563c0ffc10 0x7f563c197ca0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.020+0000 7f5643fff700 1 --2- 172.21.15.69:0/1146961725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563c100e00 0x7f563c1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.020+0000 7f5643fff700 1 -- 172.21.15.69:0/1146961725 >> 172.21.15.69:0/1146961725 conn(0x7f563c0fb210 msgr2=0x7f563c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:07.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.020+0000 7f5643fff700 1 -- 172.21.15.69:0/1146961725 shutdown_connections 2023-07-06T20:19:07.024 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.020+0000 7f5643fff700 1 -- 172.21.15.69:0/1146961725 wait complete. 2023-07-06T20:19:07.041 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:19:07.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:06 smithi156 ceph-mon[129280]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 895 B/s wr, 7 op/s; 133 B/s, 0 objects/s recovering 2023-07-06T20:19:07.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:06 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3573224053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:07.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:06 smithi098 ceph-mon[128968]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 895 B/s wr, 7 op/s; 133 B/s, 0 objects/s recovering 2023-07-06T20:19:07.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:06 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3573224053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:07.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.236+0000 7f5b1edff700 1 -- 172.21.15.69:0/3768964975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b181017a0 msgr2=0x7f5b18108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:07.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.236+0000 7f5b1edff700 1 --2- 172.21.15.69:0/3768964975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b181017a0 0x7f5b18108060 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000b0a0 tx=0x7f5b1000b3b0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.236+0000 7f5b1edff700 1 -- 172.21.15.69:0/3768964975 shutdown_connections 2023-07-06T20:19:07.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.236+0000 7f5b1edff700 1 --2- 172.21.15.69:0/3768964975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b181017a0 0x7f5b18108060 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.236+0000 7f5b1edff700 1 --2- 172.21.15.69:0/3768964975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b180ffc20 0x7f5b18100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.236+0000 7f5b1edff700 1 --2- 172.21.15.69:0/3768964975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b18100e10 0x7f5b18101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.236+0000 7f5b1edff700 1 -- 172.21.15.69:0/3768964975 >> 172.21.15.69:0/3768964975 conn(0x7f5b180fb260 msgr2=0x7f5b180fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:07.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.236+0000 7f5b1edff700 1 -- 172.21.15.69:0/3768964975 shutdown_connections 2023-07-06T20:19:07.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.236+0000 7f5b1edff700 1 -- 172.21.15.69:0/3768964975 wait complete. 2023-07-06T20:19:07.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.237+0000 7f5b1edff700 1 Processor -- start 2023-07-06T20:19:07.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.237+0000 7f5b1edff700 1 -- start start 2023-07-06T20:19:07.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1edff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b180ffc20 0x7f5b180743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:07.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1edff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b18100e10 0x7f5b18072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:07.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1edff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b181017a0 0x7f5b18072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:07.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1edff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b18073710 con 0x7f5b180ffc20 2023-07-06T20:19:07.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1edff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b18073880 con 0x7f5b181017a0 2023-07-06T20:19:07.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1edff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1819faf0 con 0x7f5b18100e10 2023-07-06T20:19:07.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1cb9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b180ffc20 0x7f5b180743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:07.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1cb9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b180ffc20 0x7f5b180743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58272/0 (socket says 172.21.15.69:58272) 2023-07-06T20:19:07.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1cb9b700 1 -- 172.21.15.69:0/235578177 learned_addr learned my addr 172.21.15.69:0/235578177 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:07.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b17fff700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b18100e10 0x7f5b18072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:07.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1d39c700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b181017a0 0x7f5b18072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:07.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1cb9b700 1 -- 172.21.15.69:0/235578177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b18100e10 msgr2=0x7f5b18072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:07.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1cb9b700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b18100e10 0x7f5b18072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1cb9b700 1 -- 172.21.15.69:0/235578177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b181017a0 msgr2=0x7f5b18072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:07.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1cb9b700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b181017a0 0x7f5b18072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.238+0000 7f5b1cb9b700 1 -- 172.21.15.69:0/235578177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b10009d00 con 0x7f5b180ffc20 2023-07-06T20:19:07.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.239+0000 7f5b1cb9b700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b180ffc20 0x7f5b180743f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e970 tx=0x7f5b0c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:07.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.239+0000 7f5b15ffb700 1 -- 172.21.15.69:0/235578177 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c009790 con 0x7f5b180ffc20 2023-07-06T20:19:07.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.239+0000 7f5b15ffb700 1 -- 172.21.15.69:0/235578177 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b0c00eec0 con 0x7f5b180ffc20 2023-07-06T20:19:07.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.239+0000 7f5b15ffb700 1 -- 172.21.15.69:0/235578177 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c003cf0 con 0x7f5b180ffc20 2023-07-06T20:19:07.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.239+0000 7f5b1edff700 1 -- 172.21.15.69:0/235578177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1819fdd0 con 0x7f5b180ffc20 2023-07-06T20:19:07.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.239+0000 7f5b1edff700 1 -- 172.21.15.69:0/235578177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b181a01d0 con 0x7f5b180ffc20 2023-07-06T20:19:07.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.240+0000 7f5b1edff700 1 -- 172.21.15.69:0/235578177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b18106ab0 con 0x7f5b180ffc20 2023-07-06T20:19:07.246 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.242+0000 7f5b15ffb700 1 -- 172.21.15.69:0/235578177 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5b0c00ccd0 con 0x7f5b180ffc20 2023-07-06T20:19:07.246 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.243+0000 7f5b15ffb700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5b080a1150 0x7f5b080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:07.246 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.243+0000 7f5b15ffb700 1 -- 172.21.15.69:0/235578177 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f5b0c014070 con 0x7f5b180ffc20 2023-07-06T20:19:07.246 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.243+0000 7f5b17fff700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5b080a1150 0x7f5b080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:07.246 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.244+0000 7f5b17fff700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5b080a1150 0x7f5b080a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009c60 tx=0x7f5b04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:07.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.247+0000 7f5b15ffb700 1 -- 172.21.15.69:0/235578177 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5b0c0962a0 con 0x7f5b180ffc20 2023-07-06T20:19:07.448 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.446+0000 7f5b1edff700 1 -- 172.21.15.69:0/235578177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b18060a80 con 0x7f5b180ffc20 2023-07-06T20:19:07.449 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.447+0000 7f5b15ffb700 1 -- 172.21.15.69:0/235578177 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b0c0998c0 con 0x7f5b180ffc20 2023-07-06T20:19:07.450 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2023-07-06T20:19:07.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.449+0000 7f5b1edff700 1 -- 172.21.15.69:0/235578177 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5b080a1150 msgr2=0x7f5b080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:07.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.449+0000 7f5b1edff700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5b080a1150 0x7f5b080a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009c60 tx=0x7f5b04009260 comp rx=0 tx=0).stop 2023-07-06T20:19:07.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.449+0000 7f5b1edff700 1 -- 172.21.15.69:0/235578177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b180ffc20 msgr2=0x7f5b180743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:07.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.449+0000 7f5b1edff700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b180ffc20 0x7f5b180743f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e970 tx=0x7f5b0c00ec80 comp rx=0 tx=0).stop 2023-07-06T20:19:07.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.449+0000 7f5b1edff700 1 -- 172.21.15.69:0/235578177 shutdown_connections 2023-07-06T20:19:07.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.449+0000 7f5b1edff700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5b080a1150 0x7f5b080a3600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.449+0000 7f5b1edff700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b181017a0 0x7f5b18072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.449+0000 7f5b1edff700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b180ffc20 0x7f5b180743f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.449+0000 7f5b1edff700 1 --2- 172.21.15.69:0/235578177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b18100e10 0x7f5b18072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.449+0000 7f5b1edff700 1 -- 172.21.15.69:0/235578177 >> 172.21.15.69:0/235578177 conn(0x7f5b180fb260 msgr2=0x7f5b180fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:07.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.449+0000 7f5b1edff700 1 -- 172.21.15.69:0/235578177 shutdown_connections 2023-07-06T20:19:07.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.449+0000 7f5b1edff700 1 -- 172.21.15.69:0/235578177 wait complete. 2023-07-06T20:19:07.470 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:19:07.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.669+0000 7fca047ff700 1 -- 172.21.15.69:0/3408522549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9fc0ffc20 msgr2=0x7fc9fc100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.669+0000 7fca047ff700 1 --2- 172.21.15.69:0/3408522549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9fc0ffc20 0x7fc9fc100030 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8009990 tx=0x7fc9f8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.669+0000 7fca047ff700 1 -- 172.21.15.69:0/3408522549 shutdown_connections 2023-07-06T20:19:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.669+0000 7fca047ff700 1 --2- 172.21.15.69:0/3408522549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9fc1017a0 0x7fc9fc108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.669+0000 7fca047ff700 1 --2- 172.21.15.69:0/3408522549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9fc0ffc20 0x7fc9fc100030 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.669+0000 7fca047ff700 1 --2- 172.21.15.69:0/3408522549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9fc100e10 0x7fc9fc101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.669+0000 7fca047ff700 1 -- 172.21.15.69:0/3408522549 >> 172.21.15.69:0/3408522549 conn(0x7fc9fc0fb260 msgr2=0x7fc9fc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.669+0000 7fca047ff700 1 -- 172.21.15.69:0/3408522549 shutdown_connections 2023-07-06T20:19:07.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.669+0000 7fca047ff700 1 -- 172.21.15.69:0/3408522549 wait complete. 2023-07-06T20:19:07.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.670+0000 7fca047ff700 1 Processor -- start 2023-07-06T20:19:07.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.670+0000 7fca047ff700 1 -- start start 2023-07-06T20:19:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca047ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9fc0ffc20 0x7fc9fc197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca047ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9fc100e10 0x7fc9fc198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:07.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca047ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9fc1017a0 0x7fc9fc19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:07.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca047ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc19f750 con 0x7fc9fc1017a0 2023-07-06T20:19:07.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca047ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc19f890 con 0x7fc9fc100e10 2023-07-06T20:19:07.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca047ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc19fa20 con 0x7fc9fc0ffc20 2023-07-06T20:19:07.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca02d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9fc1017a0 0x7fc9fc19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:07.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca02d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9fc1017a0 0x7fc9fc19f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58276/0 (socket says 172.21.15.69:58276) 2023-07-06T20:19:07.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca02d9c700 1 -- 172.21.15.69:0/1244176791 learned_addr learned my addr 172.21.15.69:0/1244176791 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:07.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca0259b700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9fc0ffc20 0x7fc9fc197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:07.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca01d9a700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9fc100e10 0x7fc9fc198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:07.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca02d9c700 1 -- 172.21.15.69:0/1244176791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9fc0ffc20 msgr2=0x7fc9fc197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:07.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca02d9c700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9fc0ffc20 0x7fc9fc197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca02d9c700 1 -- 172.21.15.69:0/1244176791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9fc100e10 msgr2=0x7fc9fc198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:07.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca02d9c700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9fc100e10 0x7fc9fc198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.671+0000 7fca02d9c700 1 -- 172.21.15.69:0/1244176791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80095f0 con 0x7fc9fc1017a0 2023-07-06T20:19:07.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.672+0000 7fca02d9c700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9fc1017a0 0x7fc9fc19f180 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400ef30 tx=0x7fc9f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:07.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.672+0000 7fc9f37fe700 1 -- 172.21.15.69:0/1244176791 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4018070 con 0x7fc9fc1017a0 2023-07-06T20:19:07.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.672+0000 7fc9f37fe700 1 -- 172.21.15.69:0/1244176791 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9f400f040 con 0x7fc9fc1017a0 2023-07-06T20:19:07.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.672+0000 7fca047ff700 1 -- 172.21.15.69:0/1244176791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9fc19fca0 con 0x7fc9fc1017a0 2023-07-06T20:19:07.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.672+0000 7fc9f37fe700 1 -- 172.21.15.69:0/1244176791 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f400cd00 con 0x7fc9fc1017a0 2023-07-06T20:19:07.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.672+0000 7fca047ff700 1 -- 172.21.15.69:0/1244176791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9fc1a01b0 con 0x7fc9fc1017a0 2023-07-06T20:19:07.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.673+0000 7fca047ff700 1 -- 172.21.15.69:0/1244176791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9fc192300 con 0x7fc9fc1017a0 2023-07-06T20:19:07.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.675+0000 7fc9f37fe700 1 -- 172.21.15.69:0/1244176791 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc9f4003a00 con 0x7fc9fc1017a0 2023-07-06T20:19:07.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.677+0000 7fc9f37fe700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc9e80a11d0 0x7fc9e80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:07.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.677+0000 7fc9f37fe700 1 -- 172.21.15.69:0/1244176791 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fc9f40cc3a0 con 0x7fc9fc1017a0 2023-07-06T20:19:07.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.678+0000 7fca0259b700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc9e80a11d0 0x7fc9e80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:07.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.678+0000 7fca0259b700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc9e80a11d0 0x7fc9e80a3680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800b2e0 tx=0x7fc9f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:07.684 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.682+0000 7fc9f37fe700 1 -- 172.21.15.69:0/1244176791 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc9f4095b50 con 0x7fc9fc1017a0 2023-07-06T20:19:07.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:07 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1146961725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:19:07.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:07 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/235578177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:19:07.886 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.884+0000 7fca047ff700 1 -- 172.21.15.69:0/1244176791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc9fc0612f0 con 0x7fc9fc1017a0 2023-07-06T20:19:07.887 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.885+0000 7fc9f37fe700 1 -- 172.21.15.69:0/1244176791 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc9f4099170 con 0x7fc9fc1017a0 2023-07-06T20:19:07.887 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2023-07-06T20:19:07.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.887+0000 7fca047ff700 1 -- 172.21.15.69:0/1244176791 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc9e80a11d0 msgr2=0x7fc9e80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:07.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.887+0000 7fca047ff700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc9e80a11d0 0x7fc9e80a3680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800b2e0 tx=0x7fc9f8005fb0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.887+0000 7fca047ff700 1 -- 172.21.15.69:0/1244176791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9fc1017a0 msgr2=0x7fc9fc19f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:07.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.887+0000 7fca047ff700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9fc1017a0 0x7fc9fc19f180 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400ef30 tx=0x7fc9f400c3a0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.887+0000 7fca047ff700 1 -- 172.21.15.69:0/1244176791 shutdown_connections 2023-07-06T20:19:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.887+0000 7fca047ff700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc9e80a11d0 0x7fc9e80a3680 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.887+0000 7fca047ff700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9fc1017a0 0x7fc9fc19f180 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.887+0000 7fca047ff700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9fc0ffc20 0x7fc9fc197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.887+0000 7fca047ff700 1 --2- 172.21.15.69:0/1244176791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9fc100e10 0x7fc9fc198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.887+0000 7fca047ff700 1 -- 172.21.15.69:0/1244176791 >> 172.21.15.69:0/1244176791 conn(0x7fc9fc0fb260 msgr2=0x7fc9fc1066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.887+0000 7fca047ff700 1 -- 172.21.15.69:0/1244176791 shutdown_connections 2023-07-06T20:19:07.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:07.887+0000 7fca047ff700 1 -- 172.21.15.69:0/1244176791 wait complete. 2023-07-06T20:19:07.907 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:19:08.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.101+0000 7f896dfff700 1 -- 172.21.15.69:0/2694159149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89680ffc30 msgr2=0x7f8968100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.101+0000 7f896dfff700 1 --2- 172.21.15.69:0/2694159149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89680ffc30 0x7f8968100040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8958009990 tx=0x7f8958009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.101+0000 7f896dfff700 1 -- 172.21.15.69:0/2694159149 shutdown_connections 2023-07-06T20:19:08.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.101+0000 7f896dfff700 1 --2- 172.21.15.69:0/2694159149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89681017b0 0x7f8968108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.101+0000 7f896dfff700 1 --2- 172.21.15.69:0/2694159149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89680ffc30 0x7f8968100040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.101+0000 7f896dfff700 1 --2- 172.21.15.69:0/2694159149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8968100e20 0x7f8968101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.101+0000 7f896dfff700 1 -- 172.21.15.69:0/2694159149 >> 172.21.15.69:0/2694159149 conn(0x7f89680fb210 msgr2=0x7f89680fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:08.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.101+0000 7f896dfff700 1 -- 172.21.15.69:0/2694159149 shutdown_connections 2023-07-06T20:19:08.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.101+0000 7f896dfff700 1 -- 172.21.15.69:0/2694159149 wait complete. 2023-07-06T20:19:08.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.103+0000 7f896dfff700 1 Processor -- start 2023-07-06T20:19:08.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.103+0000 7f896dfff700 1 -- start start 2023-07-06T20:19:08.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.103+0000 7f896dfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89680ffc30 0x7f89680743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:08.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.103+0000 7f896dfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8968100e20 0x7f8968072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:08.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.103+0000 7f896dfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89681017b0 0x7f8968072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:08.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.103+0000 7f896dfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8968073710 con 0x7f89680ffc30 2023-07-06T20:19:08.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.103+0000 7f896dfff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8968073880 con 0x7f89681017b0 2023-07-06T20:19:08.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.103+0000 7f896dfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896819f9d0 con 0x7f8968100e20 2023-07-06T20:19:08.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f89677fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89680ffc30 0x7f89680743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:08.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f89677fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89680ffc30 0x7f89680743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58288/0 (socket says 172.21.15.69:58288) 2023-07-06T20:19:08.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f89677fe700 1 -- 172.21.15.69:0/2364403502 learned_addr learned my addr 172.21.15.69:0/2364403502 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:08.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f8966ffd700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8968100e20 0x7f8968072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:08.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f8967fff700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89681017b0 0x7f8968072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:08.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f89677fe700 1 -- 172.21.15.69:0/2364403502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8968100e20 msgr2=0x7f8968072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f89677fe700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8968100e20 0x7f8968072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.108 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f89677fe700 1 -- 172.21.15.69:0/2364403502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89681017b0 msgr2=0x7f8968072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f89677fe700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89681017b0 0x7f8968072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f89677fe700 1 -- 172.21.15.69:0/2364403502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89580095f0 con 0x7f89680ffc30 2023-07-06T20:19:08.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f89677fe700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89680ffc30 0x7f89680743f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8958005740 tx=0x7f8958005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:08.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f8964ff9700 1 -- 172.21.15.69:0/2364403502 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895801d070 con 0x7f89680ffc30 2023-07-06T20:19:08.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f8964ff9700 1 -- 172.21.15.69:0/2364403502 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8958021410 con 0x7f89680ffc30 2023-07-06T20:19:08.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.104+0000 7f896dfff700 1 -- 172.21.15.69:0/2364403502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896819fc50 con 0x7f89680ffc30 2023-07-06T20:19:08.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.105+0000 7f8964ff9700 1 -- 172.21.15.69:0/2364403502 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958017650 con 0x7f89680ffc30 2023-07-06T20:19:08.109 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.105+0000 7f896dfff700 1 -- 172.21.15.69:0/2364403502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89681a00d0 con 0x7f89680ffc30 2023-07-06T20:19:08.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:07 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1146961725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:19:08.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:07 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/235578177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:19:08.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.105+0000 7f896dfff700 1 -- 172.21.15.69:0/2364403502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896810b450 con 0x7f89680ffc30 2023-07-06T20:19:08.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.107+0000 7f8964ff9700 1 -- 172.21.15.69:0/2364403502 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8958021580 con 0x7f89680ffc30 2023-07-06T20:19:08.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.109+0000 7f8964ff9700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f89500a1220 0x7f89500a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:08.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.109+0000 7f8964ff9700 1 -- 172.21.15.69:0/2364403502 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f89580cda40 con 0x7f89680ffc30 2023-07-06T20:19:08.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.109+0000 7f8966ffd700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f89500a1220 0x7f89500a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:08.112 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.110+0000 7f8966ffd700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f89500a1220 0x7f89500a36d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f895c009c60 tx=0x7f895c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:08.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.112+0000 7f8964ff9700 1 -- 172.21.15.69:0/2364403502 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8958097190 con 0x7f89680ffc30 2023-07-06T20:19:08.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:07 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1146961725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:19:08.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:07 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/235578177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:19:08.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.311+0000 7f896dfff700 1 -- 172.21.15.69:0/2364403502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8968106ad0 con 0x7f89680ffc30 2023-07-06T20:19:08.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.313+0000 7f8964ff9700 1 -- 172.21.15.69:0/2364403502 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7f895809a7b0 con 0x7f89680ffc30 2023-07-06T20:19:08.315 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:08.315 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":87,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:18:59.638711+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:19:08.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.315+0000 7f896dfff700 1 -- 172.21.15.69:0/2364403502 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f89500a1220 msgr2=0x7f89500a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.315+0000 7f896dfff700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f89500a1220 0x7f89500a36d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f895c009c60 tx=0x7f895c009260 comp rx=0 tx=0).stop 2023-07-06T20:19:08.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.315+0000 7f896dfff700 1 -- 172.21.15.69:0/2364403502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89680ffc30 msgr2=0x7f89680743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.315+0000 7f896dfff700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89680ffc30 0x7f89680743f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8958005740 tx=0x7f8958005770 comp rx=0 tx=0).stop 2023-07-06T20:19:08.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.315+0000 7f896dfff700 1 -- 172.21.15.69:0/2364403502 shutdown_connections 2023-07-06T20:19:08.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.315+0000 7f896dfff700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f89500a1220 0x7f89500a36d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.315+0000 7f896dfff700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89681017b0 0x7f8968072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.315+0000 7f896dfff700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89680ffc30 0x7f89680743f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.315+0000 7f896dfff700 1 --2- 172.21.15.69:0/2364403502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8968100e20 0x7f8968072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.315+0000 7f896dfff700 1 -- 172.21.15.69:0/2364403502 >> 172.21.15.69:0/2364403502 conn(0x7f89680fb210 msgr2=0x7f89680fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:08.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.315+0000 7f896dfff700 1 -- 172.21.15.69:0/2364403502 shutdown_connections 2023-07-06T20:19:08.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.315+0000 7f896dfff700 1 -- 172.21.15.69:0/2364403502 wait complete. 2023-07-06T20:19:08.336 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:14.022799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:17:26.523183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:17:27.543386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:19:08.336 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:19:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.534+0000 7f58525ff700 1 -- 172.21.15.69:0/2734424385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f584c101790 msgr2=0x7f584c108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.534+0000 7f58525ff700 1 --2- 172.21.15.69:0/2734424385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f584c101790 0x7f584c108050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5844009f90 tx=0x7f584400b330 comp rx=0 tx=0).stop 2023-07-06T20:19:08.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.534+0000 7f58525ff700 1 -- 172.21.15.69:0/2734424385 shutdown_connections 2023-07-06T20:19:08.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.534+0000 7f58525ff700 1 --2- 172.21.15.69:0/2734424385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f584c101790 0x7f584c108050 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.534+0000 7f58525ff700 1 --2- 172.21.15.69:0/2734424385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f584c0ffc10 0x7f584c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.534+0000 7f58525ff700 1 --2- 172.21.15.69:0/2734424385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f584c100e00 0x7f584c101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.534+0000 7f58525ff700 1 -- 172.21.15.69:0/2734424385 >> 172.21.15.69:0/2734424385 conn(0x7f584c0fb210 msgr2=0x7f584c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:08.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.534+0000 7f58525ff700 1 -- 172.21.15.69:0/2734424385 shutdown_connections 2023-07-06T20:19:08.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.534+0000 7f58525ff700 1 -- 172.21.15.69:0/2734424385 wait complete. 2023-07-06T20:19:08.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.535+0000 7f58525ff700 1 Processor -- start 2023-07-06T20:19:08.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.535+0000 7f58525ff700 1 -- start start 2023-07-06T20:19:08.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f58525ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f584c0ffc10 0x7f584c0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:08.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f58525ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f584c100e00 0x7f584c072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:08.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f58525ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f584c101790 0x7f584c072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:08.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f58525ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c073710 con 0x7f584c0ffc10 2023-07-06T20:19:08.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f58525ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c073880 con 0x7f584c100e00 2023-07-06T20:19:08.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f58525ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c19faf0 con 0x7f584c101790 2023-07-06T20:19:08.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f584bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f584c0ffc10 0x7f584c0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:08.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f584bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f584c0ffc10 0x7f584c0743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58308/0 (socket says 172.21.15.69:58308) 2023-07-06T20:19:08.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f584bfff700 1 -- 172.21.15.69:0/4154672065 learned_addr learned my addr 172.21.15.69:0/4154672065 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:08.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f584b7fe700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f584c100e00 0x7f584c072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:08.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f5850b9c700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f584c101790 0x7f584c072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:08.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f584bfff700 1 -- 172.21.15.69:0/4154672065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f584c101790 msgr2=0x7f584c072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f584bfff700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f584c101790 0x7f584c072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f584bfff700 1 -- 172.21.15.69:0/4154672065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f584c100e00 msgr2=0x7f584c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f584bfff700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f584c100e00 0x7f584c072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.536+0000 7f584bfff700 1 -- 172.21.15.69:0/4154672065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58400095f0 con 0x7f584c0ffc10 2023-07-06T20:19:08.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.537+0000 7f584bfff700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f584c0ffc10 0x7f584c0743f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f583c00e970 tx=0x7f583c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:08.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.537+0000 7f58497fa700 1 -- 172.21.15.69:0/4154672065 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c009790 con 0x7f584c0ffc10 2023-07-06T20:19:08.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.537+0000 7f58525ff700 1 -- 172.21.15.69:0/4154672065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5844009c40 con 0x7f584c0ffc10 2023-07-06T20:19:08.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.537+0000 7f58497fa700 1 -- 172.21.15.69:0/4154672065 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f583c00c7e0 con 0x7f584c0ffc10 2023-07-06T20:19:08.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.537+0000 7f58497fa700 1 -- 172.21.15.69:0/4154672065 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c003c70 con 0x7f584c0ffc10 2023-07-06T20:19:08.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.537+0000 7f58525ff700 1 -- 172.21.15.69:0/4154672065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584c1a01a0 con 0x7f584c0ffc10 2023-07-06T20:19:08.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.538+0000 7f58525ff700 1 -- 172.21.15.69:0/4154672065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584c10b440 con 0x7f584c0ffc10 2023-07-06T20:19:08.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.540+0000 7f58497fa700 1 -- 172.21.15.69:0/4154672065 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f583c00cc50 con 0x7f584c0ffc10 2023-07-06T20:19:08.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.541+0000 7f58497fa700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f58340a1150 0x7f58340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:08.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.542+0000 7f58497fa700 1 -- 172.21.15.69:0/4154672065 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f583c014070 con 0x7f584c0ffc10 2023-07-06T20:19:08.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.542+0000 7f584b7fe700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f58340a1150 0x7f58340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:08.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.542+0000 7f584b7fe700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f58340a1150 0x7f58340a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5840005fd0 tx=0x7f5840009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:08.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.546+0000 7f58497fa700 1 -- 172.21.15.69:0/4154672065 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f583c0961f0 con 0x7f584c0ffc10 2023-07-06T20:19:08.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.747+0000 7f58525ff700 1 -- 172.21.15.69:0/4154672065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f584c106a70 con 0x7f584c0ffc10 2023-07-06T20:19:08.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.748+0000 7f58497fa700 1 -- 172.21.15.69:0/4154672065 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f583c003dd0 con 0x7f584c0ffc10 2023-07-06T20:19:08.750 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2023-07-06T20:19:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.750+0000 7f58525ff700 1 -- 172.21.15.69:0/4154672065 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f58340a1150 msgr2=0x7f58340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.751+0000 7f58525ff700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f58340a1150 0x7f58340a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5840005fd0 tx=0x7f5840009310 comp rx=0 tx=0).stop 2023-07-06T20:19:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.751+0000 7f58525ff700 1 -- 172.21.15.69:0/4154672065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f584c0ffc10 msgr2=0x7f584c0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.751+0000 7f58525ff700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f584c0ffc10 0x7f584c0743f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f583c00e970 tx=0x7f583c00ec80 comp rx=0 tx=0).stop 2023-07-06T20:19:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.751+0000 7f58525ff700 1 -- 172.21.15.69:0/4154672065 shutdown_connections 2023-07-06T20:19:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.751+0000 7f58525ff700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f58340a1150 0x7f58340a3600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.751+0000 7f58525ff700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f584c101790 0x7f584c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.751+0000 7f58525ff700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f584c0ffc10 0x7f584c0743f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.751+0000 7f58525ff700 1 --2- 172.21.15.69:0/4154672065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f584c100e00 0x7f584c072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.751+0000 7f58525ff700 1 -- 172.21.15.69:0/4154672065 >> 172.21.15.69:0/4154672065 conn(0x7f584c0fb210 msgr2=0x7f584c0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:08.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.751+0000 7f58525ff700 1 -- 172.21.15.69:0/4154672065 shutdown_connections 2023-07-06T20:19:08.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.751+0000 7f58525ff700 1 -- 172.21.15.69:0/4154672065 wait complete. 2023-07-06T20:19:08.772 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:19:08.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.970+0000 7f1c217ff700 1 -- 172.21.15.69:0/96082084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c1c100e10 msgr2=0x7f1c1c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.970+0000 7f1c217ff700 1 --2- 172.21.15.69:0/96082084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c1c100e10 0x7f1c1c101260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009990 tx=0x7f1c10009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.970+0000 7f1c217ff700 1 -- 172.21.15.69:0/96082084 shutdown_connections 2023-07-06T20:19:08.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.970+0000 7f1c217ff700 1 --2- 172.21.15.69:0/96082084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c1c1017a0 0x7f1c1c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.970+0000 7f1c217ff700 1 --2- 172.21.15.69:0/96082084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c1c0ffc20 0x7f1c1c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.970+0000 7f1c217ff700 1 --2- 172.21.15.69:0/96082084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c1c100e10 0x7f1c1c101260 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.970+0000 7f1c217ff700 1 -- 172.21.15.69:0/96082084 >> 172.21.15.69:0/96082084 conn(0x7f1c1c0fb260 msgr2=0x7f1c1c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:08.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.970+0000 7f1c217ff700 1 -- 172.21.15.69:0/96082084 shutdown_connections 2023-07-06T20:19:08.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.970+0000 7f1c217ff700 1 -- 172.21.15.69:0/96082084 wait complete. 2023-07-06T20:19:08.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.971+0000 7f1c217ff700 1 Processor -- start 2023-07-06T20:19:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.971+0000 7f1c217ff700 1 -- start start 2023-07-06T20:19:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.972+0000 7f1c217ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c1c0ffc20 0x7f1c1c195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.972+0000 7f1c217ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c1c100e10 0x7f1c1c195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.972+0000 7f1c217ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c1c1017a0 0x7f1c1c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.972+0000 7f1c217ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c19d4f0 con 0x7f1c1c0ffc20 2023-07-06T20:19:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.972+0000 7f1c217ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c19d630 con 0x7f1c1c1017a0 2023-07-06T20:19:08.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.972+0000 7f1c217ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c19d7c0 con 0x7f1c1c100e10 2023-07-06T20:19:08.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.972+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c1c0ffc20 0x7f1c1c195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:08.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.972+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c1c0ffc20 0x7f1c1c195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58330/0 (socket says 172.21.15.69:58330) 2023-07-06T20:19:08.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.972+0000 7f1c1affd700 1 -- 172.21.15.69:0/558267658 learned_addr learned my addr 172.21.15.69:0/558267658 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:08.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.972+0000 7f1c1b7fe700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c1c1017a0 0x7f1c1c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:08.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.972+0000 7f1c1a7fc700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c1c100e10 0x7f1c1c195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:08.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.973+0000 7f1c1affd700 1 -- 172.21.15.69:0/558267658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c1c100e10 msgr2=0x7f1c1c195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.973+0000 7f1c1affd700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c1c100e10 0x7f1c1c195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.973+0000 7f1c1affd700 1 -- 172.21.15.69:0/558267658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c1c1017a0 msgr2=0x7f1c1c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:08.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.973+0000 7f1c1affd700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c1c1017a0 0x7f1c1c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:08.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.973+0000 7f1c1affd700 1 -- 172.21.15.69:0/558267658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c100095f0 con 0x7f1c1c0ffc20 2023-07-06T20:19:08.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.973+0000 7f1c1b7fe700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c1c1017a0 0x7f1c1c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:08.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.973+0000 7f1c1affd700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c1c0ffc20 0x7f1c1c195970 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00e970 tx=0x7f1c0c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:08.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.973+0000 7f1c03fff700 1 -- 172.21.15.69:0/558267658 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c009790 con 0x7f1c1c0ffc20 2023-07-06T20:19:08.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.973+0000 7f1c03fff700 1 -- 172.21.15.69:0/558267658 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c0c00c7e0 con 0x7f1c1c0ffc20 2023-07-06T20:19:08.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.973+0000 7f1c217ff700 1 -- 172.21.15.69:0/558267658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1c19daa0 con 0x7f1c1c0ffc20 2023-07-06T20:19:08.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.973+0000 7f1c03fff700 1 -- 172.21.15.69:0/558267658 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c003c70 con 0x7f1c1c0ffc20 2023-07-06T20:19:08.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.973+0000 7f1c217ff700 1 -- 172.21.15.69:0/558267658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1c19dfb0 con 0x7f1c1c0ffc20 2023-07-06T20:19:08.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.974+0000 7f1c217ff700 1 -- 172.21.15.69:0/558267658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1c10b440 con 0x7f1c1c0ffc20 2023-07-06T20:19:08.979 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.976+0000 7f1c03fff700 1 -- 172.21.15.69:0/558267658 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1c0c00cc50 con 0x7f1c1c0ffc20 2023-07-06T20:19:08.979 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.977+0000 7f1c03fff700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1c040a1150 0x7f1c040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.978+0000 7f1c03fff700 1 -- 172.21.15.69:0/558267658 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f1c0c014070 con 0x7f1c1c0ffc20 2023-07-06T20:19:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.978+0000 7f1c1a7fc700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1c040a1150 0x7f1c040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:08.980 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.978+0000 7f1c1a7fc700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1c040a1150 0x7f1c040a3600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000b2e0 tx=0x7f1c10005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:08.981 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:08.980+0000 7f1c03fff700 1 -- 172.21.15.69:0/558267658 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1c0c096370 con 0x7f1c1c0ffc20 2023-07-06T20:19:09.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:08 smithi069 ceph-mon[126399]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 805 B/s wr, 6 op/s; 120 B/s, 0 objects/s recovering 2023-07-06T20:19:09.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:08 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1244176791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:19:09.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:08 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2364403502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:09.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:08 smithi156 ceph-mon[129280]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 805 B/s wr, 6 op/s; 120 B/s, 0 objects/s recovering 2023-07-06T20:19:09.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:08 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1244176791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:19:09.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:08 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2364403502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:09.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:08 smithi098 ceph-mon[128968]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 805 B/s wr, 6 op/s; 120 B/s, 0 objects/s recovering 2023-07-06T20:19:09.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:08 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1244176791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:19:09.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:08 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2364403502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:09.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.177+0000 7f1c217ff700 1 -- 172.21.15.69:0/558267658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c1c106b20 con 0x7f1c1c0ffc20 2023-07-06T20:19:09.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.178+0000 7f1c03fff700 1 -- 172.21.15.69:0/558267658 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c0c099990 con 0x7f1c1c0ffc20 2023-07-06T20:19:09.180 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2023-07-06T20:19:09.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.180+0000 7f1c217ff700 1 -- 172.21.15.69:0/558267658 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1c040a1150 msgr2=0x7f1c040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:09.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.180+0000 7f1c217ff700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1c040a1150 0x7f1c040a3600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000b2e0 tx=0x7f1c10005fb0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.180+0000 7f1c217ff700 1 -- 172.21.15.69:0/558267658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c1c0ffc20 msgr2=0x7f1c1c195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:09.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.180+0000 7f1c217ff700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c1c0ffc20 0x7f1c1c195970 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00e970 tx=0x7f1c0c00ec80 comp rx=0 tx=0).stop 2023-07-06T20:19:09.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.180+0000 7f1c217ff700 1 -- 172.21.15.69:0/558267658 shutdown_connections 2023-07-06T20:19:09.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.180+0000 7f1c217ff700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1c040a1150 0x7f1c040a3600 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.180+0000 7f1c217ff700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c1c1017a0 0x7f1c1c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.180+0000 7f1c217ff700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c1c0ffc20 0x7f1c1c195970 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.180+0000 7f1c217ff700 1 --2- 172.21.15.69:0/558267658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c1c100e10 0x7f1c1c195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.180+0000 7f1c217ff700 1 -- 172.21.15.69:0/558267658 >> 172.21.15.69:0/558267658 conn(0x7f1c1c0fb260 msgr2=0x7f1c1c0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:09.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.180+0000 7f1c217ff700 1 -- 172.21.15.69:0/558267658 shutdown_connections 2023-07-06T20:19:09.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.180+0000 7f1c217ff700 1 -- 172.21.15.69:0/558267658 wait complete. 2023-07-06T20:19:09.200 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:19:09.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.397+0000 7f71ccdff700 1 -- 172.21.15.69:0/1596990855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c80ffc10 msgr2=0x7f71c8100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:09.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.397+0000 7f71ccdff700 1 --2- 172.21.15.69:0/1596990855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c80ffc10 0x7f71c8100020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009990 tx=0x7f71b8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.397+0000 7f71ccdff700 1 -- 172.21.15.69:0/1596990855 shutdown_connections 2023-07-06T20:19:09.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.397+0000 7f71ccdff700 1 --2- 172.21.15.69:0/1596990855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c8101790 0x7f71c8108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.397+0000 7f71ccdff700 1 --2- 172.21.15.69:0/1596990855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c80ffc10 0x7f71c8100020 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.397+0000 7f71ccdff700 1 --2- 172.21.15.69:0/1596990855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c8100e00 0x7f71c8101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.397+0000 7f71ccdff700 1 -- 172.21.15.69:0/1596990855 >> 172.21.15.69:0/1596990855 conn(0x7f71c80fb210 msgr2=0x7f71c80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:09.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.397+0000 7f71ccdff700 1 -- 172.21.15.69:0/1596990855 shutdown_connections 2023-07-06T20:19:09.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.397+0000 7f71ccdff700 1 -- 172.21.15.69:0/1596990855 wait complete. 2023-07-06T20:19:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.398+0000 7f71ccdff700 1 Processor -- start 2023-07-06T20:19:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.398+0000 7f71ccdff700 1 -- start start 2023-07-06T20:19:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.398+0000 7f71ccdff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c80ffc10 0x7f71c80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.398+0000 7f71ccdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c8100e00 0x7f71c8072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:09.403 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.398+0000 7f71ccdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c8101790 0x7f71c8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:09.403 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.398+0000 7f71ccdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8073710 con 0x7f71c8101790 2023-07-06T20:19:09.403 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71ccdff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8073880 con 0x7f71c80ffc10 2023-07-06T20:19:09.403 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71ccdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c819f9f0 con 0x7f71c8100e00 2023-07-06T20:19:09.403 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71c6d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c8101790 0x7f71c8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:09.403 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71c659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c80ffc10 0x7f71c80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:09.403 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71c6d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c8101790 0x7f71c8072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58344/0 (socket says 172.21.15.69:58344) 2023-07-06T20:19:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71c6d9d700 1 -- 172.21.15.69:0/356159787 learned_addr learned my addr 172.21.15.69:0/356159787 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71c5d9b700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c8100e00 0x7f71c8072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71c659c700 1 -- 172.21.15.69:0/356159787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c8100e00 msgr2=0x7f71c8072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71c659c700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c8100e00 0x7f71c8072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71c659c700 1 -- 172.21.15.69:0/356159787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c8101790 msgr2=0x7f71c8072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71c659c700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c8101790 0x7f71c8072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71c659c700 1 -- 172.21.15.69:0/356159787 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b80095f0 con 0x7f71c80ffc10 2023-07-06T20:19:09.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71c6d9d700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c8101790 0x7f71c8072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:09.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.399+0000 7f71c659c700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c80ffc10 0x7f71c80743f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f71b800b620 tx=0x7f71b8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:09.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.400+0000 7f71b77fe700 1 -- 172.21.15.69:0/356159787 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b801d070 con 0x7f71c80ffc10 2023-07-06T20:19:09.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.400+0000 7f71ccdff700 1 -- 172.21.15.69:0/356159787 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c819fc70 con 0x7f71c80ffc10 2023-07-06T20:19:09.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.400+0000 7f71b77fe700 1 -- 172.21.15.69:0/356159787 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71b8022470 con 0x7f71c80ffc10 2023-07-06T20:19:09.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.400+0000 7f71b77fe700 1 -- 172.21.15.69:0/356159787 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8017650 con 0x7f71c80ffc10 2023-07-06T20:19:09.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.400+0000 7f71ccdff700 1 -- 172.21.15.69:0/356159787 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c81a0090 con 0x7f71c80ffc10 2023-07-06T20:19:09.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.401+0000 7f71ccdff700 1 -- 172.21.15.69:0/356159787 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c810b3e0 con 0x7f71c80ffc10 2023-07-06T20:19:09.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.403+0000 7f71b77fe700 1 -- 172.21.15.69:0/356159787 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f71b80225e0 con 0x7f71c80ffc10 2023-07-06T20:19:09.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.404+0000 7f71b77fe700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71b00a1150 0x7f71b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:09.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.404+0000 7f71b77fe700 1 -- 172.21.15.69:0/356159787 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f71b80ce470 con 0x7f71c80ffc10 2023-07-06T20:19:09.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.405+0000 7f71c5d9b700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71b00a1150 0x7f71b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:09.408 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.405+0000 7f71c5d9b700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71b00a1150 0x7f71b00a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009c60 tx=0x7f71bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:09.408 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.406+0000 7f71b77fe700 1 -- 172.21.15.69:0/356159787 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f71b8097bc0 con 0x7f71c80ffc10 2023-07-06T20:19:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.603+0000 7f71ccdff700 1 -- 172.21.15.69:0/356159787 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f71c8106b40 con 0x7f71c80ffc10 2023-07-06T20:19:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.604+0000 7f71b77fe700 1 -- 172.21.15.69:0/356159787 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f71b809b1e0 con 0x7f71c80ffc10 2023-07-06T20:19:09.606 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2023-07-06T20:19:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.606+0000 7f71ccdff700 1 -- 172.21.15.69:0/356159787 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71b00a1150 msgr2=0x7f71b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.606+0000 7f71ccdff700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71b00a1150 0x7f71b00a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009c60 tx=0x7f71bc009260 comp rx=0 tx=0).stop 2023-07-06T20:19:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.606+0000 7f71ccdff700 1 -- 172.21.15.69:0/356159787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c80ffc10 msgr2=0x7f71c80743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.606+0000 7f71ccdff700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c80ffc10 0x7f71c80743f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f71b800b620 tx=0x7f71b8004730 comp rx=0 tx=0).stop 2023-07-06T20:19:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.607+0000 7f71ccdff700 1 -- 172.21.15.69:0/356159787 shutdown_connections 2023-07-06T20:19:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.607+0000 7f71ccdff700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71b00a1150 0x7f71b00a3600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.607+0000 7f71ccdff700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c8101790 0x7f71c8072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.607+0000 7f71ccdff700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c80ffc10 0x7f71c80743f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.607+0000 7f71ccdff700 1 --2- 172.21.15.69:0/356159787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c8100e00 0x7f71c8072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.607+0000 7f71ccdff700 1 -- 172.21.15.69:0/356159787 >> 172.21.15.69:0/356159787 conn(0x7f71c80fb210 msgr2=0x7f71c80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:09.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.607+0000 7f71ccdff700 1 -- 172.21.15.69:0/356159787 shutdown_connections 2023-07-06T20:19:09.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.607+0000 7f71ccdff700 1 -- 172.21.15.69:0/356159787 wait complete. 2023-07-06T20:19:09.627 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-06T20:19:09.628 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:19:09.823 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.822+0000 7efc897ff700 1 -- 172.21.15.69:0/1096051563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc84100e20 msgr2=0x7efc84101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:09.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.822+0000 7efc897ff700 1 --2- 172.21.15.69:0/1096051563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc84100e20 0x7efc84101270 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efc78009990 tx=0x7efc78009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.822+0000 7efc897ff700 1 -- 172.21.15.69:0/1096051563 shutdown_connections 2023-07-06T20:19:09.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.822+0000 7efc897ff700 1 --2- 172.21.15.69:0/1096051563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc841017b0 0x7efc84108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.822+0000 7efc897ff700 1 --2- 172.21.15.69:0/1096051563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc840ffc30 0x7efc84100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.822+0000 7efc897ff700 1 --2- 172.21.15.69:0/1096051563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc84100e20 0x7efc84101270 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.822+0000 7efc897ff700 1 -- 172.21.15.69:0/1096051563 >> 172.21.15.69:0/1096051563 conn(0x7efc840fb210 msgr2=0x7efc840fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:09.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.822+0000 7efc897ff700 1 -- 172.21.15.69:0/1096051563 shutdown_connections 2023-07-06T20:19:09.825 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.822+0000 7efc897ff700 1 -- 172.21.15.69:0/1096051563 wait complete. 2023-07-06T20:19:09.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.823+0000 7efc897ff700 1 Processor -- start 2023-07-06T20:19:09.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc897ff700 1 -- start start 2023-07-06T20:19:09.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc897ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc840ffc30 0x7efc840743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:09.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc897ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc84100e20 0x7efc84072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:09.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc897ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc841017b0 0x7efc84072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:09.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc897ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc84073710 con 0x7efc841017b0 2023-07-06T20:19:09.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc897ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc84073880 con 0x7efc84100e20 2023-07-06T20:19:09.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc897ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8419fa20 con 0x7efc840ffc30 2023-07-06T20:19:09.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc837fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc841017b0 0x7efc84072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:09.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc827fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc84100e20 0x7efc84072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:09.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc837fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc841017b0 0x7efc84072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58372/0 (socket says 172.21.15.69:58372) 2023-07-06T20:19:09.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc837fe700 1 -- 172.21.15.69:0/4014404632 learned_addr learned my addr 172.21.15.69:0/4014404632 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:09.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc82ffd700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc840ffc30 0x7efc840743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:09.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc837fe700 1 -- 172.21.15.69:0/4014404632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc840ffc30 msgr2=0x7efc840743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:09.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.824+0000 7efc837fe700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc840ffc30 0x7efc840743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.825+0000 7efc837fe700 1 -- 172.21.15.69:0/4014404632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc84100e20 msgr2=0x7efc84072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:09.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.825+0000 7efc837fe700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc84100e20 0x7efc84072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:09.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.825+0000 7efc837fe700 1 -- 172.21.15.69:0/4014404632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc780095f0 con 0x7efc841017b0 2023-07-06T20:19:09.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.825+0000 7efc837fe700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc841017b0 0x7efc84072f70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00ef30 tx=0x7efc7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:09.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.825+0000 7efc6bfff700 1 -- 172.21.15.69:0/4014404632 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c018070 con 0x7efc841017b0 2023-07-06T20:19:09.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.825+0000 7efc6bfff700 1 -- 172.21.15.69:0/4014404632 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc7c00f040 con 0x7efc841017b0 2023-07-06T20:19:09.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.825+0000 7efc897ff700 1 -- 172.21.15.69:0/4014404632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8419fca0 con 0x7efc841017b0 2023-07-06T20:19:09.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.825+0000 7efc6bfff700 1 -- 172.21.15.69:0/4014404632 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c00cd00 con 0x7efc841017b0 2023-07-06T20:19:09.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.825+0000 7efc897ff700 1 -- 172.21.15.69:0/4014404632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc841a01b0 con 0x7efc841017b0 2023-07-06T20:19:09.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.826+0000 7efc897ff700 1 -- 172.21.15.69:0/4014404632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc84107180 con 0x7efc841017b0 2023-07-06T20:19:09.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.828+0000 7efc6bfff700 1 -- 172.21.15.69:0/4014404632 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efc7c003a00 con 0x7efc841017b0 2023-07-06T20:19:09.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.830+0000 7efc6bfff700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7efc6c0a1220 0x7efc6c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:09.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.830+0000 7efc6bfff700 1 -- 172.21.15.69:0/4014404632 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7efc7c0cc3b0 con 0x7efc841017b0 2023-07-06T20:19:09.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.830+0000 7efc82ffd700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7efc6c0a1220 0x7efc6c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:09.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.831+0000 7efc82ffd700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7efc6c0a1220 0x7efc6c0a36d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efc84100c80 tx=0x7efc74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:09.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:09.834+0000 7efc6bfff700 1 -- 172.21.15.69:0/4014404632 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7efc7c095b00 con 0x7efc841017b0 2023-07-06T20:19:10.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:09 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/4154672065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:19:10.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:09 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/558267658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:19:10.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:09 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/356159787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:19:10.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.076+0000 7efc897ff700 1 -- 172.21.15.69:0/4014404632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc8402c8e0 con 0x7efc841017b0 2023-07-06T20:19:10.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.077+0000 7efc6bfff700 1 -- 172.21.15.69:0/4014404632 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10795 (secure 0 0 0) 0x7efc7c099120 con 0x7efc841017b0 2023-07-06T20:19:10.078 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:10.079 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-06T20:17:28.144582+0000","modified":"2023-07-06T20:17:33.557511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24539,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/3235525224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3235525224},{"type":"v1","addr":"172.21.15.98:6839","nonce":3235525224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:19:10.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.079+0000 7efc897ff700 1 -- 172.21.15.69:0/4014404632 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7efc6c0a1220 msgr2=0x7efc6c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:10.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.080+0000 7efc897ff700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7efc6c0a1220 0x7efc6c0a36d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efc84100c80 tx=0x7efc74009260 comp rx=0 tx=0).stop 2023-07-06T20:19:10.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.080+0000 7efc897ff700 1 -- 172.21.15.69:0/4014404632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc841017b0 msgr2=0x7efc84072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:10.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.080+0000 7efc897ff700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc841017b0 0x7efc84072f70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00ef30 tx=0x7efc7c00c3a0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.080+0000 7efc897ff700 1 -- 172.21.15.69:0/4014404632 shutdown_connections 2023-07-06T20:19:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.080+0000 7efc897ff700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7efc6c0a1220 0x7efc6c0a36d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.080+0000 7efc897ff700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc841017b0 0x7efc84072f70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.080+0000 7efc897ff700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc840ffc30 0x7efc840743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.080+0000 7efc897ff700 1 --2- 172.21.15.69:0/4014404632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc84100e20 0x7efc84072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.080+0000 7efc897ff700 1 -- 172.21.15.69:0/4014404632 >> 172.21.15.69:0/4014404632 conn(0x7efc840fb210 msgr2=0x7efc841042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.080+0000 7efc897ff700 1 -- 172.21.15.69:0/4014404632 shutdown_connections 2023-07-06T20:19:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.080+0000 7efc897ff700 1 -- 172.21.15.69:0/4014404632 wait complete. 2023-07-06T20:19:10.083 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2023-07-06T20:19:10.099 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-06T20:19:10.100 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-06T20:19:10.100 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:19:10.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:09 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/4154672065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:19:10.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:09 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/558267658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:19:10.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:09 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/356159787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:19:10.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:09 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/4154672065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:19:10.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:09 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/558267658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:19:10.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:09 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/356159787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:19:10.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.294+0000 7fa683bff700 1 -- 172.21.15.69:0/3411674851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c100e20 msgr2=0x7fa67c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:10.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.294+0000 7fa683bff700 1 --2- 172.21.15.69:0/3411674851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c100e20 0x7fa67c101270 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009990 tx=0x7fa66c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.294+0000 7fa683bff700 1 -- 172.21.15.69:0/3411674851 shutdown_connections 2023-07-06T20:19:10.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.294+0000 7fa683bff700 1 --2- 172.21.15.69:0/3411674851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa67c1017b0 0x7fa67c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.294+0000 7fa683bff700 1 --2- 172.21.15.69:0/3411674851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c0ffc30 0x7fa67c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.294+0000 7fa683bff700 1 --2- 172.21.15.69:0/3411674851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c100e20 0x7fa67c101270 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.294+0000 7fa683bff700 1 -- 172.21.15.69:0/3411674851 >> 172.21.15.69:0/3411674851 conn(0x7fa67c0fb210 msgr2=0x7fa67c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:10.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.294+0000 7fa683bff700 1 -- 172.21.15.69:0/3411674851 shutdown_connections 2023-07-06T20:19:10.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.294+0000 7fa683bff700 1 -- 172.21.15.69:0/3411674851 wait complete. 2023-07-06T20:19:10.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.295+0000 7fa683bff700 1 Processor -- start 2023-07-06T20:19:10.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.295+0000 7fa683bff700 1 -- start start 2023-07-06T20:19:10.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa683bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c0ffc30 0x7fa67c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:10.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa683bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa67c100e20 0x7fa67c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:10.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa683bff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c1017b0 0x7fa67c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:10.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa683bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c19f6f0 con 0x7fa67c0ffc30 2023-07-06T20:19:10.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa683bff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c19f830 con 0x7fa67c1017b0 2023-07-06T20:19:10.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa683bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c19f9c0 con 0x7fa67c100e20 2023-07-06T20:19:10.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa68199b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c0ffc30 0x7fa67c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:10.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa68199b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c0ffc30 0x7fa67c197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58390/0 (socket says 172.21.15.69:58390) 2023-07-06T20:19:10.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa68199b700 1 -- 172.21.15.69:0/3753223783 learned_addr learned my addr 172.21.15.69:0/3753223783 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:10.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa68119a700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa67c100e20 0x7fa67c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:10.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa68199b700 1 -- 172.21.15.69:0/3753223783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa67c100e20 msgr2=0x7fa67c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:10.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa68199b700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa67c100e20 0x7fa67c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa68199b700 1 -- 172.21.15.69:0/3753223783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c1017b0 msgr2=0x7fa67c19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:10.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa68219c700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c1017b0 0x7fa67c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:10.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa68199b700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c1017b0 0x7fa67c19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa68199b700 1 -- 172.21.15.69:0/3753223783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c0095f0 con 0x7fa67c0ffc30 2023-07-06T20:19:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa68119a700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa67c100e20 0x7fa67c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.296+0000 7fa68219c700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c1017b0 0x7fa67c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:19:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.297+0000 7fa68199b700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c0ffc30 0x7fa67c197b70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa67800e970 tx=0x7fa67800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.297+0000 7fa672ffd700 1 -- 172.21.15.69:0/3753223783 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678009790 con 0x7fa67c0ffc30 2023-07-06T20:19:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.297+0000 7fa683bff700 1 -- 172.21.15.69:0/3753223783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67c19fca0 con 0x7fa67c0ffc30 2023-07-06T20:19:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.297+0000 7fa672ffd700 1 -- 172.21.15.69:0/3753223783 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa67800eec0 con 0x7fa67c0ffc30 2023-07-06T20:19:10.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.297+0000 7fa672ffd700 1 -- 172.21.15.69:0/3753223783 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678003cf0 con 0x7fa67c0ffc30 2023-07-06T20:19:10.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.297+0000 7fa683bff700 1 -- 172.21.15.69:0/3753223783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67c1a01b0 con 0x7fa67c0ffc30 2023-07-06T20:19:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.298+0000 7fa683bff700 1 -- 172.21.15.69:0/3753223783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67c10b450 con 0x7fa67c0ffc30 2023-07-06T20:19:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.300+0000 7fa672ffd700 1 -- 172.21.15.69:0/3753223783 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa67800ccd0 con 0x7fa67c0ffc30 2023-07-06T20:19:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.301+0000 7fa672ffd700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa6680a11a0 0x7fa6680a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.301+0000 7fa672ffd700 1 -- 172.21.15.69:0/3753223783 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fa678014070 con 0x7fa67c0ffc30 2023-07-06T20:19:10.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.302+0000 7fa68119a700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa6680a11a0 0x7fa6680a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:10.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.302+0000 7fa68119a700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa6680a11a0 0x7fa6680a3650 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00b2e0 tx=0x7fa66c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:10.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.305+0000 7fa672ffd700 1 -- 172.21.15.69:0/3753223783 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa6780962b0 con 0x7fa67c0ffc30 2023-07-06T20:19:10.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.497+0000 7fa683bff700 1 -- 172.21.15.69:0/3753223783 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa67c107720 con 0x7fa6680a11a0 2023-07-06T20:19:10.510 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.510+0000 7fa672ffd700 1 -- 172.21.15.69:0/3753223783 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344244 (secure 0 0 0) 0x7fa67c107720 con 0x7fa6680a11a0 2023-07-06T20:19:10.514 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:10.516 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.514+0000 7fa683bff700 1 -- 172.21.15.69:0/3753223783 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa6680a11a0 msgr2=0x7fa6680a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:10.516 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.514+0000 7fa683bff700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa6680a11a0 0x7fa6680a3650 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00b2e0 tx=0x7fa66c005fb0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.514+0000 7fa683bff700 1 -- 172.21.15.69:0/3753223783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c0ffc30 msgr2=0x7fa67c197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.514+0000 7fa683bff700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c0ffc30 0x7fa67c197b70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa67800e970 tx=0x7fa67800ec80 comp rx=0 tx=0).stop 2023-07-06T20:19:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.514+0000 7fa683bff700 1 -- 172.21.15.69:0/3753223783 shutdown_connections 2023-07-06T20:19:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.514+0000 7fa683bff700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa6680a11a0 0x7fa6680a3650 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.514+0000 7fa683bff700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c1017b0 0x7fa67c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.514+0000 7fa683bff700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa67c0ffc30 0x7fa67c197b70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.514+0000 7fa683bff700 1 --2- 172.21.15.69:0/3753223783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa67c100e20 0x7fa67c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.514+0000 7fa683bff700 1 -- 172.21.15.69:0/3753223783 >> 172.21.15.69:0/3753223783 conn(0x7fa67c0fb210 msgr2=0x7fa67c0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.514+0000 7fa683bff700 1 -- 172.21.15.69:0/3753223783 shutdown_connections 2023-07-06T20:19:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.514+0000 7fa683bff700 1 -- 172.21.15.69:0/3753223783 wait complete. 2023-07-06T20:19:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-07-06T20:19:10.534 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2023-07-06T20:19:09.545348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654278,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":715,"num_read_kb":471,"num_write":345,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181919,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336776,"kb_used_data":13352,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605048,"statfs":{"total":1151940427776,"available":1151595569152,"internally_reserved":0,"allocated":13672448,"data_stored":7498209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":702,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":31,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1071,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002553"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574765+0000","last_change":"2023-07-06T20:17:28.575520+0000","last_active":"2023-07-06T20:19:00.574765+0000","last_peered":"2023-07-06T20:19:00.574765+0000","last_clean":"2023-07-06T20:19:00.574765+0000","last_became_active":"2023-07-06T20:17:28.575209+0000","last_became_peered":"2023-07-06T20:17:28.575209+0000","last_unstale":"2023-07-06T20:19:00.574765+0000","last_undegraded":"2023-07-06T20:19:00.574765+0000","last_fullsized":"2023-07-06T20:19:00.574765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:31:43.014901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575773+0000","last_change":"2023-07-06T20:17:28.572547+0000","last_active":"2023-07-06T20:19:00.575773+0000","last_peered":"2023-07-06T20:19:00.575773+0000","last_clean":"2023-07-06T20:19:00.575773+0000","last_became_active":"2023-07-06T20:17:28.572150+0000","last_became_peered":"2023-07-06T20:17:28.572150+0000","last_unstale":"2023-07-06T20:19:00.575773+0000","last_undegraded":"2023-07-06T20:19:00.575773+0000","last_fullsized":"2023-07-06T20:19:00.575773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:16.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575509+0000","last_change":"2023-07-06T20:17:28.574926+0000","last_active":"2023-07-06T20:19:00.575509+0000","last_peered":"2023-07-06T20:19:00.575509+0000","last_clean":"2023-07-06T20:19:00.575509+0000","last_became_active":"2023-07-06T20:17:28.574602+0000","last_became_peered":"2023-07-06T20:17:28.574602+0000","last_unstale":"2023-07-06T20:19:00.575509+0000","last_undegraded":"2023-07-06T20:19:00.575509+0000","last_fullsized":"2023-07-06T20:19:00.575509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:37:21.626981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.651733+0000","last_change":"2023-07-06T20:17:28.571533+0000","last_active":"2023-07-06T20:18:59.651733+0000","last_peered":"2023-07-06T20:18:59.651733+0000","last_clean":"2023-07-06T20:18:59.651733+0000","last_became_active":"2023-07-06T20:17:28.571099+0000","last_became_peered":"2023-07-06T20:17:28.571099+0000","last_unstale":"2023-07-06T20:18:59.651733+0000","last_undegraded":"2023-07-06T20:18:59.651733+0000","last_fullsized":"2023-07-06T20:18:59.651733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:09:44.688143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646150+0000","last_change":"2023-07-06T20:17:28.572100+0000","last_active":"2023-07-06T20:18:59.646150+0000","last_peered":"2023-07-06T20:18:59.646150+0000","last_clean":"2023-07-06T20:18:59.646150+0000","last_became_active":"2023-07-06T20:17:28.571772+0000","last_became_peered":"2023-07-06T20:17:28.571772+0000","last_unstale":"2023-07-06T20:18:59.646150+0000","last_undegraded":"2023-07-06T20:18:59.646150+0000","last_fullsized":"2023-07-06T20:18:59.646150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:20:48.756324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654341+0000","last_change":"2023-07-06T20:17:28.577816+0000","last_active":"2023-07-06T20:18:59.654341+0000","last_peered":"2023-07-06T20:18:59.654341+0000","last_clean":"2023-07-06T20:18:59.654341+0000","last_became_active":"2023-07-06T20:17:28.577382+0000","last_became_peered":"2023-07-06T20:17:28.577382+0000","last_unstale":"2023-07-06T20:18:59.654341+0000","last_undegraded":"2023-07-06T20:18:59.654341+0000","last_fullsized":"2023-07-06T20:18:59.654341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:45:41.682441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.653973+0000","last_change":"2023-07-06T20:17:28.570415+0000","last_active":"2023-07-06T20:18:59.653973+0000","last_peered":"2023-07-06T20:18:59.653973+0000","last_clean":"2023-07-06T20:18:59.653973+0000","last_became_active":"2023-07-06T20:17:28.570091+0000","last_became_peered":"2023-07-06T20:17:28.570091+0000","last_unstale":"2023-07-06T20:18:59.653973+0000","last_undegraded":"2023-07-06T20:18:59.653973+0000","last_fullsized":"2023-07-06T20:18:59.653973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:39:50.216113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575038+0000","last_change":"2023-07-06T20:17:28.568370+0000","last_active":"2023-07-06T20:19:00.575038+0000","last_peered":"2023-07-06T20:19:00.575038+0000","last_clean":"2023-07-06T20:19:00.575038+0000","last_became_active":"2023-07-06T20:17:28.568032+0000","last_became_peered":"2023-07-06T20:17:28.568032+0000","last_unstale":"2023-07-06T20:19:00.575038+0000","last_undegraded":"2023-07-06T20:19:00.575038+0000","last_fullsized":"2023-07-06T20:19:00.575038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:29:40.448529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648913+0000","last_change":"2023-07-06T20:17:28.563386+0000","last_active":"2023-07-06T20:18:59.648913+0000","last_peered":"2023-07-06T20:18:59.648913+0000","last_clean":"2023-07-06T20:18:59.648913+0000","last_became_active":"2023-07-06T20:17:28.562864+0000","last_became_peered":"2023-07-06T20:17:28.562864+0000","last_unstale":"2023-07-06T20:18:59.648913+0000","last_undegraded":"2023-07-06T20:18:59.648913+0000","last_fullsized":"2023-07-06T20:18:59.648913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:17:20.864607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.655052+0000","last_change":"2023-07-06T20:17:28.578016+0000","last_active":"2023-07-06T20:18:59.655052+0000","last_peered":"2023-07-06T20:18:59.655052+0000","last_clean":"2023-07-06T20:18:59.655052+0000","last_became_active":"2023-07-06T20:17:28.577305+0000","last_became_peered":"2023-07-06T20:17:28.577305+0000","last_unstale":"2023-07-06T20:18:59.655052+0000","last_undegraded":"2023-07-06T20:18:59.655052+0000","last_fullsized":"2023-07-06T20:18:59.655052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:04.298499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654451+0000","last_change":"2023-07-06T20:17:28.569683+0000","last_active":"2023-07-06T20:18:59.654451+0000","last_peered":"2023-07-06T20:18:59.654451+0000","last_clean":"2023-07-06T20:18:59.654451+0000","last_became_active":"2023-07-06T20:17:28.569070+0000","last_became_peered":"2023-07-06T20:17:28.569070+0000","last_unstale":"2023-07-06T20:18:59.654451+0000","last_undegraded":"2023-07-06T20:18:59.654451+0000","last_fullsized":"2023-07-06T20:18:59.654451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:32:02.630231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.651730+0000","last_change":"2023-07-06T20:17:28.579463+0000","last_active":"2023-07-06T20:18:59.651730+0000","last_peered":"2023-07-06T20:18:59.651730+0000","last_clean":"2023-07-06T20:18:59.651730+0000","last_became_active":"2023-07-06T20:17:28.579141+0000","last_became_peered":"2023-07-06T20:17:28.579141+0000","last_unstale":"2023-07-06T20:18:59.651730+0000","last_undegraded":"2023-07-06T20:18:59.651730+0000","last_fullsized":"2023-07-06T20:18:59.651730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:35:41.411836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646027+0000","last_change":"2023-07-06T20:17:58.864016+0000","last_active":"2023-07-06T20:18:59.646027+0000","last_peered":"2023-07-06T20:18:59.646027+0000","last_clean":"2023-07-06T20:18:59.646027+0000","last_became_active":"2023-07-06T20:17:58.863583+0000","last_became_peered":"2023-07-06T20:17:58.863583+0000","last_unstale":"2023-07-06T20:18:59.646027+0000","last_undegraded":"2023-07-06T20:18:59.646027+0000","last_fullsized":"2023-07-06T20:18:59.646027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:20.046705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575003+0000","last_change":"2023-07-06T20:17:28.572197+0000","last_active":"2023-07-06T20:19:00.575003+0000","last_peered":"2023-07-06T20:19:00.575003+0000","last_clean":"2023-07-06T20:19:00.575003+0000","last_became_active":"2023-07-06T20:17:28.571816+0000","last_became_peered":"2023-07-06T20:17:28.571816+0000","last_unstale":"2023-07-06T20:19:00.575003+0000","last_undegraded":"2023-07-06T20:19:00.575003+0000","last_fullsized":"2023-07-06T20:19:00.575003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:51.605941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574805+0000","last_change":"2023-07-06T20:17:58.746966+0000","last_active":"2023-07-06T20:19:00.574805+0000","last_peered":"2023-07-06T20:19:00.574805+0000","last_clean":"2023-07-06T20:19:00.574805+0000","last_became_active":"2023-07-06T20:17:58.746598+0000","last_became_peered":"2023-07-06T20:17:58.746598+0000","last_unstale":"2023-07-06T20:19:00.574805+0000","last_undegraded":"2023-07-06T20:19:00.574805+0000","last_fullsized":"2023-07-06T20:19:00.574805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:23:21.366915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648494+0000","last_change":"2023-07-06T20:17:28.573552+0000","last_active":"2023-07-06T20:18:59.648494+0000","last_peered":"2023-07-06T20:18:59.648494+0000","last_clean":"2023-07-06T20:18:59.648494+0000","last_became_active":"2023-07-06T20:17:28.573253+0000","last_became_peered":"2023-07-06T20:17:28.573253+0000","last_unstale":"2023-07-06T20:18:59.648494+0000","last_undegraded":"2023-07-06T20:18:59.648494+0000","last_fullsized":"2023-07-06T20:18:59.648494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:20:52.534070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574810+0000","last_change":"2023-07-06T20:17:28.572425+0000","last_active":"2023-07-06T20:19:00.574810+0000","last_peered":"2023-07-06T20:19:00.574810+0000","last_clean":"2023-07-06T20:19:00.574810+0000","last_became_active":"2023-07-06T20:17:28.572104+0000","last_became_peered":"2023-07-06T20:17:28.572104+0000","last_unstale":"2023-07-06T20:19:00.574810+0000","last_undegraded":"2023-07-06T20:19:00.574810+0000","last_fullsized":"2023-07-06T20:19:00.574810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:35:59.178149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574524+0000","last_change":"2023-07-06T20:17:28.573814+0000","last_active":"2023-07-06T20:19:00.574524+0000","last_peered":"2023-07-06T20:19:00.574524+0000","last_clean":"2023-07-06T20:19:00.574524+0000","last_became_active":"2023-07-06T20:17:28.573470+0000","last_became_peered":"2023-07-06T20:17:28.573470+0000","last_unstale":"2023-07-06T20:19:00.574524+0000","last_undegraded":"2023-07-06T20:19:00.574524+0000","last_fullsized":"2023-07-06T20:19:00.574524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:03:51.146800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073530+0000","last_change":"2023-07-06T20:17:58.865802+0000","last_active":"2023-07-06T20:19:00.073530+0000","last_peered":"2023-07-06T20:19:00.073530+0000","last_clean":"2023-07-06T20:19:00.073530+0000","last_became_active":"2023-07-06T20:17:58.865290+0000","last_became_peered":"2023-07-06T20:17:58.865290+0000","last_unstale":"2023-07-06T20:19:00.073530+0000","last_undegraded":"2023-07-06T20:19:00.073530+0000","last_fullsized":"2023-07-06T20:19:00.073530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:07:36.214950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574917+0000","last_change":"2023-07-06T20:17:27.603771+0000","last_active":"2023-07-06T20:19:00.574917+0000","last_peered":"2023-07-06T20:19:00.574917+0000","last_clean":"2023-07-06T20:19:00.574917+0000","last_became_active":"2023-07-06T20:17:27.603404+0000","last_became_peered":"2023-07-06T20:17:27.603404+0000","last_unstale":"2023-07-06T20:19:00.574917+0000","last_undegraded":"2023-07-06T20:19:00.574917+0000","last_fullsized":"2023-07-06T20:19:00.574917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:51:42.385000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575838+0000","last_change":"2023-07-06T20:17:28.568371+0000","last_active":"2023-07-06T20:19:00.575838+0000","last_peered":"2023-07-06T20:19:00.575838+0000","last_clean":"2023-07-06T20:19:00.575838+0000","last_became_active":"2023-07-06T20:17:28.568034+0000","last_became_peered":"2023-07-06T20:17:28.568034+0000","last_unstale":"2023-07-06T20:19:00.575838+0000","last_undegraded":"2023-07-06T20:19:00.575838+0000","last_fullsized":"2023-07-06T20:19:00.575838+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:20:30.073418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575967+0000","last_change":"2023-07-06T20:17:27.609833+0000","last_active":"2023-07-06T20:19:00.575967+0000","last_peered":"2023-07-06T20:19:00.575967+0000","last_clean":"2023-07-06T20:19:00.575967+0000","last_became_active":"2023-07-06T20:17:27.609520+0000","last_became_peered":"2023-07-06T20:17:27.609520+0000","last_unstale":"2023-07-06T20:19:00.575967+0000","last_undegraded":"2023-07-06T20:19:00.575967+0000","last_fullsized":"2023-07-06T20:19:00.575967+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:22.478396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575733+0000","last_change":"2023-07-06T20:17:28.574520+0000","last_active":"2023-07-06T20:19:00.575733+0000","last_peered":"2023-07-06T20:19:00.575733+0000","last_clean":"2023-07-06T20:19:00.575733+0000","last_became_active":"2023-07-06T20:17:28.574158+0000","last_became_peered":"2023-07-06T20:17:28.574158+0000","last_unstale":"2023-07-06T20:19:00.575733+0000","last_undegraded":"2023-07-06T20:19:00.575733+0000","last_fullsized":"2023-07-06T20:19:00.575733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:06.841394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646514+0000","last_change":"2023-07-06T20:17:27.601612+0000","last_active":"2023-07-06T20:18:59.646514+0000","last_peered":"2023-07-06T20:18:59.646514+0000","last_clean":"2023-07-06T20:18:59.646514+0000","last_became_active":"2023-07-06T20:17:27.601313+0000","last_became_peered":"2023-07-06T20:17:27.601313+0000","last_unstale":"2023-07-06T20:18:59.646514+0000","last_undegraded":"2023-07-06T20:18:59.646514+0000","last_fullsized":"2023-07-06T20:18:59.646514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:42:58.053499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648800+0000","last_change":"2023-07-06T20:17:28.566389+0000","last_active":"2023-07-06T20:18:59.648800+0000","last_peered":"2023-07-06T20:18:59.648800+0000","last_clean":"2023-07-06T20:18:59.648800+0000","last_became_active":"2023-07-06T20:17:28.566094+0000","last_became_peered":"2023-07-06T20:17:28.566094+0000","last_unstale":"2023-07-06T20:18:59.648800+0000","last_undegraded":"2023-07-06T20:18:59.648800+0000","last_fullsized":"2023-07-06T20:18:59.648800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:43:18.213143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646434+0000","last_change":"2023-07-06T20:17:27.599893+0000","last_active":"2023-07-06T20:18:59.646434+0000","last_peered":"2023-07-06T20:18:59.646434+0000","last_clean":"2023-07-06T20:18:59.646434+0000","last_became_active":"2023-07-06T20:17:27.599544+0000","last_became_peered":"2023-07-06T20:17:27.599544+0000","last_unstale":"2023-07-06T20:18:59.646434+0000","last_undegraded":"2023-07-06T20:18:59.646434+0000","last_fullsized":"2023-07-06T20:18:59.646434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:08:58.904898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703267+0000","last_change":"2023-07-06T20:17:28.575794+0000","last_active":"2023-07-06T20:18:59.703267+0000","last_peered":"2023-07-06T20:18:59.703267+0000","last_clean":"2023-07-06T20:18:59.703267+0000","last_became_active":"2023-07-06T20:17:28.575456+0000","last_became_peered":"2023-07-06T20:17:28.575456+0000","last_unstale":"2023-07-06T20:18:59.703267+0000","last_undegraded":"2023-07-06T20:18:59.703267+0000","last_fullsized":"2023-07-06T20:18:59.703267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:47:11.340543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":128,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:05.714486+0000","last_change":"2023-07-06T20:17:27.596351+0000","last_active":"2023-07-06T20:19:05.714486+0000","last_peered":"2023-07-06T20:19:05.714486+0000","last_clean":"2023-07-06T20:19:05.714486+0000","last_became_active":"2023-07-06T20:17:27.595982+0000","last_became_peered":"2023-07-06T20:17:27.595982+0000","last_unstale":"2023-07-06T20:19:05.714486+0000","last_undegraded":"2023-07-06T20:19:05.714486+0000","last_fullsized":"2023-07-06T20:19:05.714486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:33:14.017909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646053+0000","last_change":"2023-07-06T20:17:28.570018+0000","last_active":"2023-07-06T20:18:59.646053+0000","last_peered":"2023-07-06T20:18:59.646053+0000","last_clean":"2023-07-06T20:18:59.646053+0000","last_became_active":"2023-07-06T20:17:28.569674+0000","last_became_peered":"2023-07-06T20:17:28.569674+0000","last_unstale":"2023-07-06T20:18:59.646053+0000","last_undegraded":"2023-07-06T20:18:59.646053+0000","last_fullsized":"2023-07-06T20:18:59.646053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:24:03.335858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654185+0000","last_change":"2023-07-06T20:17:58.866940+0000","last_active":"2023-07-06T20:18:59.654185+0000","last_peered":"2023-07-06T20:18:59.654185+0000","last_clean":"2023-07-06T20:18:59.654185+0000","last_became_active":"2023-07-06T20:17:58.863435+0000","last_became_peered":"2023-07-06T20:17:58.863435+0000","last_unstale":"2023-07-06T20:18:59.654185+0000","last_undegraded":"2023-07-06T20:18:59.654185+0000","last_fullsized":"2023-07-06T20:18:59.654185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:34:45.362921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703189+0000","last_change":"2023-07-06T20:17:28.575249+0000","last_active":"2023-07-06T20:18:59.703189+0000","last_peered":"2023-07-06T20:18:59.703189+0000","last_clean":"2023-07-06T20:18:59.703189+0000","last_became_active":"2023-07-06T20:17:28.574954+0000","last_became_peered":"2023-07-06T20:17:28.574954+0000","last_unstale":"2023-07-06T20:18:59.703189+0000","last_undegraded":"2023-07-06T20:18:59.703189+0000","last_fullsized":"2023-07-06T20:18:59.703189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:10:03.983955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645825+0000","last_change":"2023-07-06T20:17:27.600552+0000","last_active":"2023-07-06T20:18:59.645825+0000","last_peered":"2023-07-06T20:18:59.645825+0000","last_clean":"2023-07-06T20:18:59.645825+0000","last_became_active":"2023-07-06T20:17:27.600016+0000","last_became_peered":"2023-07-06T20:17:27.600016+0000","last_unstale":"2023-07-06T20:18:59.645825+0000","last_undegraded":"2023-07-06T20:18:59.645825+0000","last_fullsized":"2023-07-06T20:18:59.645825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:08:51.272104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575191+0000","last_change":"2023-07-06T20:17:28.566364+0000","last_active":"2023-07-06T20:19:00.575191+0000","last_peered":"2023-07-06T20:19:00.575191+0000","last_clean":"2023-07-06T20:19:00.575191+0000","last_became_active":"2023-07-06T20:17:28.565474+0000","last_became_peered":"2023-07-06T20:17:28.565474+0000","last_unstale":"2023-07-06T20:19:00.575191+0000","last_undegraded":"2023-07-06T20:19:00.575191+0000","last_fullsized":"2023-07-06T20:19:00.575191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:28:42.265878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.702829+0000","last_change":"2023-07-06T20:17:27.598606+0000","last_active":"2023-07-06T20:18:59.702829+0000","last_peered":"2023-07-06T20:18:59.702829+0000","last_clean":"2023-07-06T20:18:59.702829+0000","last_became_active":"2023-07-06T20:17:27.598292+0000","last_became_peered":"2023-07-06T20:17:27.598292+0000","last_unstale":"2023-07-06T20:18:59.702829+0000","last_undegraded":"2023-07-06T20:18:59.702829+0000","last_fullsized":"2023-07-06T20:18:59.702829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:11:39.786934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574951+0000","last_change":"2023-07-06T20:17:28.571246+0000","last_active":"2023-07-06T20:19:00.574951+0000","last_peered":"2023-07-06T20:19:00.574951+0000","last_clean":"2023-07-06T20:19:00.574951+0000","last_became_active":"2023-07-06T20:17:28.570846+0000","last_became_peered":"2023-07-06T20:17:28.570846+0000","last_unstale":"2023-07-06T20:19:00.574951+0000","last_undegraded":"2023-07-06T20:19:00.574951+0000","last_fullsized":"2023-07-06T20:19:00.574951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:31:31.244148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:03.710217+0000","last_change":"2023-07-06T20:17:27.595879+0000","last_active":"2023-07-06T20:19:03.710217+0000","last_peered":"2023-07-06T20:19:03.710217+0000","last_clean":"2023-07-06T20:19:03.710217+0000","last_became_active":"2023-07-06T20:17:27.595553+0000","last_became_peered":"2023-07-06T20:17:27.595553+0000","last_unstale":"2023-07-06T20:19:03.710217+0000","last_undegraded":"2023-07-06T20:19:03.710217+0000","last_fullsized":"2023-07-06T20:19:03.710217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:01:12.807960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648843+0000","last_change":"2023-07-06T20:17:28.566816+0000","last_active":"2023-07-06T20:18:59.648843+0000","last_peered":"2023-07-06T20:18:59.648843+0000","last_clean":"2023-07-06T20:18:59.648843+0000","last_became_active":"2023-07-06T20:17:28.566565+0000","last_became_peered":"2023-07-06T20:17:28.566565+0000","last_unstale":"2023-07-06T20:18:59.648843+0000","last_undegraded":"2023-07-06T20:18:59.648843+0000","last_fullsized":"2023-07-06T20:18:59.648843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:37:37.504246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.653939+0000","last_change":"2023-07-06T20:17:58.868853+0000","last_active":"2023-07-06T20:18:59.653939+0000","last_peered":"2023-07-06T20:18:59.653939+0000","last_clean":"2023-07-06T20:18:59.653939+0000","last_became_active":"2023-07-06T20:17:58.863304+0000","last_became_peered":"2023-07-06T20:17:58.863304+0000","last_unstale":"2023-07-06T20:18:59.653939+0000","last_undegraded":"2023-07-06T20:18:59.653939+0000","last_fullsized":"2023-07-06T20:18:59.653939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:33:48.601790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575721+0000","last_change":"2023-07-06T20:17:28.572400+0000","last_active":"2023-07-06T20:19:00.575721+0000","last_peered":"2023-07-06T20:19:00.575721+0000","last_clean":"2023-07-06T20:19:00.575721+0000","last_became_active":"2023-07-06T20:17:28.572047+0000","last_became_peered":"2023-07-06T20:17:28.572047+0000","last_unstale":"2023-07-06T20:19:00.575721+0000","last_undegraded":"2023-07-06T20:19:00.575721+0000","last_fullsized":"2023-07-06T20:19:00.575721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:17:30.668701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703472+0000","last_change":"2023-07-06T20:17:27.603224+0000","last_active":"2023-07-06T20:18:59.703472+0000","last_peered":"2023-07-06T20:18:59.703472+0000","last_clean":"2023-07-06T20:18:59.703472+0000","last_became_active":"2023-07-06T20:17:27.602864+0000","last_became_peered":"2023-07-06T20:17:27.602864+0000","last_unstale":"2023-07-06T20:18:59.703472+0000","last_undegraded":"2023-07-06T20:18:59.703472+0000","last_fullsized":"2023-07-06T20:18:59.703472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:23:10.000206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648876+0000","last_change":"2023-07-06T20:17:28.572683+0000","last_active":"2023-07-06T20:18:59.648876+0000","last_peered":"2023-07-06T20:18:59.648876+0000","last_clean":"2023-07-06T20:18:59.648876+0000","last_became_active":"2023-07-06T20:17:28.572352+0000","last_became_peered":"2023-07-06T20:17:28.572352+0000","last_unstale":"2023-07-06T20:18:59.648876+0000","last_undegraded":"2023-07-06T20:18:59.648876+0000","last_fullsized":"2023-07-06T20:18:59.648876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:15:00.435753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'30","reported_seq":247,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:04.711908+0000","last_change":"2023-07-06T20:17:27.597285+0000","last_active":"2023-07-06T20:19:04.711908+0000","last_peered":"2023-07-06T20:19:04.711908+0000","last_clean":"2023-07-06T20:19:04.711908+0000","last_became_active":"2023-07-06T20:17:27.596850+0000","last_became_peered":"2023-07-06T20:17:27.596850+0000","last_unstale":"2023-07-06T20:19:04.711908+0000","last_undegraded":"2023-07-06T20:19:04.711908+0000","last_fullsized":"2023-07-06T20:19:04.711908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:22:39.869484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58776,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":195,"num_read_kb":154,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646322+0000","last_change":"2023-07-06T20:17:28.569855+0000","last_active":"2023-07-06T20:18:59.646322+0000","last_peered":"2023-07-06T20:18:59.646322+0000","last_clean":"2023-07-06T20:18:59.646322+0000","last_became_active":"2023-07-06T20:17:28.569495+0000","last_became_peered":"2023-07-06T20:17:28.569495+0000","last_unstale":"2023-07-06T20:18:59.646322+0000","last_undegraded":"2023-07-06T20:18:59.646322+0000","last_fullsized":"2023-07-06T20:18:59.646322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:59:00.755254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073397+0000","last_change":"2023-07-06T20:17:27.605898+0000","last_active":"2023-07-06T20:19:00.073397+0000","last_peered":"2023-07-06T20:19:00.073397+0000","last_clean":"2023-07-06T20:19:00.073397+0000","last_became_active":"2023-07-06T20:17:27.605605+0000","last_became_peered":"2023-07-06T20:17:27.605605+0000","last_unstale":"2023-07-06T20:19:00.073397+0000","last_undegraded":"2023-07-06T20:19:00.073397+0000","last_fullsized":"2023-07-06T20:19:00.073397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:31:31.780838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645727+0000","last_change":"2023-07-06T20:17:28.579236+0000","last_active":"2023-07-06T20:18:59.645727+0000","last_peered":"2023-07-06T20:18:59.645727+0000","last_clean":"2023-07-06T20:18:59.645727+0000","last_became_active":"2023-07-06T20:17:28.578854+0000","last_became_peered":"2023-07-06T20:17:28.578854+0000","last_unstale":"2023-07-06T20:18:59.645727+0000","last_undegraded":"2023-07-06T20:18:59.645727+0000","last_fullsized":"2023-07-06T20:18:59.645727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:20:05.459582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703207+0000","last_change":"2023-07-06T20:17:27.595724+0000","last_active":"2023-07-06T20:18:59.703207+0000","last_peered":"2023-07-06T20:18:59.703207+0000","last_clean":"2023-07-06T20:18:59.703207+0000","last_became_active":"2023-07-06T20:17:27.595360+0000","last_became_peered":"2023-07-06T20:17:27.595360+0000","last_unstale":"2023-07-06T20:18:59.703207+0000","last_undegraded":"2023-07-06T20:18:59.703207+0000","last_fullsized":"2023-07-06T20:18:59.703207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:46:07.776434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654912+0000","last_change":"2023-07-06T20:17:28.573447+0000","last_active":"2023-07-06T20:18:59.654912+0000","last_peered":"2023-07-06T20:18:59.654912+0000","last_clean":"2023-07-06T20:18:59.654912+0000","last_became_active":"2023-07-06T20:17:28.573161+0000","last_became_peered":"2023-07-06T20:17:28.573161+0000","last_unstale":"2023-07-06T20:18:59.654912+0000","last_undegraded":"2023-07-06T20:18:59.654912+0000","last_fullsized":"2023-07-06T20:18:59.654912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:04:10.288520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":137,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:05.587015+0000","last_change":"2023-07-06T20:19:00.579098+0000","last_active":"2023-07-06T20:19:05.587015+0000","last_peered":"2023-07-06T20:19:05.587015+0000","last_clean":"2023-07-06T20:19:05.587015+0000","last_became_active":"2023-07-06T20:19:00.575486+0000","last_became_peered":"2023-07-06T20:19:00.575486+0000","last_unstale":"2023-07-06T20:19:05.587015+0000","last_undegraded":"2023-07-06T20:19:05.587015+0000","last_fullsized":"2023-07-06T20:19:05.587015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:36:44.887142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1071,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.653581+0000","last_change":"2023-07-06T20:17:28.569745+0000","last_active":"2023-07-06T20:18:59.653581+0000","last_peered":"2023-07-06T20:18:59.653581+0000","last_clean":"2023-07-06T20:18:59.653581+0000","last_became_active":"2023-07-06T20:17:28.569468+0000","last_became_peered":"2023-07-06T20:17:28.569468+0000","last_unstale":"2023-07-06T20:18:59.653581+0000","last_undegraded":"2023-07-06T20:18:59.653581+0000","last_fullsized":"2023-07-06T20:18:59.653581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:04:39.612959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574256+0000","last_change":"2023-07-06T20:17:58.771936+0000","last_active":"2023-07-06T20:19:00.574256+0000","last_peered":"2023-07-06T20:19:00.574256+0000","last_clean":"2023-07-06T20:19:00.574256+0000","last_became_active":"2023-07-06T20:17:58.748196+0000","last_became_peered":"2023-07-06T20:17:58.748196+0000","last_unstale":"2023-07-06T20:19:00.574256+0000","last_undegraded":"2023-07-06T20:19:00.574256+0000","last_fullsized":"2023-07-06T20:19:00.574256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:20:15.168480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703012+0000","last_change":"2023-07-06T20:17:28.575390+0000","last_active":"2023-07-06T20:18:59.703012+0000","last_peered":"2023-07-06T20:18:59.703012+0000","last_clean":"2023-07-06T20:18:59.703012+0000","last_became_active":"2023-07-06T20:17:28.575083+0000","last_became_peered":"2023-07-06T20:17:28.575083+0000","last_unstale":"2023-07-06T20:18:59.703012+0000","last_undegraded":"2023-07-06T20:18:59.703012+0000","last_fullsized":"2023-07-06T20:18:59.703012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:05.579725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646574+0000","last_change":"2023-07-06T20:17:27.600891+0000","last_active":"2023-07-06T20:18:59.646574+0000","last_peered":"2023-07-06T20:18:59.646574+0000","last_clean":"2023-07-06T20:18:59.646574+0000","last_became_active":"2023-07-06T20:17:27.600533+0000","last_became_peered":"2023-07-06T20:17:27.600533+0000","last_unstale":"2023-07-06T20:18:59.646574+0000","last_undegraded":"2023-07-06T20:18:59.646574+0000","last_fullsized":"2023-07-06T20:18:59.646574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:44:00.501767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574645+0000","last_change":"2023-07-06T20:17:58.863073+0000","last_active":"2023-07-06T20:19:00.574645+0000","last_peered":"2023-07-06T20:19:00.574645+0000","last_clean":"2023-07-06T20:19:00.574645+0000","last_became_active":"2023-07-06T20:17:58.862613+0000","last_became_peered":"2023-07-06T20:17:58.862613+0000","last_unstale":"2023-07-06T20:19:00.574645+0000","last_undegraded":"2023-07-06T20:19:00.574645+0000","last_fullsized":"2023-07-06T20:19:00.574645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:21:52.224293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574359+0000","last_change":"2023-07-06T20:17:27.604464+0000","last_active":"2023-07-06T20:19:00.574359+0000","last_peered":"2023-07-06T20:19:00.574359+0000","last_clean":"2023-07-06T20:19:00.574359+0000","last_became_active":"2023-07-06T20:17:27.604036+0000","last_became_peered":"2023-07-06T20:17:27.604036+0000","last_unstale":"2023-07-06T20:19:00.574359+0000","last_undegraded":"2023-07-06T20:19:00.574359+0000","last_fullsized":"2023-07-06T20:19:00.574359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:59:41.175216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654818+0000","last_change":"2023-07-06T20:17:28.577302+0000","last_active":"2023-07-06T20:18:59.654818+0000","last_peered":"2023-07-06T20:18:59.654818+0000","last_clean":"2023-07-06T20:18:59.654818+0000","last_became_active":"2023-07-06T20:17:28.576998+0000","last_became_peered":"2023-07-06T20:17:28.576998+0000","last_unstale":"2023-07-06T20:18:59.654818+0000","last_undegraded":"2023-07-06T20:18:59.654818+0000","last_fullsized":"2023-07-06T20:18:59.654818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:30:35.224494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654890+0000","last_change":"2023-07-06T20:17:27.608391+0000","last_active":"2023-07-06T20:18:59.654890+0000","last_peered":"2023-07-06T20:18:59.654890+0000","last_clean":"2023-07-06T20:18:59.654890+0000","last_became_active":"2023-07-06T20:17:27.608073+0000","last_became_peered":"2023-07-06T20:17:27.608073+0000","last_unstale":"2023-07-06T20:18:59.654890+0000","last_undegraded":"2023-07-06T20:18:59.654890+0000","last_fullsized":"2023-07-06T20:18:59.654890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:58:17.469333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654516+0000","last_change":"2023-07-06T20:17:28.577554+0000","last_active":"2023-07-06T20:18:59.654516+0000","last_peered":"2023-07-06T20:18:59.654516+0000","last_clean":"2023-07-06T20:18:59.654516+0000","last_became_active":"2023-07-06T20:17:28.577208+0000","last_became_peered":"2023-07-06T20:17:28.577208+0000","last_unstale":"2023-07-06T20:18:59.654516+0000","last_undegraded":"2023-07-06T20:18:59.654516+0000","last_fullsized":"2023-07-06T20:18:59.654516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:11:56.614831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703008+0000","last_change":"2023-07-06T20:17:28.571678+0000","last_active":"2023-07-06T20:18:59.703008+0000","last_peered":"2023-07-06T20:18:59.703008+0000","last_clean":"2023-07-06T20:18:59.703008+0000","last_became_active":"2023-07-06T20:17:28.570974+0000","last_became_peered":"2023-07-06T20:17:28.570974+0000","last_unstale":"2023-07-06T20:18:59.703008+0000","last_undegraded":"2023-07-06T20:18:59.703008+0000","last_fullsized":"2023-07-06T20:18:59.703008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:53:32.536581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":121,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:07.589485+0000","last_change":"2023-07-06T20:17:27.607172+0000","last_active":"2023-07-06T20:19:07.589485+0000","last_peered":"2023-07-06T20:19:07.589485+0000","last_clean":"2023-07-06T20:19:07.589485+0000","last_became_active":"2023-07-06T20:17:27.606841+0000","last_became_peered":"2023-07-06T20:17:27.606841+0000","last_unstale":"2023-07-06T20:19:07.589485+0000","last_undegraded":"2023-07-06T20:19:07.589485+0000","last_fullsized":"2023-07-06T20:19:07.589485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:48:15.870982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574630+0000","last_change":"2023-07-06T20:17:28.567707+0000","last_active":"2023-07-06T20:19:00.574630+0000","last_peered":"2023-07-06T20:19:00.574630+0000","last_clean":"2023-07-06T20:19:00.574630+0000","last_became_active":"2023-07-06T20:17:28.567394+0000","last_became_peered":"2023-07-06T20:17:28.567394+0000","last_unstale":"2023-07-06T20:19:00.574630+0000","last_undegraded":"2023-07-06T20:19:00.574630+0000","last_fullsized":"2023-07-06T20:19:00.574630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:58:40.539787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.651620+0000","last_change":"2023-07-06T20:17:27.595746+0000","last_active":"2023-07-06T20:18:59.651620+0000","last_peered":"2023-07-06T20:18:59.651620+0000","last_clean":"2023-07-06T20:18:59.651620+0000","last_became_active":"2023-07-06T20:17:27.595039+0000","last_became_peered":"2023-07-06T20:17:27.595039+0000","last_unstale":"2023-07-06T20:18:59.651620+0000","last_undegraded":"2023-07-06T20:18:59.651620+0000","last_fullsized":"2023-07-06T20:18:59.651620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:00:27.347076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654306+0000","last_change":"2023-07-06T20:16:23.542324+0000","last_active":"2023-07-06T20:18:59.654306+0000","last_peered":"2023-07-06T20:18:59.654306+0000","last_clean":"2023-07-06T20:18:59.654306+0000","last_became_active":"2023-07-06T20:16:23.541832+0000","last_became_peered":"2023-07-06T20:16:23.541832+0000","last_unstale":"2023-07-06T20:18:59.654306+0000","last_undegraded":"2023-07-06T20:18:59.654306+0000","last_fullsized":"2023-07-06T20:18:59.654306+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_clean_scrub_stamp":"2023-07-06T20:13:14.072924+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:08:55.801420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645653+0000","last_change":"2023-07-06T20:17:28.577309+0000","last_active":"2023-07-06T20:18:59.645653+0000","last_peered":"2023-07-06T20:18:59.645653+0000","last_clean":"2023-07-06T20:18:59.645653+0000","last_became_active":"2023-07-06T20:17:28.576962+0000","last_became_peered":"2023-07-06T20:17:28.576962+0000","last_unstale":"2023-07-06T20:18:59.645653+0000","last_undegraded":"2023-07-06T20:18:59.645653+0000","last_fullsized":"2023-07-06T20:18:59.645653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:53.192162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654524+0000","last_change":"2023-07-06T20:17:27.601487+0000","last_active":"2023-07-06T20:18:59.654524+0000","last_peered":"2023-07-06T20:18:59.654524+0000","last_clean":"2023-07-06T20:18:59.654524+0000","last_became_active":"2023-07-06T20:17:27.601195+0000","last_became_peered":"2023-07-06T20:17:27.601195+0000","last_unstale":"2023-07-06T20:18:59.654524+0000","last_undegraded":"2023-07-06T20:18:59.654524+0000","last_fullsized":"2023-07-06T20:18:59.654524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:01:12.039199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646358+0000","last_change":"2023-07-06T20:17:27.610374+0000","last_active":"2023-07-06T20:18:59.646358+0000","last_peered":"2023-07-06T20:18:59.646358+0000","last_clean":"2023-07-06T20:18:59.646358+0000","last_became_active":"2023-07-06T20:17:27.610104+0000","last_became_peered":"2023-07-06T20:17:27.610104+0000","last_unstale":"2023-07-06T20:18:59.646358+0000","last_undegraded":"2023-07-06T20:18:59.646358+0000","last_fullsized":"2023-07-06T20:18:59.646358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:51:41.440974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645823+0000","last_change":"2023-07-06T20:17:28.570146+0000","last_active":"2023-07-06T20:18:59.645823+0000","last_peered":"2023-07-06T20:18:59.645823+0000","last_clean":"2023-07-06T20:18:59.645823+0000","last_became_active":"2023-07-06T20:17:28.569341+0000","last_became_peered":"2023-07-06T20:17:28.569341+0000","last_unstale":"2023-07-06T20:18:59.645823+0000","last_undegraded":"2023-07-06T20:18:59.645823+0000","last_fullsized":"2023-07-06T20:18:59.645823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:25:56.302477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:02.380205+0000","last_change":"2023-07-06T20:17:27.604630+0000","last_active":"2023-07-06T20:19:02.380205+0000","last_peered":"2023-07-06T20:19:02.380205+0000","last_clean":"2023-07-06T20:19:02.380205+0000","last_became_active":"2023-07-06T20:17:27.604118+0000","last_became_peered":"2023-07-06T20:17:27.604118+0000","last_unstale":"2023-07-06T20:19:02.380205+0000","last_undegraded":"2023-07-06T20:19:02.380205+0000","last_fullsized":"2023-07-06T20:19:02.380205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:04:38.195720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575728+0000","last_change":"2023-07-06T20:17:28.571753+0000","last_active":"2023-07-06T20:19:00.575728+0000","last_peered":"2023-07-06T20:19:00.575728+0000","last_clean":"2023-07-06T20:19:00.575728+0000","last_became_active":"2023-07-06T20:17:28.571073+0000","last_became_peered":"2023-07-06T20:17:28.571073+0000","last_unstale":"2023-07-06T20:19:00.575728+0000","last_undegraded":"2023-07-06T20:19:00.575728+0000","last_fullsized":"2023-07-06T20:19:00.575728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:49:51.350250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.651893+0000","last_change":"2023-07-06T20:17:27.594354+0000","last_active":"2023-07-06T20:18:59.651893+0000","last_peered":"2023-07-06T20:18:59.651893+0000","last_clean":"2023-07-06T20:18:59.651893+0000","last_became_active":"2023-07-06T20:17:27.594042+0000","last_became_peered":"2023-07-06T20:17:27.594042+0000","last_unstale":"2023-07-06T20:18:59.651893+0000","last_undegraded":"2023-07-06T20:18:59.651893+0000","last_fullsized":"2023-07-06T20:18:59.651893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:33:21.518458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575621+0000","last_change":"2023-07-06T20:17:28.576170+0000","last_active":"2023-07-06T20:19:00.575621+0000","last_peered":"2023-07-06T20:19:00.575621+0000","last_clean":"2023-07-06T20:19:00.575621+0000","last_became_active":"2023-07-06T20:17:28.575493+0000","last_became_peered":"2023-07-06T20:17:28.575493+0000","last_unstale":"2023-07-06T20:19:00.575621+0000","last_undegraded":"2023-07-06T20:19:00.575621+0000","last_fullsized":"2023-07-06T20:19:00.575621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:12:05.377174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648489+0000","last_change":"2023-07-06T20:17:27.602984+0000","last_active":"2023-07-06T20:18:59.648489+0000","last_peered":"2023-07-06T20:18:59.648489+0000","last_clean":"2023-07-06T20:18:59.648489+0000","last_became_active":"2023-07-06T20:17:27.602720+0000","last_became_peered":"2023-07-06T20:17:27.602720+0000","last_unstale":"2023-07-06T20:18:59.648489+0000","last_undegraded":"2023-07-06T20:18:59.648489+0000","last_fullsized":"2023-07-06T20:18:59.648489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:59:43.619260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646022+0000","last_change":"2023-07-06T20:17:28.565954+0000","last_active":"2023-07-06T20:18:59.646022+0000","last_peered":"2023-07-06T20:18:59.646022+0000","last_clean":"2023-07-06T20:18:59.646022+0000","last_became_active":"2023-07-06T20:17:28.565300+0000","last_became_peered":"2023-07-06T20:17:28.565300+0000","last_unstale":"2023-07-06T20:18:59.646022+0000","last_undegraded":"2023-07-06T20:18:59.646022+0000","last_fullsized":"2023-07-06T20:18:59.646022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:57:19.951265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575343+0000","last_change":"2023-07-06T20:17:27.597859+0000","last_active":"2023-07-06T20:19:00.575343+0000","last_peered":"2023-07-06T20:19:00.575343+0000","last_clean":"2023-07-06T20:19:00.575343+0000","last_became_active":"2023-07-06T20:17:27.597479+0000","last_became_peered":"2023-07-06T20:17:27.597479+0000","last_unstale":"2023-07-06T20:19:00.575343+0000","last_undegraded":"2023-07-06T20:19:00.575343+0000","last_fullsized":"2023-07-06T20:19:00.575343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:44:33.032535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574266+0000","last_change":"2023-07-06T20:17:28.573069+0000","last_active":"2023-07-06T20:19:00.574266+0000","last_peered":"2023-07-06T20:19:00.574266+0000","last_clean":"2023-07-06T20:19:00.574266+0000","last_became_active":"2023-07-06T20:17:28.572742+0000","last_became_peered":"2023-07-06T20:17:28.572742+0000","last_unstale":"2023-07-06T20:19:00.574266+0000","last_undegraded":"2023-07-06T20:19:00.574266+0000","last_fullsized":"2023-07-06T20:19:00.574266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:00:57.440359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073240+0000","last_change":"2023-07-06T20:17:27.599259+0000","last_active":"2023-07-06T20:19:00.073240+0000","last_peered":"2023-07-06T20:19:00.073240+0000","last_clean":"2023-07-06T20:19:00.073240+0000","last_became_active":"2023-07-06T20:17:27.598883+0000","last_became_peered":"2023-07-06T20:17:27.598883+0000","last_unstale":"2023-07-06T20:19:00.073240+0000","last_undegraded":"2023-07-06T20:19:00.073240+0000","last_fullsized":"2023-07-06T20:19:00.073240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:06:44.671018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575340+0000","last_change":"2023-07-06T20:17:28.573874+0000","last_active":"2023-07-06T20:19:00.575340+0000","last_peered":"2023-07-06T20:19:00.575340+0000","last_clean":"2023-07-06T20:19:00.575340+0000","last_became_active":"2023-07-06T20:17:28.573379+0000","last_became_peered":"2023-07-06T20:17:28.573379+0000","last_unstale":"2023-07-06T20:19:00.575340+0000","last_undegraded":"2023-07-06T20:19:00.575340+0000","last_fullsized":"2023-07-06T20:19:00.575340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:26:16.579909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646204+0000","last_change":"2023-07-06T20:17:27.601474+0000","last_active":"2023-07-06T20:18:59.646204+0000","last_peered":"2023-07-06T20:18:59.646204+0000","last_clean":"2023-07-06T20:18:59.646204+0000","last_became_active":"2023-07-06T20:17:27.601099+0000","last_became_peered":"2023-07-06T20:17:27.601099+0000","last_unstale":"2023-07-06T20:18:59.646204+0000","last_undegraded":"2023-07-06T20:18:59.646204+0000","last_fullsized":"2023-07-06T20:18:59.646204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:36:33.224709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654754+0000","last_change":"2023-07-06T20:17:28.571170+0000","last_active":"2023-07-06T20:18:59.654754+0000","last_peered":"2023-07-06T20:18:59.654754+0000","last_clean":"2023-07-06T20:18:59.654754+0000","last_became_active":"2023-07-06T20:17:28.570874+0000","last_became_peered":"2023-07-06T20:17:28.570874+0000","last_unstale":"2023-07-06T20:18:59.654754+0000","last_undegraded":"2023-07-06T20:18:59.654754+0000","last_fullsized":"2023-07-06T20:18:59.654754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:05.977773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073444+0000","last_change":"2023-07-06T20:17:27.603604+0000","last_active":"2023-07-06T20:19:00.073444+0000","last_peered":"2023-07-06T20:19:00.073444+0000","last_clean":"2023-07-06T20:19:00.073444+0000","last_became_active":"2023-07-06T20:17:27.603015+0000","last_became_peered":"2023-07-06T20:17:27.603015+0000","last_unstale":"2023-07-06T20:19:00.073444+0000","last_undegraded":"2023-07-06T20:19:00.073444+0000","last_fullsized":"2023-07-06T20:19:00.073444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:26:02.215516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654631+0000","last_change":"2023-07-06T20:17:28.567906+0000","last_active":"2023-07-06T20:18:59.654631+0000","last_peered":"2023-07-06T20:18:59.654631+0000","last_clean":"2023-07-06T20:18:59.654631+0000","last_became_active":"2023-07-06T20:17:28.567509+0000","last_became_peered":"2023-07-06T20:17:28.567509+0000","last_unstale":"2023-07-06T20:18:59.654631+0000","last_undegraded":"2023-07-06T20:18:59.654631+0000","last_fullsized":"2023-07-06T20:18:59.654631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:51:09.883831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703298+0000","last_change":"2023-07-06T20:17:27.598323+0000","last_active":"2023-07-06T20:18:59.703298+0000","last_peered":"2023-07-06T20:18:59.703298+0000","last_clean":"2023-07-06T20:18:59.703298+0000","last_became_active":"2023-07-06T20:17:27.598032+0000","last_became_peered":"2023-07-06T20:17:27.598032+0000","last_unstale":"2023-07-06T20:18:59.703298+0000","last_undegraded":"2023-07-06T20:18:59.703298+0000","last_fullsized":"2023-07-06T20:18:59.703298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:05.424357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654137+0000","last_change":"2023-07-06T20:17:28.571756+0000","last_active":"2023-07-06T20:18:59.654137+0000","last_peered":"2023-07-06T20:18:59.654137+0000","last_clean":"2023-07-06T20:18:59.654137+0000","last_became_active":"2023-07-06T20:17:28.571440+0000","last_became_peered":"2023-07-06T20:17:28.571440+0000","last_unstale":"2023-07-06T20:18:59.654137+0000","last_undegraded":"2023-07-06T20:18:59.654137+0000","last_fullsized":"2023-07-06T20:18:59.654137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:48.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645836+0000","last_change":"2023-07-06T20:17:27.602620+0000","last_active":"2023-07-06T20:18:59.645836+0000","last_peered":"2023-07-06T20:18:59.645836+0000","last_clean":"2023-07-06T20:18:59.645836+0000","last_became_active":"2023-07-06T20:17:27.602224+0000","last_became_peered":"2023-07-06T20:17:27.602224+0000","last_unstale":"2023-07-06T20:18:59.645836+0000","last_undegraded":"2023-07-06T20:18:59.645836+0000","last_fullsized":"2023-07-06T20:18:59.645836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:18:17.166344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654194+0000","last_change":"2023-07-06T20:17:28.569203+0000","last_active":"2023-07-06T20:18:59.654194+0000","last_peered":"2023-07-06T20:18:59.654194+0000","last_clean":"2023-07-06T20:18:59.654194+0000","last_became_active":"2023-07-06T20:17:28.568948+0000","last_became_peered":"2023-07-06T20:17:28.568948+0000","last_unstale":"2023-07-06T20:18:59.654194+0000","last_undegraded":"2023-07-06T20:18:59.654194+0000","last_fullsized":"2023-07-06T20:18:59.654194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:49:16.105664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574493+0000","last_change":"2023-07-06T20:17:28.568829+0000","last_active":"2023-07-06T20:19:00.574493+0000","last_peered":"2023-07-06T20:19:00.574493+0000","last_clean":"2023-07-06T20:19:00.574493+0000","last_became_active":"2023-07-06T20:17:28.568511+0000","last_became_peered":"2023-07-06T20:17:28.568511+0000","last_unstale":"2023-07-06T20:19:00.574493+0000","last_undegraded":"2023-07-06T20:19:00.574493+0000","last_fullsized":"2023-07-06T20:19:00.574493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:08.295954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073259+0000","last_change":"2023-07-06T20:17:58.863188+0000","last_active":"2023-07-06T20:19:00.073259+0000","last_peered":"2023-07-06T20:19:00.073259+0000","last_clean":"2023-07-06T20:19:00.073259+0000","last_became_active":"2023-07-06T20:17:58.862743+0000","last_became_peered":"2023-07-06T20:17:58.862743+0000","last_unstale":"2023-07-06T20:19:00.073259+0000","last_undegraded":"2023-07-06T20:19:00.073259+0000","last_fullsized":"2023-07-06T20:19:00.073259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:07:20.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073191+0000","last_change":"2023-07-06T20:17:28.572264+0000","last_active":"2023-07-06T20:19:00.073191+0000","last_peered":"2023-07-06T20:19:00.073191+0000","last_clean":"2023-07-06T20:19:00.073191+0000","last_became_active":"2023-07-06T20:17:28.571914+0000","last_became_peered":"2023-07-06T20:17:28.571914+0000","last_unstale":"2023-07-06T20:19:00.073191+0000","last_undegraded":"2023-07-06T20:19:00.073191+0000","last_fullsized":"2023-07-06T20:19:00.073191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:01:02.267552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575399+0000","last_change":"2023-07-06T20:17:28.581374+0000","last_active":"2023-07-06T20:19:00.575399+0000","last_peered":"2023-07-06T20:19:00.575399+0000","last_clean":"2023-07-06T20:19:00.575399+0000","last_became_active":"2023-07-06T20:17:28.580995+0000","last_became_peered":"2023-07-06T20:17:28.580995+0000","last_unstale":"2023-07-06T20:19:00.575399+0000","last_undegraded":"2023-07-06T20:19:00.575399+0000","last_fullsized":"2023-07-06T20:19:00.575399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:08:18.718991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645952+0000","last_change":"2023-07-06T20:17:28.578926+0000","last_active":"2023-07-06T20:18:59.645952+0000","last_peered":"2023-07-06T20:18:59.645952+0000","last_clean":"2023-07-06T20:18:59.645952+0000","last_became_active":"2023-07-06T20:17:28.578620+0000","last_became_peered":"2023-07-06T20:17:28.578620+0000","last_unstale":"2023-07-06T20:18:59.645952+0000","last_undegraded":"2023-07-06T20:18:59.645952+0000","last_fullsized":"2023-07-06T20:18:59.645952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:17:06.451679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574477+0000","last_change":"2023-07-06T20:17:58.864937+0000","last_active":"2023-07-06T20:19:00.574477+0000","last_peered":"2023-07-06T20:19:00.574477+0000","last_clean":"2023-07-06T20:19:00.574477+0000","last_became_active":"2023-07-06T20:17:58.864457+0000","last_became_peered":"2023-07-06T20:17:58.864457+0000","last_unstale":"2023-07-06T20:19:00.574477+0000","last_undegraded":"2023-07-06T20:19:00.574477+0000","last_fullsized":"2023-07-06T20:19:00.574477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:53:40.789746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073661+0000","last_change":"2023-07-06T20:17:28.567965+0000","last_active":"2023-07-06T20:19:00.073661+0000","last_peered":"2023-07-06T20:19:00.073661+0000","last_clean":"2023-07-06T20:19:00.073661+0000","last_became_active":"2023-07-06T20:17:28.567618+0000","last_became_peered":"2023-07-06T20:17:28.567618+0000","last_unstale":"2023-07-06T20:19:00.073661+0000","last_undegraded":"2023-07-06T20:19:00.073661+0000","last_fullsized":"2023-07-06T20:19:00.073661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:20:53.386538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575649+0000","last_change":"2023-07-06T20:17:28.571934+0000","last_active":"2023-07-06T20:19:00.575649+0000","last_peered":"2023-07-06T20:19:00.575649+0000","last_clean":"2023-07-06T20:19:00.575649+0000","last_became_active":"2023-07-06T20:17:28.571472+0000","last_became_peered":"2023-07-06T20:17:28.571472+0000","last_unstale":"2023-07-06T20:19:00.575649+0000","last_undegraded":"2023-07-06T20:19:00.575649+0000","last_fullsized":"2023-07-06T20:19:00.575649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:22.858255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.702897+0000","last_change":"2023-07-06T20:17:28.571904+0000","last_active":"2023-07-06T20:18:59.702897+0000","last_peered":"2023-07-06T20:18:59.702897+0000","last_clean":"2023-07-06T20:18:59.702897+0000","last_became_active":"2023-07-06T20:17:28.571375+0000","last_became_peered":"2023-07-06T20:17:28.571375+0000","last_unstale":"2023-07-06T20:18:59.702897+0000","last_undegraded":"2023-07-06T20:18:59.702897+0000","last_fullsized":"2023-07-06T20:18:59.702897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:32:15.205545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575210+0000","last_change":"2023-07-06T20:17:28.569139+0000","last_active":"2023-07-06T20:19:00.575210+0000","last_peered":"2023-07-06T20:19:00.575210+0000","last_clean":"2023-07-06T20:19:00.575210+0000","last_became_active":"2023-07-06T20:17:28.568828+0000","last_became_peered":"2023-07-06T20:17:28.568828+0000","last_unstale":"2023-07-06T20:19:00.575210+0000","last_undegraded":"2023-07-06T20:19:00.575210+0000","last_fullsized":"2023-07-06T20:19:00.575210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:32.012178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648570+0000","last_change":"2023-07-06T20:17:28.562352+0000","last_active":"2023-07-06T20:18:59.648570+0000","last_peered":"2023-07-06T20:18:59.648570+0000","last_clean":"2023-07-06T20:18:59.648570+0000","last_became_active":"2023-07-06T20:17:28.562128+0000","last_became_peered":"2023-07-06T20:17:28.562128+0000","last_unstale":"2023-07-06T20:18:59.648570+0000","last_undegraded":"2023-07-06T20:18:59.648570+0000","last_fullsized":"2023-07-06T20:18:59.648570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:18:12.118763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645970+0000","last_change":"2023-07-06T20:17:28.578940+0000","last_active":"2023-07-06T20:18:59.645970+0000","last_peered":"2023-07-06T20:18:59.645970+0000","last_clean":"2023-07-06T20:18:59.645970+0000","last_became_active":"2023-07-06T20:17:28.578660+0000","last_became_peered":"2023-07-06T20:17:28.578660+0000","last_unstale":"2023-07-06T20:18:59.645970+0000","last_undegraded":"2023-07-06T20:18:59.645970+0000","last_fullsized":"2023-07-06T20:18:59.645970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:00:23.290920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574415+0000","last_change":"2023-07-06T20:17:28.573964+0000","last_active":"2023-07-06T20:19:00.574415+0000","last_peered":"2023-07-06T20:19:00.574415+0000","last_clean":"2023-07-06T20:19:00.574415+0000","last_became_active":"2023-07-06T20:17:28.573618+0000","last_became_peered":"2023-07-06T20:17:28.573618+0000","last_unstale":"2023-07-06T20:19:00.574415+0000","last_undegraded":"2023-07-06T20:19:00.574415+0000","last_fullsized":"2023-07-06T20:19:00.574415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:06:11.474673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63664,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":589,"num_read_kb":362,"num_write":162,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1183,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":202062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449121,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":462935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.72599999999999998,"15min":0.67300000000000004},"min":{"1min":0.45600000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":4.968,"5min":4.968,"15min":4.968},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.04,"5min":0.71899999999999997,"15min":0.66500000000000004},"min":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":5.0019999999999998,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63800000000000001,"15min":0.629},"min":{"1min":0.45300000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.79700000000000004,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.36599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.72399999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69699999999999995,"15min":0.69099999999999995},"min":{"1min":0.48899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.622,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":1.006,"15min":1.006},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66400000000000003,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92800000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.47699999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90400000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.89100000000000001,"5min":1.097,"15min":1.097},"last":0.752}]},{"osd":7,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78900000000000003,"15min":0.79500000000000004},"min":{"1min":0.53700000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.98599999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68000000000000005}]},{"osd":8,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.80800000000000005}]},{"osd":9,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.60999999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.96799999999999997,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.89800000000000002,"15min":0.86799999999999999},"min":{"1min":0.67000000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":0.84399999999999997}]},{"osd":10,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78100000000000003,"15min":0.78800000000000003},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88200000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.66200000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97099999999999997,"5min":1.208,"15min":1.208},"last":0.65400000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1616,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1654784,"data_stored":1112186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.66700000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.434,"15min":0.436},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55000000000000004,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67600000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.499}]},{"osd":2,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86599999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.46200000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.624},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.32500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67700000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.496,"5min":0.439,"15min":0.439},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67500000000000004,"15min":0.67300000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.751}]},{"osd":7,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.48099999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.875,"5min":0.998,"15min":0.998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74299999999999999,"15min":0.748},"min":{"1min":0.57099999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.71899999999999997}]},{"osd":9,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.94699999999999995,"15min":0.98099999999999998},"min":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93999999999999995,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.91100000000000003,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.81399999999999995}]},{"osd":11,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73999999999999999,"15min":0.73899999999999999},"min":{"1min":0.57699999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.0049999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.872},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78300000000000003,"15min":0.78200000000000003},"min":{"1min":0.54200000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.85499999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":461676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.49299999999999999,"15min":0.503},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.93600000000000005,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83499999999999996,"5min":5.7919999999999998,"15min":5.7919999999999998},"last":0.435}]},{"osd":1,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.69099999999999995,"15min":0.70999999999999996},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89600000000000002,"5min":1.202,"15min":1.202},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.66800000000000004,"15min":0.68500000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.81999999999999995}]},{"osd":2,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.98499999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69699999999999995,"15min":0.71099999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.72699999999999998,"15min":0.66900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2729999999999999,"5min":16.628,"15min":16.628},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.083,"5min":11.332000000000001,"15min":11.332000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69399999999999995,"15min":0.70499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.73599999999999999,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.78900000000000003,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.80700000000000005,"15min":0.83299999999999996},"min":{"1min":0.51400000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91200000000000003,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79000000000000004,"15min":0.81599999999999995},"min":{"1min":0.58999999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.016,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.76500000000000001,"15min":0.78500000000000003},"min":{"1min":0.54100000000000004,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.115,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.77500000000000002,"15min":0.82499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84499999999999997,"15min":0.88},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.85899999999999999,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.86499999999999999,"15min":0.90400000000000003},"min":{"1min":0.625,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.054,"5min":1.339,"15min":1.339},"last":0.76700000000000002}]},{"osd":8,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.88200000000000001},"min":{"1min":0.504,"5min":0.496,"15min":0.496},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.751},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.877,"15min":0.89600000000000002},"min":{"1min":0.60299999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2230000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":0.72799999999999998}]},{"osd":10,"last update":"Thu Jul 6 20:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.85799999999999998,"15min":0.84999999999999998},"min":{"1min":0.68700000000000006,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.80500000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103213,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":461203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58699999999999997}]},{"osd":1,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.60099999999999998,"15min":0.57599999999999996},"min":{"1min":0.54800000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.48199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81000000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.45900000000000002,"15min":0.44800000000000001},"min":{"1min":0.34300000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.48099999999999998,"15min":0.47199999999999998},"min":{"1min":0.32400000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71299999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.48099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67700000000000005},"min":{"1min":0.54700000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.629,"15min":0.61399999999999999},"min":{"1min":0.48099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.66800000000000004,"15min":0.64700000000000002},"min":{"1min":0.621,"5min":0.433,"15min":0.433},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.89300000000000002}]},{"osd":5,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.622,"15min":0.60099999999999998},"min":{"1min":0.40200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.97999999999999998,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.74299999999999999,"15min":0.72099999999999997},"min":{"1min":0.64700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.73799999999999999,"15min":0.71999999999999997},"min":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.66100000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.0489999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.86799999999999999,"15min":0.90600000000000003},"min":{"1min":0.56499999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.091,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.95999999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.78800000000000003,"15min":0.74399999999999999},"min":{"1min":0.628,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.044},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.80500000000000005,"15min":0.76900000000000002},"min":{"1min":0.64400000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.97199999999999998}]},{"osd":10,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.76400000000000001,"15min":0.73899999999999999},"min":{"1min":0.69999999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.022},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.77300000000000002,"15min":0.75700000000000001},"min":{"1min":0.70399999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.0049999999999999}]},{"osd":11,"last update":"Thu Jul 6 20:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.996}]}]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1528,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1564672,"data_stored":1053036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51300000000000001,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85299999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.39400000000000002,"15min":0.38400000000000001},"min":{"1min":0.34699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.41299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.625,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.374}]},{"osd":2,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.40400000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80400000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.68000000000000005,"15min":0.622},"min":{"1min":0.40100000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.034,"5min":0.67300000000000004,"15min":0.59599999999999997},"min":{"1min":0.47899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.048,"5min":6.048,"15min":6.048},"last":0.53700000000000003}]},{"osd":4,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.32300000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.502},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.499,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59299999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.44700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.755,"15min":0.68100000000000005},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":8.1579999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.77000000000000002,"15min":0.70299999999999996},"min":{"1min":0.48799999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":8.1980000000000004,"5min":8.1980000000000004,"15min":8.1980000000000004},"last":0.66800000000000004}]},{"osd":8,"last update":"Thu Jul 6 20:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64600000000000002,"15min":0.63},"min":{"1min":0.47799999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.66600000000000004,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90500000000000003,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65400000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70599999999999996,"15min":0.70299999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71999999999999997,"15min":0.70399999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95099999999999996,"5min":1.419,"15min":1.419},"last":0.64000000000000001}]},{"osd":10,"last update":"Thu Jul 6 20:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.55300000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":1.014,"15min":1.014},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.54200000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54800000000000004}]},{"osd":11,"last update":"Thu Jul 6 20:18:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.755},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.73999999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":461071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49099999999999999,"15min":0.47399999999999998},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.91600000000000004,"5min":1.552,"15min":1.552},"last":0.875},{"interface":"front","average":{"1min":0.439,"5min":0.496,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66400000000000003,"5min":1.389,"15min":1.389},"last":0.53300000000000003}]},{"osd":1,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.71599999999999997,"15min":0.66800000000000004},"min":{"1min":0.55600000000000005,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.66800000000000004,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91000000000000003,"5min":1.089,"15min":1.089},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.68500000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.755},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94699999999999995,"5min":1.04,"15min":1.04},"last":0.93700000000000006}]},{"osd":3,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.60199999999999998,"15min":0.622},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.98999999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.95499999999999996,"5min":1.254,"15min":1.254},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86399999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.623}]},{"osd":5,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72999999999999998,"15min":0.76000000000000001},"min":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0109999999999999,"5min":1.393,"15min":1.393},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.75600000000000001,"15min":0.76100000000000001},"min":{"1min":0.505,"5min":0.496,"15min":0.496},"max":{"1min":0.90600000000000003,"5min":1.131,"15min":1.131},"last":1.0229999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.79400000000000004,"15min":0.77800000000000002},"min":{"1min":0.55100000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.008},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.78700000000000003,"15min":0.78500000000000003},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.95599999999999996,"5min":1.274,"15min":1.274},"last":0.68799999999999994}]},{"osd":8,"last update":"Thu Jul 6 20:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81399999999999995,"15min":0.84399999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.077,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80300000000000005,"15min":0.82299999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0029999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Thu Jul 6 20:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.56299999999999994,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.05},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.89000000000000001,"15min":0.90200000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":1.091}]},{"osd":10,"last update":"Thu Jul 6 20:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92200000000000004,"15min":0.91400000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":3.8410000000000002,"5min":3.8410000000000002,"15min":3.8410000000000002},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.88100000000000001,"15min":0.89700000000000002},"min":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0269999999999999,"5min":1.244,"15min":1.244},"last":1.0780000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":520058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.495},"min":{"1min":0.32700000000000001,"5min":0.318,"15min":0.248},"max":{"1min":0.82199999999999995,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.53100000000000003},"min":{"1min":0.44600000000000001,"5min":0.34499999999999997,"15min":0.248},"max":{"1min":0.86099999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56899999999999995,"15min":0.51300000000000001},"min":{"1min":0.41599999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.93600000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.59399999999999997,"15min":0.52000000000000002},"min":{"1min":0.60699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94999999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.997}]},{"osd":3,"last update":"Thu Jul 6 20:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60499999999999998,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.90200000000000002,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.56499999999999995},"min":{"1min":0.42499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89000000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61499999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.42899999999999999,"15min":0.39300000000000002},"min":{"1min":0.33800000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.46999999999999997,"15min":0.42599999999999999},"min":{"1min":0.33500000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jul 6 20:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.60099999999999998,"15min":0.54900000000000004},"min":{"1min":0.55300000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90000000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.64100000000000001,"15min":0.57499999999999996},"min":{"1min":0.58999999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0289999999999999,"5min":1.042,"15min":1.042},"last":0.92900000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69699999999999995,"15min":0.66300000000000003},"min":{"1min":0.52400000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.437,"5min":0.372,"15min":0.372},"max":{"1min":0.874,"5min":1.653,"15min":1.653},"last":0.97999999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.624,"15min":0.59199999999999997},"min":{"1min":0.42399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.59599999999999997,"15min":0.55500000000000005},"min":{"1min":0.5,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.0309999999999999}]},{"osd":8,"last update":"Thu Jul 6 20:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.69999999999999996,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96999999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.77000000000000002,"15min":0.73399999999999999},"min":{"1min":0.55100000000000005,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":1.101}]},{"osd":9,"last update":"Thu Jul 6 20:19:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.76600000000000001,"15min":0.746},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0129999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78900000000000003,"15min":0.77100000000000002},"min":{"1min":0.56000000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0189999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.014}]},{"osd":10,"last update":"Thu Jul 6 20:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76900000000000002,"15min":0.76100000000000001},"min":{"1min":0.47999999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.82799999999999996,"15min":0.81599999999999995},"min":{"1min":0.67300000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.86099999999999999}]},{"osd":11,"last update":"Thu Jul 6 20:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":462336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.317,"15min":0.317},"max":{"1min":4.1130000000000004,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.628,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.6400000000000001,"5min":4.1379999999999999,"15min":4.1379999999999999},"last":0.499}]},{"osd":2,"last update":"Thu Jul 6 20:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.63700000000000001,"15min":0.58999999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91100000000000003,"5min":1.085,"15min":1.085},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.66000000000000003,"15min":0.58599999999999997},"min":{"1min":0.44600000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97199999999999998,"5min":1.258,"15min":1.258},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.73799999999999999},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0049999999999999,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jul 6 20:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67500000000000004,"15min":0.64500000000000002},"min":{"1min":0.51300000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91000000000000003,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.46800000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95599999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.69299999999999995},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.03,"5min":1.131,"15min":1.131},"last":0.505},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79700000000000004,"15min":0.79600000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.179,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.84399999999999997}]},{"osd":8,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.78900000000000003,"15min":0.78600000000000003},"min":{"1min":0.54800000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.7650000000000001,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79700000000000004,"15min":0.81899999999999995},"min":{"1min":0.54100000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.073,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.65400000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.79000000000000004,"15min":0.82199999999999995},"min":{"1min":0.51100000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97799999999999998,"5min":1.161,"15min":1.161},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77500000000000002,"15min":0.79100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.7130000000000001,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.63100000000000001}]},{"osd":10,"last update":"Thu Jul 6 20:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.877,"15min":0.879},"min":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.681,"5min":2.681,"15min":2.681},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.86699999999999999,"15min":0.85499999999999998},"min":{"1min":0.54400000000000004,"5min":0.498,"15min":0.498},"max":{"1min":3.9630000000000001,"5min":3.9630000000000001,"15min":3.9630000000000001},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.83299999999999996,"15min":0.81399999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.9209999999999998,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.111,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.89800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":1044480,"data_stored":527046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.33500000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.89400000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84099999999999997,"5min":0.876,"15min":0.876},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.39800000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.87,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":1,"15min":1},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83499999999999996,"5min":0.995,"15min":0.995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.50600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jul 6 20:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49299999999999999,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77200000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.34499999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78400000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66500000000000004,"15min":0.64300000000000002},"min":{"1min":0.53100000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.97599999999999998,"5min":1.056,"15min":1.056},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.70599999999999996,"15min":0.66000000000000003},"min":{"1min":0.60399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0149999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.92700000000000005}]},{"osd":7,"last update":"Thu Jul 6 20:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.624},"min":{"1min":0.435,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81899999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91000000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.432,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.89100000000000001}]},{"osd":9,"last update":"Thu Jul 6 20:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.77300000000000002,"15min":0.76200000000000001},"min":{"1min":0.48099999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.81399999999999995,"15min":0.79600000000000004},"min":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.98099999999999998}]},{"osd":10,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.746,"15min":0.73499999999999999},"min":{"1min":0.56299999999999994,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.98999999999999999}]},{"osd":11,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.94499999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":461388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41399999999999998,"15min":0.38500000000000001},"min":{"1min":0.30499999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.51600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.45700000000000002,"15min":0.40999999999999998},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.36699999999999999}]},{"osd":1,"last update":"Thu Jul 6 20:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65800000000000003,"15min":0.58499999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81599999999999995,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.5},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59799999999999998,"15min":0.57199999999999995},"min":{"1min":0.45200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95599999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:18:50 2023","interfaces":[{"interface":"back","average":{"1min":2.181,"5min":0.94199999999999995,"15min":0.69599999999999995},"min":{"1min":0.51900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":36.344000000000001,"5min":36.344000000000001,"15min":36.344000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67600000000000005,"15min":0.625},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93700000000000006,"5min":3.6720000000000002,"15min":3.6720000000000002},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jul 6 20:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.73199999999999998,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":6.9420000000000002,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.58699999999999997},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.442}]},{"osd":5,"last update":"Thu Jul 6 20:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.70599999999999996,"15min":0.66300000000000003},"min":{"1min":0.48899999999999999,"5min":0.441,"15min":0.441},"max":{"1min":4.5389999999999997,"5min":4.5389999999999997,"15min":4.5389999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66800000000000004,"15min":0.63},"min":{"1min":0.51600000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64200000000000002,"15min":0.61799999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86599999999999999,"5min":1.052,"15min":1.052},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.56799999999999995},"min":{"1min":0.40799999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95199999999999996,"5min":0.998,"15min":0.998},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.79900000000000004,"15min":0.76500000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.1859999999999999,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":0.73199999999999998}]},{"osd":8,"last update":"Thu Jul 6 20:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.58299999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:19:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.77300000000000002,"15min":0.80200000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92400000000000004,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.71699999999999997,"15min":0.73499999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.71599999999999997}]},{"osd":10,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.046,"5min":1.173,"15min":1.173},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.077,"5min":0.89300000000000002,"15min":0.86199999999999999},"min":{"1min":0.68999999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":5.8040000000000003,"5min":5.8040000000000003,"15min":5.8040000000000003},"last":0.70599999999999996}]},{"osd":11,"last update":"Thu Jul 6 20:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":6.79,"5min":6.79,"15min":6.79},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.752}]}]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":462297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.55200000000000005,"15min":0.501},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":6.1539999999999999,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.51900000000000002,"15min":0.49299999999999999},"min":{"1min":0.45700000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.35799999999999998},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58399999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.379},{"interface":"front","average":{"1min":0.44,"5min":0.39600000000000002,"15min":0.35799999999999998},"min":{"1min":0.311,"5min":0.219,"15min":0.219},"max":{"1min":0.55900000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.40500000000000003}]},{"osd":2,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.47199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.753},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.44400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86699999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.53600000000000003},"min":{"1min":0.52200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.099,"5min":5.593,"15min":5.593},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.123,"5min":0.71699999999999997,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.378,"15min":0.378},"max":{"1min":9.7959999999999994,"5min":9.7959999999999994,"15min":9.7959999999999994},"last":0.44}]},{"osd":5,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.61599999999999999,"15min":0.56699999999999995},"min":{"1min":0.57599999999999996,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.63100000000000001,"15min":0.57999999999999996},"min":{"1min":0.47999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90900000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.40899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91600000000000004,"5min":1.109,"15min":1.109},"last":0.621},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.004,"5min":1.163,"15min":1.163},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.35499999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.97699999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.372,"5min":0.314,"15min":0.314},"max":{"1min":0.91200000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Thu Jul 6 20:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.754,"15min":0.74199999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0780000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.52300000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96599999999999997,"5min":1.194,"15min":1.194},"last":0.76500000000000001}]},{"osd":9,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71699999999999997,"15min":0.72499999999999998},"min":{"1min":0.54900000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.88100000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.92000000000000004,"15min":0.86899999999999999},"min":{"1min":0.63900000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0329999999999999,"5min":8.3369999999999997,"15min":8.3369999999999997},"last":0.81699999999999995}]},{"osd":10,"last update":"Thu Jul 6 20:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.82999999999999996,"15min":0.83299999999999996},"min":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.054,"5min":1.258,"15min":1.258},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.55300000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89700000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.71499999999999997}]},{"osd":11,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79300000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1576960,"data_stored":1052977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.53000000000000003},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81699999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.79200000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.42899999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.41299999999999998,"15min":0.40100000000000002},"min":{"1min":0.315,"5min":0.253,"15min":0.253},"max":{"1min":0.66100000000000003,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.31900000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.76400000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.58099999999999996},"min":{"1min":0.497,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.747},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.44800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.70299999999999996}]},{"osd":6,"last update":"Thu Jul 6 20:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.435,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66400000000000003,"15min":0.64200000000000002},"min":{"1min":0.51700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.85199999999999998,"5min":1.01,"15min":1.01},"last":0.76800000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.81799999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.46400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.45200000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.78200000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.61899999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.59799999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.84199999999999997}]},{"osd":10,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.53300000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.92600000000000005,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.57699999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.96599999999999997,"5min":1.109,"15min":1.109},"last":0.88700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:19:10.542 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:19:10.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.769+0000 7f0eb3bff700 1 -- 172.21.15.69:0/1115589777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0eac100680 msgr2=0x7f0eac102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:10.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.769+0000 7f0eb3bff700 1 --2- 172.21.15.69:0/1115589777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0eac100680 0x7f0eac102a60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009990 tx=0x7f0ea8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.769+0000 7f0eb3bff700 1 -- 172.21.15.69:0/1115589777 shutdown_connections 2023-07-06T20:19:10.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.769+0000 7f0eb3bff700 1 --2- 172.21.15.69:0/1115589777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0eac1058c0 0x7f0eac107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.769+0000 7f0eb3bff700 1 --2- 172.21.15.69:0/1115589777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0eac100680 0x7f0eac102a60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.769+0000 7f0eb3bff700 1 --2- 172.21.15.69:0/1115589777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac102fa0 0x7f0eac105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.769+0000 7f0eb3bff700 1 -- 172.21.15.69:0/1115589777 >> 172.21.15.69:0/1115589777 conn(0x7f0eac0fa100 msgr2=0x7f0eac0fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:10.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.769+0000 7f0eb3bff700 1 -- 172.21.15.69:0/1115589777 shutdown_connections 2023-07-06T20:19:10.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.769+0000 7f0eb3bff700 1 -- 172.21.15.69:0/1115589777 wait complete. 2023-07-06T20:19:10.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.770+0000 7f0eb3bff700 1 Processor -- start 2023-07-06T20:19:10.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.770+0000 7f0eb3bff700 1 -- start start 2023-07-06T20:19:10.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.770+0000 7f0eb3bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0eac100680 0x7f0eac0ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:10.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.770+0000 7f0eb3bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0eac102fa0 0x7f0eac0fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:10.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.770+0000 7f0eb3bff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac1058c0 0x7f0eac0feb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:10.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.770+0000 7f0eb3bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac1997b0 con 0x7f0eac102fa0 2023-07-06T20:19:10.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.770+0000 7f0eb3bff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0ff0b0 con 0x7f0eac1058c0 2023-07-06T20:19:10.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.770+0000 7f0eb3bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0ff270 con 0x7f0eac100680 2023-07-06T20:19:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.770+0000 7f0eb199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0eac100680 0x7f0eac0ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.771+0000 7f0eb199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0eac100680 0x7f0eac0ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:59694/0 (socket says 172.21.15.69:59694) 2023-07-06T20:19:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.771+0000 7f0eb199b700 1 -- 172.21.15.69:0/138944517 learned_addr learned my addr 172.21.15.69:0/138944517 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.771+0000 7f0eb119a700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0eac102fa0 0x7f0eac0fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.771+0000 7f0eb219c700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac1058c0 0x7f0eac0feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.771+0000 7f0eb199b700 1 -- 172.21.15.69:0/138944517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac1058c0 msgr2=0x7f0eac0feb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.771+0000 7f0eb199b700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac1058c0 0x7f0eac0feb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.771+0000 7f0eb199b700 1 -- 172.21.15.69:0/138944517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0eac102fa0 msgr2=0x7f0eac0fe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:10.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.771+0000 7f0eb199b700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0eac102fa0 0x7f0eac0fe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.771+0000 7f0eb199b700 1 -- 172.21.15.69:0/138944517 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea80095f0 con 0x7f0eac100680 2023-07-06T20:19:10.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.771+0000 7f0eb199b700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0eac100680 0x7f0eac0ffff0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800b680 tx=0x7f0ea8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:10.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.771+0000 7f0ea2ffd700 1 -- 172.21.15.69:0/138944517 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea801d070 con 0x7f0eac100680 2023-07-06T20:19:10.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.772+0000 7f0eb3bff700 1 -- 172.21.15.69:0/138944517 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eac1a4160 con 0x7f0eac100680 2023-07-06T20:19:10.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.772+0000 7f0ea2ffd700 1 -- 172.21.15.69:0/138944517 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ea8022470 con 0x7f0eac100680 2023-07-06T20:19:10.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.772+0000 7f0ea2ffd700 1 -- 172.21.15.69:0/138944517 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8017650 con 0x7f0eac100680 2023-07-06T20:19:10.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.772+0000 7f0eb3bff700 1 -- 172.21.15.69:0/138944517 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eac1a44d0 con 0x7f0eac100680 2023-07-06T20:19:10.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.773+0000 7f0eb3bff700 1 -- 172.21.15.69:0/138944517 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eac10b400 con 0x7f0eac100680 2023-07-06T20:19:10.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.775+0000 7f0ea2ffd700 1 -- 172.21.15.69:0/138944517 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0ea80225e0 con 0x7f0eac100680 2023-07-06T20:19:10.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.776+0000 7f0ea2ffd700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0e980a11a0 0x7f0e980a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:10.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.776+0000 7f0ea2ffd700 1 -- 172.21.15.69:0/138944517 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f0ea80cdf30 con 0x7f0eac100680 2023-07-06T20:19:10.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.776+0000 7f0eb119a700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0e980a11a0 0x7f0e980a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:10.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.777+0000 7f0eb119a700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0e980a11a0 0x7f0e980a3650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009c60 tx=0x7f0e9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:10.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.778+0000 7f0ea2ffd700 1 -- 172.21.15.69:0/138944517 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0ea80fa770 con 0x7f0eac100680 2023-07-06T20:19:10.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.973+0000 7f0eb3bff700 1 -- 172.21.15.69:0/138944517 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0eac1021d0 con 0x7f0e980a11a0 2023-07-06T20:19:10.985 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.984+0000 7f0ea2ffd700 1 -- 172.21.15.69:0/138944517 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344244 (secure 0 0 0) 0x7f0eac1021d0 con 0x7f0e980a11a0 2023-07-06T20:19:10.989 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:10.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.988+0000 7f0eb3bff700 1 -- 172.21.15.69:0/138944517 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0e980a11a0 msgr2=0x7f0e980a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:10.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.988+0000 7f0eb3bff700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0e980a11a0 0x7f0e980a3650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009c60 tx=0x7f0e9c009260 comp rx=0 tx=0).stop 2023-07-06T20:19:10.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.988+0000 7f0eb3bff700 1 -- 172.21.15.69:0/138944517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0eac100680 msgr2=0x7f0eac0ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:10.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.988+0000 7f0eb3bff700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0eac100680 0x7f0eac0ffff0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800b680 tx=0x7f0ea8005740 comp rx=0 tx=0).stop 2023-07-06T20:19:10.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.988+0000 7f0eb3bff700 1 -- 172.21.15.69:0/138944517 shutdown_connections 2023-07-06T20:19:10.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.988+0000 7f0eb3bff700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0e980a11a0 0x7f0e980a3650 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.988+0000 7f0eb3bff700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac1058c0 0x7f0eac0feb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.988+0000 7f0eb3bff700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0eac100680 0x7f0eac0ffff0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.988+0000 7f0eb3bff700 1 --2- 172.21.15.69:0/138944517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0eac102fa0 0x7f0eac0fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:10.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.988+0000 7f0eb3bff700 1 -- 172.21.15.69:0/138944517 >> 172.21.15.69:0/138944517 conn(0x7f0eac0fa100 msgr2=0x7f0eac101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:10.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.988+0000 7f0eb3bff700 1 -- 172.21.15.69:0/138944517 shutdown_connections 2023-07-06T20:19:10.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:10.988+0000 7f0eb3bff700 1 -- 172.21.15.69:0/138944517 wait complete. 2023-07-06T20:19:10.993 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-07-06T20:19:11.006 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:10 smithi069 ceph-mon[126399]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 716 B/s wr, 6 op/s; 107 B/s, 0 objects/s recovering 2023-07-06T20:19:11.006 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:10 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/4014404632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:19:11.008 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2023-07-06T20:19:09.545348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654278,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":715,"num_read_kb":471,"num_write":345,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181919,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336776,"kb_used_data":13352,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605048,"statfs":{"total":1151940427776,"available":1151595569152,"internally_reserved":0,"allocated":13672448,"data_stored":7498209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":702,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":31,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1071,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002553"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574765+0000","last_change":"2023-07-06T20:17:28.575520+0000","last_active":"2023-07-06T20:19:00.574765+0000","last_peered":"2023-07-06T20:19:00.574765+0000","last_clean":"2023-07-06T20:19:00.574765+0000","last_became_active":"2023-07-06T20:17:28.575209+0000","last_became_peered":"2023-07-06T20:17:28.575209+0000","last_unstale":"2023-07-06T20:19:00.574765+0000","last_undegraded":"2023-07-06T20:19:00.574765+0000","last_fullsized":"2023-07-06T20:19:00.574765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:31:43.014901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575773+0000","last_change":"2023-07-06T20:17:28.572547+0000","last_active":"2023-07-06T20:19:00.575773+0000","last_peered":"2023-07-06T20:19:00.575773+0000","last_clean":"2023-07-06T20:19:00.575773+0000","last_became_active":"2023-07-06T20:17:28.572150+0000","last_became_peered":"2023-07-06T20:17:28.572150+0000","last_unstale":"2023-07-06T20:19:00.575773+0000","last_undegraded":"2023-07-06T20:19:00.575773+0000","last_fullsized":"2023-07-06T20:19:00.575773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:16.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575509+0000","last_change":"2023-07-06T20:17:28.574926+0000","last_active":"2023-07-06T20:19:00.575509+0000","last_peered":"2023-07-06T20:19:00.575509+0000","last_clean":"2023-07-06T20:19:00.575509+0000","last_became_active":"2023-07-06T20:17:28.574602+0000","last_became_peered":"2023-07-06T20:17:28.574602+0000","last_unstale":"2023-07-06T20:19:00.575509+0000","last_undegraded":"2023-07-06T20:19:00.575509+0000","last_fullsized":"2023-07-06T20:19:00.575509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:37:21.626981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.651733+0000","last_change":"2023-07-06T20:17:28.571533+0000","last_active":"2023-07-06T20:18:59.651733+0000","last_peered":"2023-07-06T20:18:59.651733+0000","last_clean":"2023-07-06T20:18:59.651733+0000","last_became_active":"2023-07-06T20:17:28.571099+0000","last_became_peered":"2023-07-06T20:17:28.571099+0000","last_unstale":"2023-07-06T20:18:59.651733+0000","last_undegraded":"2023-07-06T20:18:59.651733+0000","last_fullsized":"2023-07-06T20:18:59.651733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:09:44.688143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646150+0000","last_change":"2023-07-06T20:17:28.572100+0000","last_active":"2023-07-06T20:18:59.646150+0000","last_peered":"2023-07-06T20:18:59.646150+0000","last_clean":"2023-07-06T20:18:59.646150+0000","last_became_active":"2023-07-06T20:17:28.571772+0000","last_became_peered":"2023-07-06T20:17:28.571772+0000","last_unstale":"2023-07-06T20:18:59.646150+0000","last_undegraded":"2023-07-06T20:18:59.646150+0000","last_fullsized":"2023-07-06T20:18:59.646150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:20:48.756324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654341+0000","last_change":"2023-07-06T20:17:28.577816+0000","last_active":"2023-07-06T20:18:59.654341+0000","last_peered":"2023-07-06T20:18:59.654341+0000","last_clean":"2023-07-06T20:18:59.654341+0000","last_became_active":"2023-07-06T20:17:28.577382+0000","last_became_peered":"2023-07-06T20:17:28.577382+0000","last_unstale":"2023-07-06T20:18:59.654341+0000","last_undegraded":"2023-07-06T20:18:59.654341+0000","last_fullsized":"2023-07-06T20:18:59.654341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:45:41.682441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.653973+0000","last_change":"2023-07-06T20:17:28.570415+0000","last_active":"2023-07-06T20:18:59.653973+0000","last_peered":"2023-07-06T20:18:59.653973+0000","last_clean":"2023-07-06T20:18:59.653973+0000","last_became_active":"2023-07-06T20:17:28.570091+0000","last_became_peered":"2023-07-06T20:17:28.570091+0000","last_unstale":"2023-07-06T20:18:59.653973+0000","last_undegraded":"2023-07-06T20:18:59.653973+0000","last_fullsized":"2023-07-06T20:18:59.653973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:39:50.216113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575038+0000","last_change":"2023-07-06T20:17:28.568370+0000","last_active":"2023-07-06T20:19:00.575038+0000","last_peered":"2023-07-06T20:19:00.575038+0000","last_clean":"2023-07-06T20:19:00.575038+0000","last_became_active":"2023-07-06T20:17:28.568032+0000","last_became_peered":"2023-07-06T20:17:28.568032+0000","last_unstale":"2023-07-06T20:19:00.575038+0000","last_undegraded":"2023-07-06T20:19:00.575038+0000","last_fullsized":"2023-07-06T20:19:00.575038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:29:40.448529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648913+0000","last_change":"2023-07-06T20:17:28.563386+0000","last_active":"2023-07-06T20:18:59.648913+0000","last_peered":"2023-07-06T20:18:59.648913+0000","last_clean":"2023-07-06T20:18:59.648913+0000","last_became_active":"2023-07-06T20:17:28.562864+0000","last_became_peered":"2023-07-06T20:17:28.562864+0000","last_unstale":"2023-07-06T20:18:59.648913+0000","last_undegraded":"2023-07-06T20:18:59.648913+0000","last_fullsized":"2023-07-06T20:18:59.648913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:17:20.864607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.655052+0000","last_change":"2023-07-06T20:17:28.578016+0000","last_active":"2023-07-06T20:18:59.655052+0000","last_peered":"2023-07-06T20:18:59.655052+0000","last_clean":"2023-07-06T20:18:59.655052+0000","last_became_active":"2023-07-06T20:17:28.577305+0000","last_became_peered":"2023-07-06T20:17:28.577305+0000","last_unstale":"2023-07-06T20:18:59.655052+0000","last_undegraded":"2023-07-06T20:18:59.655052+0000","last_fullsized":"2023-07-06T20:18:59.655052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:04.298499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654451+0000","last_change":"2023-07-06T20:17:28.569683+0000","last_active":"2023-07-06T20:18:59.654451+0000","last_peered":"2023-07-06T20:18:59.654451+0000","last_clean":"2023-07-06T20:18:59.654451+0000","last_became_active":"2023-07-06T20:17:28.569070+0000","last_became_peered":"2023-07-06T20:17:28.569070+0000","last_unstale":"2023-07-06T20:18:59.654451+0000","last_undegraded":"2023-07-06T20:18:59.654451+0000","last_fullsized":"2023-07-06T20:18:59.654451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:32:02.630231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.651730+0000","last_change":"2023-07-06T20:17:28.579463+0000","last_active":"2023-07-06T20:18:59.651730+0000","last_peered":"2023-07-06T20:18:59.651730+0000","last_clean":"2023-07-06T20:18:59.651730+0000","last_became_active":"2023-07-06T20:17:28.579141+0000","last_became_peered":"2023-07-06T20:17:28.579141+0000","last_unstale":"2023-07-06T20:18:59.651730+0000","last_undegraded":"2023-07-06T20:18:59.651730+0000","last_fullsized":"2023-07-06T20:18:59.651730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:35:41.411836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646027+0000","last_change":"2023-07-06T20:17:58.864016+0000","last_active":"2023-07-06T20:18:59.646027+0000","last_peered":"2023-07-06T20:18:59.646027+0000","last_clean":"2023-07-06T20:18:59.646027+0000","last_became_active":"2023-07-06T20:17:58.863583+0000","last_became_peered":"2023-07-06T20:17:58.863583+0000","last_unstale":"2023-07-06T20:18:59.646027+0000","last_undegraded":"2023-07-06T20:18:59.646027+0000","last_fullsized":"2023-07-06T20:18:59.646027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:20.046705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575003+0000","last_change":"2023-07-06T20:17:28.572197+0000","last_active":"2023-07-06T20:19:00.575003+0000","last_peered":"2023-07-06T20:19:00.575003+0000","last_clean":"2023-07-06T20:19:00.575003+0000","last_became_active":"2023-07-06T20:17:28.571816+0000","last_became_peered":"2023-07-06T20:17:28.571816+0000","last_unstale":"2023-07-06T20:19:00.575003+0000","last_undegraded":"2023-07-06T20:19:00.575003+0000","last_fullsized":"2023-07-06T20:19:00.575003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:51.605941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574805+0000","last_change":"2023-07-06T20:17:58.746966+0000","last_active":"2023-07-06T20:19:00.574805+0000","last_peered":"2023-07-06T20:19:00.574805+0000","last_clean":"2023-07-06T20:19:00.574805+0000","last_became_active":"2023-07-06T20:17:58.746598+0000","last_became_peered":"2023-07-06T20:17:58.746598+0000","last_unstale":"2023-07-06T20:19:00.574805+0000","last_undegraded":"2023-07-06T20:19:00.574805+0000","last_fullsized":"2023-07-06T20:19:00.574805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:23:21.366915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648494+0000","last_change":"2023-07-06T20:17:28.573552+0000","last_active":"2023-07-06T20:18:59.648494+0000","last_peered":"2023-07-06T20:18:59.648494+0000","last_clean":"2023-07-06T20:18:59.648494+0000","last_became_active":"2023-07-06T20:17:28.573253+0000","last_became_peered":"2023-07-06T20:17:28.573253+0000","last_unstale":"2023-07-06T20:18:59.648494+0000","last_undegraded":"2023-07-06T20:18:59.648494+0000","last_fullsized":"2023-07-06T20:18:59.648494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:20:52.534070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574810+0000","last_change":"2023-07-06T20:17:28.572425+0000","last_active":"2023-07-06T20:19:00.574810+0000","last_peered":"2023-07-06T20:19:00.574810+0000","last_clean":"2023-07-06T20:19:00.574810+0000","last_became_active":"2023-07-06T20:17:28.572104+0000","last_became_peered":"2023-07-06T20:17:28.572104+0000","last_unstale":"2023-07-06T20:19:00.574810+0000","last_undegraded":"2023-07-06T20:19:00.574810+0000","last_fullsized":"2023-07-06T20:19:00.574810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:35:59.178149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574524+0000","last_change":"2023-07-06T20:17:28.573814+0000","last_active":"2023-07-06T20:19:00.574524+0000","last_peered":"2023-07-06T20:19:00.574524+0000","last_clean":"2023-07-06T20:19:00.574524+0000","last_became_active":"2023-07-06T20:17:28.573470+0000","last_became_peered":"2023-07-06T20:17:28.573470+0000","last_unstale":"2023-07-06T20:19:00.574524+0000","last_undegraded":"2023-07-06T20:19:00.574524+0000","last_fullsized":"2023-07-06T20:19:00.574524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:03:51.146800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073530+0000","last_change":"2023-07-06T20:17:58.865802+0000","last_active":"2023-07-06T20:19:00.073530+0000","last_peered":"2023-07-06T20:19:00.073530+0000","last_clean":"2023-07-06T20:19:00.073530+0000","last_became_active":"2023-07-06T20:17:58.865290+0000","last_became_peered":"2023-07-06T20:17:58.865290+0000","last_unstale":"2023-07-06T20:19:00.073530+0000","last_undegraded":"2023-07-06T20:19:00.073530+0000","last_fullsized":"2023-07-06T20:19:00.073530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:07:36.214950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574917+0000","last_change":"2023-07-06T20:17:27.603771+0000","last_active":"2023-07-06T20:19:00.574917+0000","last_peered":"2023-07-06T20:19:00.574917+0000","last_clean":"2023-07-06T20:19:00.574917+0000","last_became_active":"2023-07-06T20:17:27.603404+0000","last_became_peered":"2023-07-06T20:17:27.603404+0000","last_unstale":"2023-07-06T20:19:00.574917+0000","last_undegraded":"2023-07-06T20:19:00.574917+0000","last_fullsized":"2023-07-06T20:19:00.574917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:51:42.385000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575838+0000","last_change":"2023-07-06T20:17:28.568371+0000","last_active":"2023-07-06T20:19:00.575838+0000","last_peered":"2023-07-06T20:19:00.575838+0000","last_clean":"2023-07-06T20:19:00.575838+0000","last_became_active":"2023-07-06T20:17:28.568034+0000","last_became_peered":"2023-07-06T20:17:28.568034+0000","last_unstale":"2023-07-06T20:19:00.575838+0000","last_undegraded":"2023-07-06T20:19:00.575838+0000","last_fullsized":"2023-07-06T20:19:00.575838+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:20:30.073418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575967+0000","last_change":"2023-07-06T20:17:27.609833+0000","last_active":"2023-07-06T20:19:00.575967+0000","last_peered":"2023-07-06T20:19:00.575967+0000","last_clean":"2023-07-06T20:19:00.575967+0000","last_became_active":"2023-07-06T20:17:27.609520+0000","last_became_peered":"2023-07-06T20:17:27.609520+0000","last_unstale":"2023-07-06T20:19:00.575967+0000","last_undegraded":"2023-07-06T20:19:00.575967+0000","last_fullsized":"2023-07-06T20:19:00.575967+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:22.478396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575733+0000","last_change":"2023-07-06T20:17:28.574520+0000","last_active":"2023-07-06T20:19:00.575733+0000","last_peered":"2023-07-06T20:19:00.575733+0000","last_clean":"2023-07-06T20:19:00.575733+0000","last_became_active":"2023-07-06T20:17:28.574158+0000","last_became_peered":"2023-07-06T20:17:28.574158+0000","last_unstale":"2023-07-06T20:19:00.575733+0000","last_undegraded":"2023-07-06T20:19:00.575733+0000","last_fullsized":"2023-07-06T20:19:00.575733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:06.841394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646514+0000","last_change":"2023-07-06T20:17:27.601612+0000","last_active":"2023-07-06T20:18:59.646514+0000","last_peered":"2023-07-06T20:18:59.646514+0000","last_clean":"2023-07-06T20:18:59.646514+0000","last_became_active":"2023-07-06T20:17:27.601313+0000","last_became_peered":"2023-07-06T20:17:27.601313+0000","last_unstale":"2023-07-06T20:18:59.646514+0000","last_undegraded":"2023-07-06T20:18:59.646514+0000","last_fullsized":"2023-07-06T20:18:59.646514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:42:58.053499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648800+0000","last_change":"2023-07-06T20:17:28.566389+0000","last_active":"2023-07-06T20:18:59.648800+0000","last_peered":"2023-07-06T20:18:59.648800+0000","last_clean":"2023-07-06T20:18:59.648800+0000","last_became_active":"2023-07-06T20:17:28.566094+0000","last_became_peered":"2023-07-06T20:17:28.566094+0000","last_unstale":"2023-07-06T20:18:59.648800+0000","last_undegraded":"2023-07-06T20:18:59.648800+0000","last_fullsized":"2023-07-06T20:18:59.648800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:43:18.213143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646434+0000","last_change":"2023-07-06T20:17:27.599893+0000","last_active":"2023-07-06T20:18:59.646434+0000","last_peered":"2023-07-06T20:18:59.646434+0000","last_clean":"2023-07-06T20:18:59.646434+0000","last_became_active":"2023-07-06T20:17:27.599544+0000","last_became_peered":"2023-07-06T20:17:27.599544+0000","last_unstale":"2023-07-06T20:18:59.646434+0000","last_undegraded":"2023-07-06T20:18:59.646434+0000","last_fullsized":"2023-07-06T20:18:59.646434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:08:58.904898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703267+0000","last_change":"2023-07-06T20:17:28.575794+0000","last_active":"2023-07-06T20:18:59.703267+0000","last_peered":"2023-07-06T20:18:59.703267+0000","last_clean":"2023-07-06T20:18:59.703267+0000","last_became_active":"2023-07-06T20:17:28.575456+0000","last_became_peered":"2023-07-06T20:17:28.575456+0000","last_unstale":"2023-07-06T20:18:59.703267+0000","last_undegraded":"2023-07-06T20:18:59.703267+0000","last_fullsized":"2023-07-06T20:18:59.703267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:47:11.340543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":128,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:05.714486+0000","last_change":"2023-07-06T20:17:27.596351+0000","last_active":"2023-07-06T20:19:05.714486+0000","last_peered":"2023-07-06T20:19:05.714486+0000","last_clean":"2023-07-06T20:19:05.714486+0000","last_became_active":"2023-07-06T20:17:27.595982+0000","last_became_peered":"2023-07-06T20:17:27.595982+0000","last_unstale":"2023-07-06T20:19:05.714486+0000","last_undegraded":"2023-07-06T20:19:05.714486+0000","last_fullsized":"2023-07-06T20:19:05.714486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:33:14.017909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646053+0000","last_change":"2023-07-06T20:17:28.570018+0000","last_active":"2023-07-06T20:18:59.646053+0000","last_peered":"2023-07-06T20:18:59.646053+0000","last_clean":"2023-07-06T20:18:59.646053+0000","last_became_active":"2023-07-06T20:17:28.569674+0000","last_became_peered":"2023-07-06T20:17:28.569674+0000","last_unstale":"2023-07-06T20:18:59.646053+0000","last_undegraded":"2023-07-06T20:18:59.646053+0000","last_fullsized":"2023-07-06T20:18:59.646053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:24:03.335858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654185+0000","last_change":"2023-07-06T20:17:58.866940+0000","last_active":"2023-07-06T20:18:59.654185+0000","last_peered":"2023-07-06T20:18:59.654185+0000","last_clean":"2023-07-06T20:18:59.654185+0000","last_became_active":"2023-07-06T20:17:58.863435+0000","last_became_peered":"2023-07-06T20:17:58.863435+0000","last_unstale":"2023-07-06T20:18:59.654185+0000","last_undegraded":"2023-07-06T20:18:59.654185+0000","last_fullsized":"2023-07-06T20:18:59.654185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:34:45.362921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703189+0000","last_change":"2023-07-06T20:17:28.575249+0000","last_active":"2023-07-06T20:18:59.703189+0000","last_peered":"2023-07-06T20:18:59.703189+0000","last_clean":"2023-07-06T20:18:59.703189+0000","last_became_active":"2023-07-06T20:17:28.574954+0000","last_became_peered":"2023-07-06T20:17:28.574954+0000","last_unstale":"2023-07-06T20:18:59.703189+0000","last_undegraded":"2023-07-06T20:18:59.703189+0000","last_fullsized":"2023-07-06T20:18:59.703189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:10:03.983955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645825+0000","last_change":"2023-07-06T20:17:27.600552+0000","last_active":"2023-07-06T20:18:59.645825+0000","last_peered":"2023-07-06T20:18:59.645825+0000","last_clean":"2023-07-06T20:18:59.645825+0000","last_became_active":"2023-07-06T20:17:27.600016+0000","last_became_peered":"2023-07-06T20:17:27.600016+0000","last_unstale":"2023-07-06T20:18:59.645825+0000","last_undegraded":"2023-07-06T20:18:59.645825+0000","last_fullsized":"2023-07-06T20:18:59.645825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:08:51.272104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575191+0000","last_change":"2023-07-06T20:17:28.566364+0000","last_active":"2023-07-06T20:19:00.575191+0000","last_peered":"2023-07-06T20:19:00.575191+0000","last_clean":"2023-07-06T20:19:00.575191+0000","last_became_active":"2023-07-06T20:17:28.565474+0000","last_became_peered":"2023-07-06T20:17:28.565474+0000","last_unstale":"2023-07-06T20:19:00.575191+0000","last_undegraded":"2023-07-06T20:19:00.575191+0000","last_fullsized":"2023-07-06T20:19:00.575191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:28:42.265878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.702829+0000","last_change":"2023-07-06T20:17:27.598606+0000","last_active":"2023-07-06T20:18:59.702829+0000","last_peered":"2023-07-06T20:18:59.702829+0000","last_clean":"2023-07-06T20:18:59.702829+0000","last_became_active":"2023-07-06T20:17:27.598292+0000","last_became_peered":"2023-07-06T20:17:27.598292+0000","last_unstale":"2023-07-06T20:18:59.702829+0000","last_undegraded":"2023-07-06T20:18:59.702829+0000","last_fullsized":"2023-07-06T20:18:59.702829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:11:39.786934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574951+0000","last_change":"2023-07-06T20:17:28.571246+0000","last_active":"2023-07-06T20:19:00.574951+0000","last_peered":"2023-07-06T20:19:00.574951+0000","last_clean":"2023-07-06T20:19:00.574951+0000","last_became_active":"2023-07-06T20:17:28.570846+0000","last_became_peered":"2023-07-06T20:17:28.570846+0000","last_unstale":"2023-07-06T20:19:00.574951+0000","last_undegraded":"2023-07-06T20:19:00.574951+0000","last_fullsized":"2023-07-06T20:19:00.574951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:31:31.244148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:03.710217+0000","last_change":"2023-07-06T20:17:27.595879+0000","last_active":"2023-07-06T20:19:03.710217+0000","last_peered":"2023-07-06T20:19:03.710217+0000","last_clean":"2023-07-06T20:19:03.710217+0000","last_became_active":"2023-07-06T20:17:27.595553+0000","last_became_peered":"2023-07-06T20:17:27.595553+0000","last_unstale":"2023-07-06T20:19:03.710217+0000","last_undegraded":"2023-07-06T20:19:03.710217+0000","last_fullsized":"2023-07-06T20:19:03.710217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:01:12.807960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648843+0000","last_change":"2023-07-06T20:17:28.566816+0000","last_active":"2023-07-06T20:18:59.648843+0000","last_peered":"2023-07-06T20:18:59.648843+0000","last_clean":"2023-07-06T20:18:59.648843+0000","last_became_active":"2023-07-06T20:17:28.566565+0000","last_became_peered":"2023-07-06T20:17:28.566565+0000","last_unstale":"2023-07-06T20:18:59.648843+0000","last_undegraded":"2023-07-06T20:18:59.648843+0000","last_fullsized":"2023-07-06T20:18:59.648843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:37:37.504246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.653939+0000","last_change":"2023-07-06T20:17:58.868853+0000","last_active":"2023-07-06T20:18:59.653939+0000","last_peered":"2023-07-06T20:18:59.653939+0000","last_clean":"2023-07-06T20:18:59.653939+0000","last_became_active":"2023-07-06T20:17:58.863304+0000","last_became_peered":"2023-07-06T20:17:58.863304+0000","last_unstale":"2023-07-06T20:18:59.653939+0000","last_undegraded":"2023-07-06T20:18:59.653939+0000","last_fullsized":"2023-07-06T20:18:59.653939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:33:48.601790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575721+0000","last_change":"2023-07-06T20:17:28.572400+0000","last_active":"2023-07-06T20:19:00.575721+0000","last_peered":"2023-07-06T20:19:00.575721+0000","last_clean":"2023-07-06T20:19:00.575721+0000","last_became_active":"2023-07-06T20:17:28.572047+0000","last_became_peered":"2023-07-06T20:17:28.572047+0000","last_unstale":"2023-07-06T20:19:00.575721+0000","last_undegraded":"2023-07-06T20:19:00.575721+0000","last_fullsized":"2023-07-06T20:19:00.575721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:17:30.668701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703472+0000","last_change":"2023-07-06T20:17:27.603224+0000","last_active":"2023-07-06T20:18:59.703472+0000","last_peered":"2023-07-06T20:18:59.703472+0000","last_clean":"2023-07-06T20:18:59.703472+0000","last_became_active":"2023-07-06T20:17:27.602864+0000","last_became_peered":"2023-07-06T20:17:27.602864+0000","last_unstale":"2023-07-06T20:18:59.703472+0000","last_undegraded":"2023-07-06T20:18:59.703472+0000","last_fullsized":"2023-07-06T20:18:59.703472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:23:10.000206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648876+0000","last_change":"2023-07-06T20:17:28.572683+0000","last_active":"2023-07-06T20:18:59.648876+0000","last_peered":"2023-07-06T20:18:59.648876+0000","last_clean":"2023-07-06T20:18:59.648876+0000","last_became_active":"2023-07-06T20:17:28.572352+0000","last_became_peered":"2023-07-06T20:17:28.572352+0000","last_unstale":"2023-07-06T20:18:59.648876+0000","last_undegraded":"2023-07-06T20:18:59.648876+0000","last_fullsized":"2023-07-06T20:18:59.648876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:15:00.435753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'30","reported_seq":247,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:04.711908+0000","last_change":"2023-07-06T20:17:27.597285+0000","last_active":"2023-07-06T20:19:04.711908+0000","last_peered":"2023-07-06T20:19:04.711908+0000","last_clean":"2023-07-06T20:19:04.711908+0000","last_became_active":"2023-07-06T20:17:27.596850+0000","last_became_peered":"2023-07-06T20:17:27.596850+0000","last_unstale":"2023-07-06T20:19:04.711908+0000","last_undegraded":"2023-07-06T20:19:04.711908+0000","last_fullsized":"2023-07-06T20:19:04.711908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:22:39.869484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58776,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":195,"num_read_kb":154,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646322+0000","last_change":"2023-07-06T20:17:28.569855+0000","last_active":"2023-07-06T20:18:59.646322+0000","last_peered":"2023-07-06T20:18:59.646322+0000","last_clean":"2023-07-06T20:18:59.646322+0000","last_became_active":"2023-07-06T20:17:28.569495+0000","last_became_peered":"2023-07-06T20:17:28.569495+0000","last_unstale":"2023-07-06T20:18:59.646322+0000","last_undegraded":"2023-07-06T20:18:59.646322+0000","last_fullsized":"2023-07-06T20:18:59.646322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:59:00.755254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073397+0000","last_change":"2023-07-06T20:17:27.605898+0000","last_active":"2023-07-06T20:19:00.073397+0000","last_peered":"2023-07-06T20:19:00.073397+0000","last_clean":"2023-07-06T20:19:00.073397+0000","last_became_active":"2023-07-06T20:17:27.605605+0000","last_became_peered":"2023-07-06T20:17:27.605605+0000","last_unstale":"2023-07-06T20:19:00.073397+0000","last_undegraded":"2023-07-06T20:19:00.073397+0000","last_fullsized":"2023-07-06T20:19:00.073397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:31:31.780838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645727+0000","last_change":"2023-07-06T20:17:28.579236+0000","last_active":"2023-07-06T20:18:59.645727+0000","last_peered":"2023-07-06T20:18:59.645727+0000","last_clean":"2023-07-06T20:18:59.645727+0000","last_became_active":"2023-07-06T20:17:28.578854+0000","last_became_peered":"2023-07-06T20:17:28.578854+0000","last_unstale":"2023-07-06T20:18:59.645727+0000","last_undegraded":"2023-07-06T20:18:59.645727+0000","last_fullsized":"2023-07-06T20:18:59.645727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:20:05.459582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703207+0000","last_change":"2023-07-06T20:17:27.595724+0000","last_active":"2023-07-06T20:18:59.703207+0000","last_peered":"2023-07-06T20:18:59.703207+0000","last_clean":"2023-07-06T20:18:59.703207+0000","last_became_active":"2023-07-06T20:17:27.595360+0000","last_became_peered":"2023-07-06T20:17:27.595360+0000","last_unstale":"2023-07-06T20:18:59.703207+0000","last_undegraded":"2023-07-06T20:18:59.703207+0000","last_fullsized":"2023-07-06T20:18:59.703207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:46:07.776434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654912+0000","last_change":"2023-07-06T20:17:28.573447+0000","last_active":"2023-07-06T20:18:59.654912+0000","last_peered":"2023-07-06T20:18:59.654912+0000","last_clean":"2023-07-06T20:18:59.654912+0000","last_became_active":"2023-07-06T20:17:28.573161+0000","last_became_peered":"2023-07-06T20:17:28.573161+0000","last_unstale":"2023-07-06T20:18:59.654912+0000","last_undegraded":"2023-07-06T20:18:59.654912+0000","last_fullsized":"2023-07-06T20:18:59.654912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:04:10.288520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":137,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:05.587015+0000","last_change":"2023-07-06T20:19:00.579098+0000","last_active":"2023-07-06T20:19:05.587015+0000","last_peered":"2023-07-06T20:19:05.587015+0000","last_clean":"2023-07-06T20:19:05.587015+0000","last_became_active":"2023-07-06T20:19:00.575486+0000","last_became_peered":"2023-07-06T20:19:00.575486+0000","last_unstale":"2023-07-06T20:19:05.587015+0000","last_undegraded":"2023-07-06T20:19:05.587015+0000","last_fullsized":"2023-07-06T20:19:05.587015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:36:44.887142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1071,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.653581+0000","last_change":"2023-07-06T20:17:28.569745+0000","last_active":"2023-07-06T20:18:59.653581+0000","last_peered":"2023-07-06T20:18:59.653581+0000","last_clean":"2023-07-06T20:18:59.653581+0000","last_became_active":"2023-07-06T20:17:28.569468+0000","last_became_peered":"2023-07-06T20:17:28.569468+0000","last_unstale":"2023-07-06T20:18:59.653581+0000","last_undegraded":"2023-07-06T20:18:59.653581+0000","last_fullsized":"2023-07-06T20:18:59.653581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:04:39.612959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574256+0000","last_change":"2023-07-06T20:17:58.771936+0000","last_active":"2023-07-06T20:19:00.574256+0000","last_peered":"2023-07-06T20:19:00.574256+0000","last_clean":"2023-07-06T20:19:00.574256+0000","last_became_active":"2023-07-06T20:17:58.748196+0000","last_became_peered":"2023-07-06T20:17:58.748196+0000","last_unstale":"2023-07-06T20:19:00.574256+0000","last_undegraded":"2023-07-06T20:19:00.574256+0000","last_fullsized":"2023-07-06T20:19:00.574256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:20:15.168480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703012+0000","last_change":"2023-07-06T20:17:28.575390+0000","last_active":"2023-07-06T20:18:59.703012+0000","last_peered":"2023-07-06T20:18:59.703012+0000","last_clean":"2023-07-06T20:18:59.703012+0000","last_became_active":"2023-07-06T20:17:28.575083+0000","last_became_peered":"2023-07-06T20:17:28.575083+0000","last_unstale":"2023-07-06T20:18:59.703012+0000","last_undegraded":"2023-07-06T20:18:59.703012+0000","last_fullsized":"2023-07-06T20:18:59.703012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:05.579725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646574+0000","last_change":"2023-07-06T20:17:27.600891+0000","last_active":"2023-07-06T20:18:59.646574+0000","last_peered":"2023-07-06T20:18:59.646574+0000","last_clean":"2023-07-06T20:18:59.646574+0000","last_became_active":"2023-07-06T20:17:27.600533+0000","last_became_peered":"2023-07-06T20:17:27.600533+0000","last_unstale":"2023-07-06T20:18:59.646574+0000","last_undegraded":"2023-07-06T20:18:59.646574+0000","last_fullsized":"2023-07-06T20:18:59.646574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:44:00.501767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574645+0000","last_change":"2023-07-06T20:17:58.863073+0000","last_active":"2023-07-06T20:19:00.574645+0000","last_peered":"2023-07-06T20:19:00.574645+0000","last_clean":"2023-07-06T20:19:00.574645+0000","last_became_active":"2023-07-06T20:17:58.862613+0000","last_became_peered":"2023-07-06T20:17:58.862613+0000","last_unstale":"2023-07-06T20:19:00.574645+0000","last_undegraded":"2023-07-06T20:19:00.574645+0000","last_fullsized":"2023-07-06T20:19:00.574645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:21:52.224293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574359+0000","last_change":"2023-07-06T20:17:27.604464+0000","last_active":"2023-07-06T20:19:00.574359+0000","last_peered":"2023-07-06T20:19:00.574359+0000","last_clean":"2023-07-06T20:19:00.574359+0000","last_became_active":"2023-07-06T20:17:27.604036+0000","last_became_peered":"2023-07-06T20:17:27.604036+0000","last_unstale":"2023-07-06T20:19:00.574359+0000","last_undegraded":"2023-07-06T20:19:00.574359+0000","last_fullsized":"2023-07-06T20:19:00.574359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:59:41.175216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654818+0000","last_change":"2023-07-06T20:17:28.577302+0000","last_active":"2023-07-06T20:18:59.654818+0000","last_peered":"2023-07-06T20:18:59.654818+0000","last_clean":"2023-07-06T20:18:59.654818+0000","last_became_active":"2023-07-06T20:17:28.576998+0000","last_became_peered":"2023-07-06T20:17:28.576998+0000","last_unstale":"2023-07-06T20:18:59.654818+0000","last_undegraded":"2023-07-06T20:18:59.654818+0000","last_fullsized":"2023-07-06T20:18:59.654818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:30:35.224494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654890+0000","last_change":"2023-07-06T20:17:27.608391+0000","last_active":"2023-07-06T20:18:59.654890+0000","last_peered":"2023-07-06T20:18:59.654890+0000","last_clean":"2023-07-06T20:18:59.654890+0000","last_became_active":"2023-07-06T20:17:27.608073+0000","last_became_peered":"2023-07-06T20:17:27.608073+0000","last_unstale":"2023-07-06T20:18:59.654890+0000","last_undegraded":"2023-07-06T20:18:59.654890+0000","last_fullsized":"2023-07-06T20:18:59.654890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:58:17.469333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654516+0000","last_change":"2023-07-06T20:17:28.577554+0000","last_active":"2023-07-06T20:18:59.654516+0000","last_peered":"2023-07-06T20:18:59.654516+0000","last_clean":"2023-07-06T20:18:59.654516+0000","last_became_active":"2023-07-06T20:17:28.577208+0000","last_became_peered":"2023-07-06T20:17:28.577208+0000","last_unstale":"2023-07-06T20:18:59.654516+0000","last_undegraded":"2023-07-06T20:18:59.654516+0000","last_fullsized":"2023-07-06T20:18:59.654516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:11:56.614831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703008+0000","last_change":"2023-07-06T20:17:28.571678+0000","last_active":"2023-07-06T20:18:59.703008+0000","last_peered":"2023-07-06T20:18:59.703008+0000","last_clean":"2023-07-06T20:18:59.703008+0000","last_became_active":"2023-07-06T20:17:28.570974+0000","last_became_peered":"2023-07-06T20:17:28.570974+0000","last_unstale":"2023-07-06T20:18:59.703008+0000","last_undegraded":"2023-07-06T20:18:59.703008+0000","last_fullsized":"2023-07-06T20:18:59.703008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:53:32.536581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":121,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:07.589485+0000","last_change":"2023-07-06T20:17:27.607172+0000","last_active":"2023-07-06T20:19:07.589485+0000","last_peered":"2023-07-06T20:19:07.589485+0000","last_clean":"2023-07-06T20:19:07.589485+0000","last_became_active":"2023-07-06T20:17:27.606841+0000","last_became_peered":"2023-07-06T20:17:27.606841+0000","last_unstale":"2023-07-06T20:19:07.589485+0000","last_undegraded":"2023-07-06T20:19:07.589485+0000","last_fullsized":"2023-07-06T20:19:07.589485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:48:15.870982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574630+0000","last_change":"2023-07-06T20:17:28.567707+0000","last_active":"2023-07-06T20:19:00.574630+0000","last_peered":"2023-07-06T20:19:00.574630+0000","last_clean":"2023-07-06T20:19:00.574630+0000","last_became_active":"2023-07-06T20:17:28.567394+0000","last_became_peered":"2023-07-06T20:17:28.567394+0000","last_unstale":"2023-07-06T20:19:00.574630+0000","last_undegraded":"2023-07-06T20:19:00.574630+0000","last_fullsized":"2023-07-06T20:19:00.574630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:58:40.539787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.651620+0000","last_change":"2023-07-06T20:17:27.595746+0000","last_active":"2023-07-06T20:18:59.651620+0000","last_peered":"2023-07-06T20:18:59.651620+0000","last_clean":"2023-07-06T20:18:59.651620+0000","last_became_active":"2023-07-06T20:17:27.595039+0000","last_became_peered":"2023-07-06T20:17:27.595039+0000","last_unstale":"2023-07-06T20:18:59.651620+0000","last_undegraded":"2023-07-06T20:18:59.651620+0000","last_fullsized":"2023-07-06T20:18:59.651620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:00:27.347076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654306+0000","last_change":"2023-07-06T20:16:23.542324+0000","last_active":"2023-07-06T20:18:59.654306+0000","last_peered":"2023-07-06T20:18:59.654306+0000","last_clean":"2023-07-06T20:18:59.654306+0000","last_became_active":"2023-07-06T20:16:23.541832+0000","last_became_peered":"2023-07-06T20:16:23.541832+0000","last_unstale":"2023-07-06T20:18:59.654306+0000","last_undegraded":"2023-07-06T20:18:59.654306+0000","last_fullsized":"2023-07-06T20:18:59.654306+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_clean_scrub_stamp":"2023-07-06T20:13:14.072924+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:08:55.801420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645653+0000","last_change":"2023-07-06T20:17:28.577309+0000","last_active":"2023-07-06T20:18:59.645653+0000","last_peered":"2023-07-06T20:18:59.645653+0000","last_clean":"2023-07-06T20:18:59.645653+0000","last_became_active":"2023-07-06T20:17:28.576962+0000","last_became_peered":"2023-07-06T20:17:28.576962+0000","last_unstale":"2023-07-06T20:18:59.645653+0000","last_undegraded":"2023-07-06T20:18:59.645653+0000","last_fullsized":"2023-07-06T20:18:59.645653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:53.192162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654524+0000","last_change":"2023-07-06T20:17:27.601487+0000","last_active":"2023-07-06T20:18:59.654524+0000","last_peered":"2023-07-06T20:18:59.654524+0000","last_clean":"2023-07-06T20:18:59.654524+0000","last_became_active":"2023-07-06T20:17:27.601195+0000","last_became_peered":"2023-07-06T20:17:27.601195+0000","last_unstale":"2023-07-06T20:18:59.654524+0000","last_undegraded":"2023-07-06T20:18:59.654524+0000","last_fullsized":"2023-07-06T20:18:59.654524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:01:12.039199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646358+0000","last_change":"2023-07-06T20:17:27.610374+0000","last_active":"2023-07-06T20:18:59.646358+0000","last_peered":"2023-07-06T20:18:59.646358+0000","last_clean":"2023-07-06T20:18:59.646358+0000","last_became_active":"2023-07-06T20:17:27.610104+0000","last_became_peered":"2023-07-06T20:17:27.610104+0000","last_unstale":"2023-07-06T20:18:59.646358+0000","last_undegraded":"2023-07-06T20:18:59.646358+0000","last_fullsized":"2023-07-06T20:18:59.646358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:51:41.440974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645823+0000","last_change":"2023-07-06T20:17:28.570146+0000","last_active":"2023-07-06T20:18:59.645823+0000","last_peered":"2023-07-06T20:18:59.645823+0000","last_clean":"2023-07-06T20:18:59.645823+0000","last_became_active":"2023-07-06T20:17:28.569341+0000","last_became_peered":"2023-07-06T20:17:28.569341+0000","last_unstale":"2023-07-06T20:18:59.645823+0000","last_undegraded":"2023-07-06T20:18:59.645823+0000","last_fullsized":"2023-07-06T20:18:59.645823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:25:56.302477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:02.380205+0000","last_change":"2023-07-06T20:17:27.604630+0000","last_active":"2023-07-06T20:19:02.380205+0000","last_peered":"2023-07-06T20:19:02.380205+0000","last_clean":"2023-07-06T20:19:02.380205+0000","last_became_active":"2023-07-06T20:17:27.604118+0000","last_became_peered":"2023-07-06T20:17:27.604118+0000","last_unstale":"2023-07-06T20:19:02.380205+0000","last_undegraded":"2023-07-06T20:19:02.380205+0000","last_fullsized":"2023-07-06T20:19:02.380205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:04:38.195720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575728+0000","last_change":"2023-07-06T20:17:28.571753+0000","last_active":"2023-07-06T20:19:00.575728+0000","last_peered":"2023-07-06T20:19:00.575728+0000","last_clean":"2023-07-06T20:19:00.575728+0000","last_became_active":"2023-07-06T20:17:28.571073+0000","last_became_peered":"2023-07-06T20:17:28.571073+0000","last_unstale":"2023-07-06T20:19:00.575728+0000","last_undegraded":"2023-07-06T20:19:00.575728+0000","last_fullsized":"2023-07-06T20:19:00.575728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:49:51.350250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.651893+0000","last_change":"2023-07-06T20:17:27.594354+0000","last_active":"2023-07-06T20:18:59.651893+0000","last_peered":"2023-07-06T20:18:59.651893+0000","last_clean":"2023-07-06T20:18:59.651893+0000","last_became_active":"2023-07-06T20:17:27.594042+0000","last_became_peered":"2023-07-06T20:17:27.594042+0000","last_unstale":"2023-07-06T20:18:59.651893+0000","last_undegraded":"2023-07-06T20:18:59.651893+0000","last_fullsized":"2023-07-06T20:18:59.651893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:33:21.518458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575621+0000","last_change":"2023-07-06T20:17:28.576170+0000","last_active":"2023-07-06T20:19:00.575621+0000","last_peered":"2023-07-06T20:19:00.575621+0000","last_clean":"2023-07-06T20:19:00.575621+0000","last_became_active":"2023-07-06T20:17:28.575493+0000","last_became_peered":"2023-07-06T20:17:28.575493+0000","last_unstale":"2023-07-06T20:19:00.575621+0000","last_undegraded":"2023-07-06T20:19:00.575621+0000","last_fullsized":"2023-07-06T20:19:00.575621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:12:05.377174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648489+0000","last_change":"2023-07-06T20:17:27.602984+0000","last_active":"2023-07-06T20:18:59.648489+0000","last_peered":"2023-07-06T20:18:59.648489+0000","last_clean":"2023-07-06T20:18:59.648489+0000","last_became_active":"2023-07-06T20:17:27.602720+0000","last_became_peered":"2023-07-06T20:17:27.602720+0000","last_unstale":"2023-07-06T20:18:59.648489+0000","last_undegraded":"2023-07-06T20:18:59.648489+0000","last_fullsized":"2023-07-06T20:18:59.648489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:59:43.619260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646022+0000","last_change":"2023-07-06T20:17:28.565954+0000","last_active":"2023-07-06T20:18:59.646022+0000","last_peered":"2023-07-06T20:18:59.646022+0000","last_clean":"2023-07-06T20:18:59.646022+0000","last_became_active":"2023-07-06T20:17:28.565300+0000","last_became_peered":"2023-07-06T20:17:28.565300+0000","last_unstale":"2023-07-06T20:18:59.646022+0000","last_undegraded":"2023-07-06T20:18:59.646022+0000","last_fullsized":"2023-07-06T20:18:59.646022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:57:19.951265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575343+0000","last_change":"2023-07-06T20:17:27.597859+0000","last_active":"2023-07-06T20:19:00.575343+0000","last_peered":"2023-07-06T20:19:00.575343+0000","last_clean":"2023-07-06T20:19:00.575343+0000","last_became_active":"2023-07-06T20:17:27.597479+0000","last_became_peered":"2023-07-06T20:17:27.597479+0000","last_unstale":"2023-07-06T20:19:00.575343+0000","last_undegraded":"2023-07-06T20:19:00.575343+0000","last_fullsized":"2023-07-06T20:19:00.575343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:44:33.032535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574266+0000","last_change":"2023-07-06T20:17:28.573069+0000","last_active":"2023-07-06T20:19:00.574266+0000","last_peered":"2023-07-06T20:19:00.574266+0000","last_clean":"2023-07-06T20:19:00.574266+0000","last_became_active":"2023-07-06T20:17:28.572742+0000","last_became_peered":"2023-07-06T20:17:28.572742+0000","last_unstale":"2023-07-06T20:19:00.574266+0000","last_undegraded":"2023-07-06T20:19:00.574266+0000","last_fullsized":"2023-07-06T20:19:00.574266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:00:57.440359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073240+0000","last_change":"2023-07-06T20:17:27.599259+0000","last_active":"2023-07-06T20:19:00.073240+0000","last_peered":"2023-07-06T20:19:00.073240+0000","last_clean":"2023-07-06T20:19:00.073240+0000","last_became_active":"2023-07-06T20:17:27.598883+0000","last_became_peered":"2023-07-06T20:17:27.598883+0000","last_unstale":"2023-07-06T20:19:00.073240+0000","last_undegraded":"2023-07-06T20:19:00.073240+0000","last_fullsized":"2023-07-06T20:19:00.073240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:06:44.671018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575340+0000","last_change":"2023-07-06T20:17:28.573874+0000","last_active":"2023-07-06T20:19:00.575340+0000","last_peered":"2023-07-06T20:19:00.575340+0000","last_clean":"2023-07-06T20:19:00.575340+0000","last_became_active":"2023-07-06T20:17:28.573379+0000","last_became_peered":"2023-07-06T20:17:28.573379+0000","last_unstale":"2023-07-06T20:19:00.575340+0000","last_undegraded":"2023-07-06T20:19:00.575340+0000","last_fullsized":"2023-07-06T20:19:00.575340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:26:16.579909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646204+0000","last_change":"2023-07-06T20:17:27.601474+0000","last_active":"2023-07-06T20:18:59.646204+0000","last_peered":"2023-07-06T20:18:59.646204+0000","last_clean":"2023-07-06T20:18:59.646204+0000","last_became_active":"2023-07-06T20:17:27.601099+0000","last_became_peered":"2023-07-06T20:17:27.601099+0000","last_unstale":"2023-07-06T20:18:59.646204+0000","last_undegraded":"2023-07-06T20:18:59.646204+0000","last_fullsized":"2023-07-06T20:18:59.646204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:36:33.224709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654754+0000","last_change":"2023-07-06T20:17:28.571170+0000","last_active":"2023-07-06T20:18:59.654754+0000","last_peered":"2023-07-06T20:18:59.654754+0000","last_clean":"2023-07-06T20:18:59.654754+0000","last_became_active":"2023-07-06T20:17:28.570874+0000","last_became_peered":"2023-07-06T20:17:28.570874+0000","last_unstale":"2023-07-06T20:18:59.654754+0000","last_undegraded":"2023-07-06T20:18:59.654754+0000","last_fullsized":"2023-07-06T20:18:59.654754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:05.977773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073444+0000","last_change":"2023-07-06T20:17:27.603604+0000","last_active":"2023-07-06T20:19:00.073444+0000","last_peered":"2023-07-06T20:19:00.073444+0000","last_clean":"2023-07-06T20:19:00.073444+0000","last_became_active":"2023-07-06T20:17:27.603015+0000","last_became_peered":"2023-07-06T20:17:27.603015+0000","last_unstale":"2023-07-06T20:19:00.073444+0000","last_undegraded":"2023-07-06T20:19:00.073444+0000","last_fullsized":"2023-07-06T20:19:00.073444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:26:02.215516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654631+0000","last_change":"2023-07-06T20:17:28.567906+0000","last_active":"2023-07-06T20:18:59.654631+0000","last_peered":"2023-07-06T20:18:59.654631+0000","last_clean":"2023-07-06T20:18:59.654631+0000","last_became_active":"2023-07-06T20:17:28.567509+0000","last_became_peered":"2023-07-06T20:17:28.567509+0000","last_unstale":"2023-07-06T20:18:59.654631+0000","last_undegraded":"2023-07-06T20:18:59.654631+0000","last_fullsized":"2023-07-06T20:18:59.654631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:51:09.883831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703298+0000","last_change":"2023-07-06T20:17:27.598323+0000","last_active":"2023-07-06T20:18:59.703298+0000","last_peered":"2023-07-06T20:18:59.703298+0000","last_clean":"2023-07-06T20:18:59.703298+0000","last_became_active":"2023-07-06T20:17:27.598032+0000","last_became_peered":"2023-07-06T20:17:27.598032+0000","last_unstale":"2023-07-06T20:18:59.703298+0000","last_undegraded":"2023-07-06T20:18:59.703298+0000","last_fullsized":"2023-07-06T20:18:59.703298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:05.424357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654137+0000","last_change":"2023-07-06T20:17:28.571756+0000","last_active":"2023-07-06T20:18:59.654137+0000","last_peered":"2023-07-06T20:18:59.654137+0000","last_clean":"2023-07-06T20:18:59.654137+0000","last_became_active":"2023-07-06T20:17:28.571440+0000","last_became_peered":"2023-07-06T20:17:28.571440+0000","last_unstale":"2023-07-06T20:18:59.654137+0000","last_undegraded":"2023-07-06T20:18:59.654137+0000","last_fullsized":"2023-07-06T20:18:59.654137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:48.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645836+0000","last_change":"2023-07-06T20:17:27.602620+0000","last_active":"2023-07-06T20:18:59.645836+0000","last_peered":"2023-07-06T20:18:59.645836+0000","last_clean":"2023-07-06T20:18:59.645836+0000","last_became_active":"2023-07-06T20:17:27.602224+0000","last_became_peered":"2023-07-06T20:17:27.602224+0000","last_unstale":"2023-07-06T20:18:59.645836+0000","last_undegraded":"2023-07-06T20:18:59.645836+0000","last_fullsized":"2023-07-06T20:18:59.645836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:18:17.166344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654194+0000","last_change":"2023-07-06T20:17:28.569203+0000","last_active":"2023-07-06T20:18:59.654194+0000","last_peered":"2023-07-06T20:18:59.654194+0000","last_clean":"2023-07-06T20:18:59.654194+0000","last_became_active":"2023-07-06T20:17:28.568948+0000","last_became_peered":"2023-07-06T20:17:28.568948+0000","last_unstale":"2023-07-06T20:18:59.654194+0000","last_undegraded":"2023-07-06T20:18:59.654194+0000","last_fullsized":"2023-07-06T20:18:59.654194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:49:16.105664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574493+0000","last_change":"2023-07-06T20:17:28.568829+0000","last_active":"2023-07-06T20:19:00.574493+0000","last_peered":"2023-07-06T20:19:00.574493+0000","last_clean":"2023-07-06T20:19:00.574493+0000","last_became_active":"2023-07-06T20:17:28.568511+0000","last_became_peered":"2023-07-06T20:17:28.568511+0000","last_unstale":"2023-07-06T20:19:00.574493+0000","last_undegraded":"2023-07-06T20:19:00.574493+0000","last_fullsized":"2023-07-06T20:19:00.574493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:08.295954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073259+0000","last_change":"2023-07-06T20:17:58.863188+0000","last_active":"2023-07-06T20:19:00.073259+0000","last_peered":"2023-07-06T20:19:00.073259+0000","last_clean":"2023-07-06T20:19:00.073259+0000","last_became_active":"2023-07-06T20:17:58.862743+0000","last_became_peered":"2023-07-06T20:17:58.862743+0000","last_unstale":"2023-07-06T20:19:00.073259+0000","last_undegraded":"2023-07-06T20:19:00.073259+0000","last_fullsized":"2023-07-06T20:19:00.073259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:07:20.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073191+0000","last_change":"2023-07-06T20:17:28.572264+0000","last_active":"2023-07-06T20:19:00.073191+0000","last_peered":"2023-07-06T20:19:00.073191+0000","last_clean":"2023-07-06T20:19:00.073191+0000","last_became_active":"2023-07-06T20:17:28.571914+0000","last_became_peered":"2023-07-06T20:17:28.571914+0000","last_unstale":"2023-07-06T20:19:00.073191+0000","last_undegraded":"2023-07-06T20:19:00.073191+0000","last_fullsized":"2023-07-06T20:19:00.073191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:01:02.267552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575399+0000","last_change":"2023-07-06T20:17:28.581374+0000","last_active":"2023-07-06T20:19:00.575399+0000","last_peered":"2023-07-06T20:19:00.575399+0000","last_clean":"2023-07-06T20:19:00.575399+0000","last_became_active":"2023-07-06T20:17:28.580995+0000","last_became_peered":"2023-07-06T20:17:28.580995+0000","last_unstale":"2023-07-06T20:19:00.575399+0000","last_undegraded":"2023-07-06T20:19:00.575399+0000","last_fullsized":"2023-07-06T20:19:00.575399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:08:18.718991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645952+0000","last_change":"2023-07-06T20:17:28.578926+0000","last_active":"2023-07-06T20:18:59.645952+0000","last_peered":"2023-07-06T20:18:59.645952+0000","last_clean":"2023-07-06T20:18:59.645952+0000","last_became_active":"2023-07-06T20:17:28.578620+0000","last_became_peered":"2023-07-06T20:17:28.578620+0000","last_unstale":"2023-07-06T20:18:59.645952+0000","last_undegraded":"2023-07-06T20:18:59.645952+0000","last_fullsized":"2023-07-06T20:18:59.645952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:17:06.451679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574477+0000","last_change":"2023-07-06T20:17:58.864937+0000","last_active":"2023-07-06T20:19:00.574477+0000","last_peered":"2023-07-06T20:19:00.574477+0000","last_clean":"2023-07-06T20:19:00.574477+0000","last_became_active":"2023-07-06T20:17:58.864457+0000","last_became_peered":"2023-07-06T20:17:58.864457+0000","last_unstale":"2023-07-06T20:19:00.574477+0000","last_undegraded":"2023-07-06T20:19:00.574477+0000","last_fullsized":"2023-07-06T20:19:00.574477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:53:40.789746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073661+0000","last_change":"2023-07-06T20:17:28.567965+0000","last_active":"2023-07-06T20:19:00.073661+0000","last_peered":"2023-07-06T20:19:00.073661+0000","last_clean":"2023-07-06T20:19:00.073661+0000","last_became_active":"2023-07-06T20:17:28.567618+0000","last_became_peered":"2023-07-06T20:17:28.567618+0000","last_unstale":"2023-07-06T20:19:00.073661+0000","last_undegraded":"2023-07-06T20:19:00.073661+0000","last_fullsized":"2023-07-06T20:19:00.073661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:20:53.386538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575649+0000","last_change":"2023-07-06T20:17:28.571934+0000","last_active":"2023-07-06T20:19:00.575649+0000","last_peered":"2023-07-06T20:19:00.575649+0000","last_clean":"2023-07-06T20:19:00.575649+0000","last_became_active":"2023-07-06T20:17:28.571472+0000","last_became_peered":"2023-07-06T20:17:28.571472+0000","last_unstale":"2023-07-06T20:19:00.575649+0000","last_undegraded":"2023-07-06T20:19:00.575649+0000","last_fullsized":"2023-07-06T20:19:00.575649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:22.858255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.702897+0000","last_change":"2023-07-06T20:17:28.571904+0000","last_active":"2023-07-06T20:18:59.702897+0000","last_peered":"2023-07-06T20:18:59.702897+0000","last_clean":"2023-07-06T20:18:59.702897+0000","last_became_active":"2023-07-06T20:17:28.571375+0000","last_became_peered":"2023-07-06T20:17:28.571375+0000","last_unstale":"2023-07-06T20:18:59.702897+0000","last_undegraded":"2023-07-06T20:18:59.702897+0000","last_fullsized":"2023-07-06T20:18:59.702897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:32:15.205545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575210+0000","last_change":"2023-07-06T20:17:28.569139+0000","last_active":"2023-07-06T20:19:00.575210+0000","last_peered":"2023-07-06T20:19:00.575210+0000","last_clean":"2023-07-06T20:19:00.575210+0000","last_became_active":"2023-07-06T20:17:28.568828+0000","last_became_peered":"2023-07-06T20:17:28.568828+0000","last_unstale":"2023-07-06T20:19:00.575210+0000","last_undegraded":"2023-07-06T20:19:00.575210+0000","last_fullsized":"2023-07-06T20:19:00.575210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:32.012178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648570+0000","last_change":"2023-07-06T20:17:28.562352+0000","last_active":"2023-07-06T20:18:59.648570+0000","last_peered":"2023-07-06T20:18:59.648570+0000","last_clean":"2023-07-06T20:18:59.648570+0000","last_became_active":"2023-07-06T20:17:28.562128+0000","last_became_peered":"2023-07-06T20:17:28.562128+0000","last_unstale":"2023-07-06T20:18:59.648570+0000","last_undegraded":"2023-07-06T20:18:59.648570+0000","last_fullsized":"2023-07-06T20:18:59.648570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:18:12.118763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645970+0000","last_change":"2023-07-06T20:17:28.578940+0000","last_active":"2023-07-06T20:18:59.645970+0000","last_peered":"2023-07-06T20:18:59.645970+0000","last_clean":"2023-07-06T20:18:59.645970+0000","last_became_active":"2023-07-06T20:17:28.578660+0000","last_became_peered":"2023-07-06T20:17:28.578660+0000","last_unstale":"2023-07-06T20:18:59.645970+0000","last_undegraded":"2023-07-06T20:18:59.645970+0000","last_fullsized":"2023-07-06T20:18:59.645970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:00:23.290920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574415+0000","last_change":"2023-07-06T20:17:28.573964+0000","last_active":"2023-07-06T20:19:00.574415+0000","last_peered":"2023-07-06T20:19:00.574415+0000","last_clean":"2023-07-06T20:19:00.574415+0000","last_became_active":"2023-07-06T20:17:28.573618+0000","last_became_peered":"2023-07-06T20:17:28.573618+0000","last_unstale":"2023-07-06T20:19:00.574415+0000","last_undegraded":"2023-07-06T20:19:00.574415+0000","last_fullsized":"2023-07-06T20:19:00.574415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:06:11.474673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63664,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":589,"num_read_kb":362,"num_write":162,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1183,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":202062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449121,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":462935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.72599999999999998,"15min":0.67300000000000004},"min":{"1min":0.45600000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":4.968,"5min":4.968,"15min":4.968},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.04,"5min":0.71899999999999997,"15min":0.66500000000000004},"min":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":5.0019999999999998,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63800000000000001,"15min":0.629},"min":{"1min":0.45300000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.79700000000000004,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.36599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.72399999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69699999999999995,"15min":0.69099999999999995},"min":{"1min":0.48899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.622,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":1.006,"15min":1.006},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66400000000000003,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92800000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.47699999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90400000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.89100000000000001,"5min":1.097,"15min":1.097},"last":0.752}]},{"osd":7,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78900000000000003,"15min":0.79500000000000004},"min":{"1min":0.53700000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.98599999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68000000000000005}]},{"osd":8,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.80800000000000005}]},{"osd":9,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.60999999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.96799999999999997,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.89800000000000002,"15min":0.86799999999999999},"min":{"1min":0.67000000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":0.84399999999999997}]},{"osd":10,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78100000000000003,"15min":0.78800000000000003},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88200000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.66200000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97099999999999997,"5min":1.208,"15min":1.208},"last":0.65400000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1616,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1654784,"data_stored":1112186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.66700000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.434,"15min":0.436},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55000000000000004,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67600000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.499}]},{"osd":2,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86599999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.46200000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.624},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.32500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67700000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.496,"5min":0.439,"15min":0.439},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67500000000000004,"15min":0.67300000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.751}]},{"osd":7,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.48099999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.875,"5min":0.998,"15min":0.998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74299999999999999,"15min":0.748},"min":{"1min":0.57099999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.71899999999999997}]},{"osd":9,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.94699999999999995,"15min":0.98099999999999998},"min":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93999999999999995,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.91100000000000003,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.81399999999999995}]},{"osd":11,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73999999999999999,"15min":0.73899999999999999},"min":{"1min":0.57699999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.0049999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.872},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78300000000000003,"15min":0.78200000000000003},"min":{"1min":0.54200000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.85499999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":461676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.49299999999999999,"15min":0.503},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.93600000000000005,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83499999999999996,"5min":5.7919999999999998,"15min":5.7919999999999998},"last":0.435}]},{"osd":1,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.69099999999999995,"15min":0.70999999999999996},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89600000000000002,"5min":1.202,"15min":1.202},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.66800000000000004,"15min":0.68500000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.81999999999999995}]},{"osd":2,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.98499999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69699999999999995,"15min":0.71099999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.72699999999999998,"15min":0.66900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2729999999999999,"5min":16.628,"15min":16.628},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.083,"5min":11.332000000000001,"15min":11.332000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69399999999999995,"15min":0.70499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.73599999999999999,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.78900000000000003,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.80700000000000005,"15min":0.83299999999999996},"min":{"1min":0.51400000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91200000000000003,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79000000000000004,"15min":0.81599999999999995},"min":{"1min":0.58999999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.016,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.76500000000000001,"15min":0.78500000000000003},"min":{"1min":0.54100000000000004,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.115,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.77500000000000002,"15min":0.82499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84499999999999997,"15min":0.88},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.85899999999999999,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.86499999999999999,"15min":0.90400000000000003},"min":{"1min":0.625,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.054,"5min":1.339,"15min":1.339},"last":0.76700000000000002}]},{"osd":8,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.88200000000000001},"min":{"1min":0.504,"5min":0.496,"15min":0.496},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.751},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.877,"15min":0.89600000000000002},"min":{"1min":0.60299999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2230000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":0.72799999999999998}]},{"osd":10,"last update":"Thu Jul 6 20:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.85799999999999998,"15min":0.84999999999999998},"min":{"1min":0.68700000000000006,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.80500000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103213,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":461203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58699999999999997}]},{"osd":1,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.60099999999999998,"15min":0.57599999999999996},"min":{"1min":0.54800000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.48199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81000000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.45900000000000002,"15min":0.44800000000000001},"min":{"1min":0.34300000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.48099999999999998,"15min":0.47199999999999998},"min":{"1min":0.32400000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71299999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.48099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67700000000000005},"min":{"1min":0.54700000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.629,"15min":0.61399999999999999},"min":{"1min":0.48099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.66800000000000004,"15min":0.64700000000000002},"min":{"1min":0.621,"5min":0.433,"15min":0.433},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.89300000000000002}]},{"osd":5,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.622,"15min":0.60099999999999998},"min":{"1min":0.40200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.97999999999999998,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.74299999999999999,"15min":0.72099999999999997},"min":{"1min":0.64700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.73799999999999999,"15min":0.71999999999999997},"min":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.66100000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.0489999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.86799999999999999,"15min":0.90600000000000003},"min":{"1min":0.56499999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.091,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.95999999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.78800000000000003,"15min":0.74399999999999999},"min":{"1min":0.628,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.044},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.80500000000000005,"15min":0.76900000000000002},"min":{"1min":0.64400000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.97199999999999998}]},{"osd":10,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.76400000000000001,"15min":0.73899999999999999},"min":{"1min":0.69999999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.022},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.77300000000000002,"15min":0.75700000000000001},"min":{"1min":0.70399999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.0049999999999999}]},{"osd":11,"last update":"Thu Jul 6 20:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.996}]}]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1528,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1564672,"data_stored":1053036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51300000000000001,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85299999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.39400000000000002,"15min":0.38400000000000001},"min":{"1min":0.34699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.41299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.625,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.374}]},{"osd":2,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.40400000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80400000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.68000000000000005,"15min":0.622},"min":{"1min":0.40100000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.034,"5min":0.67300000000000004,"15min":0.59599999999999997},"min":{"1min":0.47899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.048,"5min":6.048,"15min":6.048},"last":0.53700000000000003}]},{"osd":4,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.32300000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.502},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.499,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59299999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.44700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.755,"15min":0.68100000000000005},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":8.1579999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.77000000000000002,"15min":0.70299999999999996},"min":{"1min":0.48799999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":8.1980000000000004,"5min":8.1980000000000004,"15min":8.1980000000000004},"last":0.66800000000000004}]},{"osd":8,"last update":"Thu Jul 6 20:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64600000000000002,"15min":0.63},"min":{"1min":0.47799999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.66600000000000004,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90500000000000003,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65400000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70599999999999996,"15min":0.70299999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71999999999999997,"15min":0.70399999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95099999999999996,"5min":1.419,"15min":1.419},"last":0.64000000000000001}]},{"osd":10,"last update":"Thu Jul 6 20:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.55300000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":1.014,"15min":1.014},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.54200000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54800000000000004}]},{"osd":11,"last update":"Thu Jul 6 20:18:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.755},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.73999999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":461071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49099999999999999,"15min":0.47399999999999998},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.91600000000000004,"5min":1.552,"15min":1.552},"last":0.875},{"interface":"front","average":{"1min":0.439,"5min":0.496,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66400000000000003,"5min":1.389,"15min":1.389},"last":0.53300000000000003}]},{"osd":1,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.71599999999999997,"15min":0.66800000000000004},"min":{"1min":0.55600000000000005,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.66800000000000004,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91000000000000003,"5min":1.089,"15min":1.089},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.68500000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.755},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94699999999999995,"5min":1.04,"15min":1.04},"last":0.93700000000000006}]},{"osd":3,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.60199999999999998,"15min":0.622},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.98999999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.95499999999999996,"5min":1.254,"15min":1.254},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86399999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.623}]},{"osd":5,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72999999999999998,"15min":0.76000000000000001},"min":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0109999999999999,"5min":1.393,"15min":1.393},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.75600000000000001,"15min":0.76100000000000001},"min":{"1min":0.505,"5min":0.496,"15min":0.496},"max":{"1min":0.90600000000000003,"5min":1.131,"15min":1.131},"last":1.0229999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.79400000000000004,"15min":0.77800000000000002},"min":{"1min":0.55100000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.008},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.78700000000000003,"15min":0.78500000000000003},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.95599999999999996,"5min":1.274,"15min":1.274},"last":0.68799999999999994}]},{"osd":8,"last update":"Thu Jul 6 20:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81399999999999995,"15min":0.84399999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.077,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80300000000000005,"15min":0.82299999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0029999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Thu Jul 6 20:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.56299999999999994,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.05},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.89000000000000001,"15min":0.90200000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":1.091}]},{"osd":10,"last update":"Thu Jul 6 20:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92200000000000004,"15min":0.91400000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":3.8410000000000002,"5min":3.8410000000000002,"15min":3.8410000000000002},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.88100000000000001,"15min":0.89700000000000002},"min":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0269999999999999,"5min":1.244,"15min":1.244},"last":1.0780000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":520058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.495},"min":{"1min":0.32700000000000001,"5min":0.318,"15min":0.248},"max":{"1min":0.82199999999999995,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.53100000000000003},"min":{"1min":0.44600000000000001,"5min":0.34499999999999997,"15min":0.248},"max":{"1min":0.86099999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56899999999999995,"15min":0.51300000000000001},"min":{"1min":0.41599999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.93600000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.59399999999999997,"15min":0.52000000000000002},"min":{"1min":0.60699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94999999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.997}]},{"osd":3,"last update":"Thu Jul 6 20:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60499999999999998,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.90200000000000002,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.56499999999999995},"min":{"1min":0.42499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89000000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61499999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.42899999999999999,"15min":0.39300000000000002},"min":{"1min":0.33800000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.46999999999999997,"15min":0.42599999999999999},"min":{"1min":0.33500000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jul 6 20:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.60099999999999998,"15min":0.54900000000000004},"min":{"1min":0.55300000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90000000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.64100000000000001,"15min":0.57499999999999996},"min":{"1min":0.58999999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0289999999999999,"5min":1.042,"15min":1.042},"last":0.92900000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69699999999999995,"15min":0.66300000000000003},"min":{"1min":0.52400000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.437,"5min":0.372,"15min":0.372},"max":{"1min":0.874,"5min":1.653,"15min":1.653},"last":0.97999999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.624,"15min":0.59199999999999997},"min":{"1min":0.42399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.59599999999999997,"15min":0.55500000000000005},"min":{"1min":0.5,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.0309999999999999}]},{"osd":8,"last update":"Thu Jul 6 20:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.69999999999999996,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96999999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.77000000000000002,"15min":0.73399999999999999},"min":{"1min":0.55100000000000005,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":1.101}]},{"osd":9,"last update":"Thu Jul 6 20:19:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.76600000000000001,"15min":0.746},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0129999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78900000000000003,"15min":0.77100000000000002},"min":{"1min":0.56000000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0189999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.014}]},{"osd":10,"last update":"Thu Jul 6 20:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76900000000000002,"15min":0.76100000000000001},"min":{"1min":0.47999999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.82799999999999996,"15min":0.81599999999999995},"min":{"1min":0.67300000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.86099999999999999}]},{"osd":11,"last update":"Thu Jul 6 20:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":462336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.317,"15min":0.317},"max":{"1min":4.1130000000000004,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.628,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.6400000000000001,"5min":4.1379999999999999,"15min":4.1379999999999999},"last":0.499}]},{"osd":2,"last update":"Thu Jul 6 20:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.63700000000000001,"15min":0.58999999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91100000000000003,"5min":1.085,"15min":1.085},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.66000000000000003,"15min":0.58599999999999997},"min":{"1min":0.44600000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97199999999999998,"5min":1.258,"15min":1.258},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.73799999999999999},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0049999999999999,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jul 6 20:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67500000000000004,"15min":0.64500000000000002},"min":{"1min":0.51300000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91000000000000003,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.46800000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95599999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.69299999999999995},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.03,"5min":1.131,"15min":1.131},"last":0.505},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79700000000000004,"15min":0.79600000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.179,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.84399999999999997}]},{"osd":8,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.78900000000000003,"15min":0.78600000000000003},"min":{"1min":0.54800000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.7650000000000001,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79700000000000004,"15min":0.81899999999999995},"min":{"1min":0.54100000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.073,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.65400000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.79000000000000004,"15min":0.82199999999999995},"min":{"1min":0.51100000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97799999999999998,"5min":1.161,"15min":1.161},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77500000000000002,"15min":0.79100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.7130000000000001,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.63100000000000001}]},{"osd":10,"last update":"Thu Jul 6 20:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.877,"15min":0.879},"min":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.681,"5min":2.681,"15min":2.681},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.86699999999999999,"15min":0.85499999999999998},"min":{"1min":0.54400000000000004,"5min":0.498,"15min":0.498},"max":{"1min":3.9630000000000001,"5min":3.9630000000000001,"15min":3.9630000000000001},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.83299999999999996,"15min":0.81399999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.9209999999999998,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.111,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.89800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":1044480,"data_stored":527046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.33500000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.89400000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84099999999999997,"5min":0.876,"15min":0.876},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.39800000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.87,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":1,"15min":1},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83499999999999996,"5min":0.995,"15min":0.995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.50600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jul 6 20:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49299999999999999,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77200000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.34499999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78400000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66500000000000004,"15min":0.64300000000000002},"min":{"1min":0.53100000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.97599999999999998,"5min":1.056,"15min":1.056},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.70599999999999996,"15min":0.66000000000000003},"min":{"1min":0.60399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0149999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.92700000000000005}]},{"osd":7,"last update":"Thu Jul 6 20:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.624},"min":{"1min":0.435,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81899999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91000000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.432,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.89100000000000001}]},{"osd":9,"last update":"Thu Jul 6 20:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.77300000000000002,"15min":0.76200000000000001},"min":{"1min":0.48099999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.81399999999999995,"15min":0.79600000000000004},"min":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.98099999999999998}]},{"osd":10,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.746,"15min":0.73499999999999999},"min":{"1min":0.56299999999999994,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.98999999999999999}]},{"osd":11,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.94499999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":461388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41399999999999998,"15min":0.38500000000000001},"min":{"1min":0.30499999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.51600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.45700000000000002,"15min":0.40999999999999998},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.36699999999999999}]},{"osd":1,"last update":"Thu Jul 6 20:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65800000000000003,"15min":0.58499999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81599999999999995,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.5},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59799999999999998,"15min":0.57199999999999995},"min":{"1min":0.45200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95599999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:18:50 2023","interfaces":[{"interface":"back","average":{"1min":2.181,"5min":0.94199999999999995,"15min":0.69599999999999995},"min":{"1min":0.51900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":36.344000000000001,"5min":36.344000000000001,"15min":36.344000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67600000000000005,"15min":0.625},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93700000000000006,"5min":3.6720000000000002,"15min":3.6720000000000002},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jul 6 20:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.73199999999999998,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":6.9420000000000002,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.58699999999999997},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.442}]},{"osd":5,"last update":"Thu Jul 6 20:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.70599999999999996,"15min":0.66300000000000003},"min":{"1min":0.48899999999999999,"5min":0.441,"15min":0.441},"max":{"1min":4.5389999999999997,"5min":4.5389999999999997,"15min":4.5389999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66800000000000004,"15min":0.63},"min":{"1min":0.51600000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64200000000000002,"15min":0.61799999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86599999999999999,"5min":1.052,"15min":1.052},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.56799999999999995},"min":{"1min":0.40799999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95199999999999996,"5min":0.998,"15min":0.998},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.79900000000000004,"15min":0.76500000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.1859999999999999,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":0.73199999999999998}]},{"osd":8,"last update":"Thu Jul 6 20:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.58299999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:19:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.77300000000000002,"15min":0.80200000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92400000000000004,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.71699999999999997,"15min":0.73499999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.71599999999999997}]},{"osd":10,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.046,"5min":1.173,"15min":1.173},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.077,"5min":0.89300000000000002,"15min":0.86199999999999999},"min":{"1min":0.68999999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":5.8040000000000003,"5min":5.8040000000000003,"15min":5.8040000000000003},"last":0.70599999999999996}]},{"osd":11,"last update":"Thu Jul 6 20:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":6.79,"5min":6.79,"15min":6.79},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.752}]}]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":462297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.55200000000000005,"15min":0.501},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":6.1539999999999999,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.51900000000000002,"15min":0.49299999999999999},"min":{"1min":0.45700000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.35799999999999998},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58399999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.379},{"interface":"front","average":{"1min":0.44,"5min":0.39600000000000002,"15min":0.35799999999999998},"min":{"1min":0.311,"5min":0.219,"15min":0.219},"max":{"1min":0.55900000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.40500000000000003}]},{"osd":2,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.47199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.753},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.44400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86699999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.53600000000000003},"min":{"1min":0.52200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.099,"5min":5.593,"15min":5.593},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.123,"5min":0.71699999999999997,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.378,"15min":0.378},"max":{"1min":9.7959999999999994,"5min":9.7959999999999994,"15min":9.7959999999999994},"last":0.44}]},{"osd":5,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.61599999999999999,"15min":0.56699999999999995},"min":{"1min":0.57599999999999996,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.63100000000000001,"15min":0.57999999999999996},"min":{"1min":0.47999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90900000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.40899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91600000000000004,"5min":1.109,"15min":1.109},"last":0.621},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.004,"5min":1.163,"15min":1.163},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.35499999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.97699999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.372,"5min":0.314,"15min":0.314},"max":{"1min":0.91200000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Thu Jul 6 20:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.754,"15min":0.74199999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0780000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.52300000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96599999999999997,"5min":1.194,"15min":1.194},"last":0.76500000000000001}]},{"osd":9,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71699999999999997,"15min":0.72499999999999998},"min":{"1min":0.54900000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.88100000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.92000000000000004,"15min":0.86899999999999999},"min":{"1min":0.63900000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0329999999999999,"5min":8.3369999999999997,"15min":8.3369999999999997},"last":0.81699999999999995}]},{"osd":10,"last update":"Thu Jul 6 20:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.82999999999999996,"15min":0.83299999999999996},"min":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.054,"5min":1.258,"15min":1.258},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.55300000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89700000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.71499999999999997}]},{"osd":11,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79300000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1576960,"data_stored":1052977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.53000000000000003},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81699999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.79200000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.42899999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.41299999999999998,"15min":0.40100000000000002},"min":{"1min":0.315,"5min":0.253,"15min":0.253},"max":{"1min":0.66100000000000003,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.31900000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.76400000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.58099999999999996},"min":{"1min":0.497,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.747},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.44800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.70299999999999996}]},{"osd":6,"last update":"Thu Jul 6 20:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.435,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66400000000000003,"15min":0.64200000000000002},"min":{"1min":0.51700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.85199999999999998,"5min":1.01,"15min":1.01},"last":0.76800000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.81799999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.46400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.45200000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.78200000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.61899999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.59799999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.84199999999999997}]},{"osd":10,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.53300000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.92600000000000005,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.57699999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.96599999999999997,"5min":1.109,"15min":1.109},"last":0.88700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:19:11.020 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-06T20:19:11.021 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T20:19:11.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:10 smithi156 ceph-mon[129280]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 716 B/s wr, 6 op/s; 107 B/s, 0 objects/s recovering 2023-07-06T20:19:11.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:10 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/4014404632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:19:11.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:10 smithi098 ceph-mon[128968]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 716 B/s wr, 6 op/s; 107 B/s, 0 objects/s recovering 2023-07-06T20:19:11.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:10 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/4014404632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:19:11.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.243+0000 7f3fe1dff700 1 -- 172.21.15.69:0/3947547412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fdc0ffc30 msgr2=0x7f3fdc100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:11.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.243+0000 7f3fe1dff700 1 --2- 172.21.15.69:0/3947547412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fdc0ffc30 0x7f3fdc100040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009990 tx=0x7f3fcc009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.243+0000 7f3fe1dff700 1 -- 172.21.15.69:0/3947547412 shutdown_connections 2023-07-06T20:19:11.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.243+0000 7f3fe1dff700 1 --2- 172.21.15.69:0/3947547412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc1017b0 0x7f3fdc108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.243+0000 7f3fe1dff700 1 --2- 172.21.15.69:0/3947547412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fdc0ffc30 0x7f3fdc100040 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.243+0000 7f3fe1dff700 1 --2- 172.21.15.69:0/3947547412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc100e20 0x7f3fdc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.243+0000 7f3fe1dff700 1 -- 172.21.15.69:0/3947547412 >> 172.21.15.69:0/3947547412 conn(0x7f3fdc0fb210 msgr2=0x7f3fdc0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:11.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.243+0000 7f3fe1dff700 1 -- 172.21.15.69:0/3947547412 shutdown_connections 2023-07-06T20:19:11.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.243+0000 7f3fe1dff700 1 -- 172.21.15.69:0/3947547412 wait complete. 2023-07-06T20:19:11.247 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.244+0000 7f3fe1dff700 1 Processor -- start 2023-07-06T20:19:11.247 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.244+0000 7f3fe1dff700 1 -- start start 2023-07-06T20:19:11.247 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.244+0000 7f3fe1dff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fdc0ffc30 0x7f3fdc197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:11.247 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.244+0000 7f3fe1dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc100e20 0x7f3fdc1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:11.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.244+0000 7f3fe1dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc1017b0 0x7f3fdc19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:11.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.244+0000 7f3fe1dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc19f720 con 0x7f3fdc100e20 2023-07-06T20:19:11.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.244+0000 7f3fe1dff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc19f860 con 0x7f3fdc0ffc30 2023-07-06T20:19:11.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.244+0000 7f3fe1dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc19f9f0 con 0x7f3fdc1017b0 2023-07-06T20:19:11.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.245+0000 7f3fdaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc100e20 0x7f3fdc1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:11.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.245+0000 7f3fdaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc100e20 0x7f3fdc1980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58436/0 (socket says 172.21.15.69:58436) 2023-07-06T20:19:11.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.245+0000 7f3fdaffd700 1 -- 172.21.15.69:0/543935338 learned_addr learned my addr 172.21.15.69:0/543935338 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.245+0000 7f3fdb7fe700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fdc0ffc30 0x7f3fdc197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.245+0000 7f3fdbfff700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc1017b0 0x7f3fdc19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.245+0000 7f3fdaffd700 1 -- 172.21.15.69:0/543935338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc1017b0 msgr2=0x7f3fdc19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.245+0000 7f3fdaffd700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc1017b0 0x7f3fdc19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.245+0000 7f3fdaffd700 1 -- 172.21.15.69:0/543935338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fdc0ffc30 msgr2=0x7f3fdc197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.245+0000 7f3fdaffd700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fdc0ffc30 0x7f3fdc197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.245+0000 7f3fdaffd700 1 -- 172.21.15.69:0/543935338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fcc0095f0 con 0x7f3fdc100e20 2023-07-06T20:19:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.245+0000 7f3fdb7fe700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fdc0ffc30 0x7f3fdc197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:11.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.245+0000 7f3fdaffd700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc100e20 0x7f3fdc1980e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000e9a0 tx=0x7f3fd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:11.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.246+0000 7f3fd8ff9700 1 -- 172.21.15.69:0/543935338 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0009790 con 0x7f3fdc100e20 2023-07-06T20:19:11.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.246+0000 7f3fe1dff700 1 -- 172.21.15.69:0/543935338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc19fcd0 con 0x7f3fdc100e20 2023-07-06T20:19:11.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.246+0000 7f3fd8ff9700 1 -- 172.21.15.69:0/543935338 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fd0010430 con 0x7f3fdc100e20 2023-07-06T20:19:11.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.246+0000 7f3fd8ff9700 1 -- 172.21.15.69:0/543935338 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0018680 con 0x7f3fdc100e20 2023-07-06T20:19:11.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.246+0000 7f3fe1dff700 1 -- 172.21.15.69:0/543935338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc1a0180 con 0x7f3fdc100e20 2023-07-06T20:19:11.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.247+0000 7f3fe1dff700 1 -- 172.21.15.69:0/543935338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc192310 con 0x7f3fdc100e20 2023-07-06T20:19:11.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.249+0000 7f3fd8ff9700 1 -- 172.21.15.69:0/543935338 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3fd00105a0 con 0x7f3fdc100e20 2023-07-06T20:19:11.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.250+0000 7f3fd8ff9700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fc40a11a0 0x7f3fc40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:11.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.250+0000 7f3fd8ff9700 1 -- 172.21.15.69:0/543935338 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f3fd0014070 con 0x7f3fdc100e20 2023-07-06T20:19:11.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.251+0000 7f3fdb7fe700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fc40a11a0 0x7f3fc40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:11.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.251+0000 7f3fdb7fe700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fc40a11a0 0x7f3fc40a3650 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00b2e0 tx=0x7f3fcc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:11.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.254+0000 7f3fd8ff9700 1 -- 172.21.15.69:0/543935338 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3fd0051770 con 0x7f3fdc100e20 2023-07-06T20:19:11.448 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.446+0000 7f3fe1dff700 1 -- 172.21.15.69:0/543935338 --> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3fdc060a80 con 0x7f3fc40a11a0 2023-07-06T20:19:11.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.457+0000 7f3fd8ff9700 1 -- 172.21.15.69:0/543935338 <== mgr.14152 v2:172.21.15.69:6800/4215258217 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344244 (secure 0 0 0) 0x7f3fdc060a80 con 0x7f3fc40a11a0 2023-07-06T20:19:11.464 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:11.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.462+0000 7f3fe1dff700 1 -- 172.21.15.69:0/543935338 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fc40a11a0 msgr2=0x7f3fc40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:11.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.462+0000 7f3fe1dff700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fc40a11a0 0x7f3fc40a3650 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00b2e0 tx=0x7f3fcc005fb0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.462+0000 7f3fe1dff700 1 -- 172.21.15.69:0/543935338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc100e20 msgr2=0x7f3fdc1980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:11.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.462+0000 7f3fe1dff700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc100e20 0x7f3fdc1980e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000e9a0 tx=0x7f3fd000ecb0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.462+0000 7f3fe1dff700 1 -- 172.21.15.69:0/543935338 shutdown_connections 2023-07-06T20:19:11.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.462+0000 7f3fe1dff700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3fc40a11a0 0x7f3fc40a3650 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.462+0000 7f3fe1dff700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fdc1017b0 0x7f3fdc19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.462+0000 7f3fe1dff700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fdc0ffc30 0x7f3fdc197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.462+0000 7f3fe1dff700 1 --2- 172.21.15.69:0/543935338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fdc100e20 0x7f3fdc1980e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.462+0000 7f3fe1dff700 1 -- 172.21.15.69:0/543935338 >> 172.21.15.69:0/543935338 conn(0x7f3fdc0fb210 msgr2=0x7f3fdc106710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:11.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.462+0000 7f3fe1dff700 1 -- 172.21.15.69:0/543935338 shutdown_connections 2023-07-06T20:19:11.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.462+0000 7f3fe1dff700 1 -- 172.21.15.69:0/543935338 wait complete. 2023-07-06T20:19:11.471 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2023-07-06T20:19:11.481 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2023-07-06T20:19:09.545348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654278,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":715,"num_read_kb":471,"num_write":345,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181919,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336776,"kb_used_data":13352,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605048,"statfs":{"total":1151940427776,"available":1151595569152,"internally_reserved":0,"allocated":13672448,"data_stored":7498209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":702,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":31,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1071,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002553"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574765+0000","last_change":"2023-07-06T20:17:28.575520+0000","last_active":"2023-07-06T20:19:00.574765+0000","last_peered":"2023-07-06T20:19:00.574765+0000","last_clean":"2023-07-06T20:19:00.574765+0000","last_became_active":"2023-07-06T20:17:28.575209+0000","last_became_peered":"2023-07-06T20:17:28.575209+0000","last_unstale":"2023-07-06T20:19:00.574765+0000","last_undegraded":"2023-07-06T20:19:00.574765+0000","last_fullsized":"2023-07-06T20:19:00.574765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:31:43.014901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575773+0000","last_change":"2023-07-06T20:17:28.572547+0000","last_active":"2023-07-06T20:19:00.575773+0000","last_peered":"2023-07-06T20:19:00.575773+0000","last_clean":"2023-07-06T20:19:00.575773+0000","last_became_active":"2023-07-06T20:17:28.572150+0000","last_became_peered":"2023-07-06T20:17:28.572150+0000","last_unstale":"2023-07-06T20:19:00.575773+0000","last_undegraded":"2023-07-06T20:19:00.575773+0000","last_fullsized":"2023-07-06T20:19:00.575773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:16.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575509+0000","last_change":"2023-07-06T20:17:28.574926+0000","last_active":"2023-07-06T20:19:00.575509+0000","last_peered":"2023-07-06T20:19:00.575509+0000","last_clean":"2023-07-06T20:19:00.575509+0000","last_became_active":"2023-07-06T20:17:28.574602+0000","last_became_peered":"2023-07-06T20:17:28.574602+0000","last_unstale":"2023-07-06T20:19:00.575509+0000","last_undegraded":"2023-07-06T20:19:00.575509+0000","last_fullsized":"2023-07-06T20:19:00.575509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:37:21.626981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.651733+0000","last_change":"2023-07-06T20:17:28.571533+0000","last_active":"2023-07-06T20:18:59.651733+0000","last_peered":"2023-07-06T20:18:59.651733+0000","last_clean":"2023-07-06T20:18:59.651733+0000","last_became_active":"2023-07-06T20:17:28.571099+0000","last_became_peered":"2023-07-06T20:17:28.571099+0000","last_unstale":"2023-07-06T20:18:59.651733+0000","last_undegraded":"2023-07-06T20:18:59.651733+0000","last_fullsized":"2023-07-06T20:18:59.651733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:09:44.688143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646150+0000","last_change":"2023-07-06T20:17:28.572100+0000","last_active":"2023-07-06T20:18:59.646150+0000","last_peered":"2023-07-06T20:18:59.646150+0000","last_clean":"2023-07-06T20:18:59.646150+0000","last_became_active":"2023-07-06T20:17:28.571772+0000","last_became_peered":"2023-07-06T20:17:28.571772+0000","last_unstale":"2023-07-06T20:18:59.646150+0000","last_undegraded":"2023-07-06T20:18:59.646150+0000","last_fullsized":"2023-07-06T20:18:59.646150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:20:48.756324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654341+0000","last_change":"2023-07-06T20:17:28.577816+0000","last_active":"2023-07-06T20:18:59.654341+0000","last_peered":"2023-07-06T20:18:59.654341+0000","last_clean":"2023-07-06T20:18:59.654341+0000","last_became_active":"2023-07-06T20:17:28.577382+0000","last_became_peered":"2023-07-06T20:17:28.577382+0000","last_unstale":"2023-07-06T20:18:59.654341+0000","last_undegraded":"2023-07-06T20:18:59.654341+0000","last_fullsized":"2023-07-06T20:18:59.654341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:45:41.682441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.653973+0000","last_change":"2023-07-06T20:17:28.570415+0000","last_active":"2023-07-06T20:18:59.653973+0000","last_peered":"2023-07-06T20:18:59.653973+0000","last_clean":"2023-07-06T20:18:59.653973+0000","last_became_active":"2023-07-06T20:17:28.570091+0000","last_became_peered":"2023-07-06T20:17:28.570091+0000","last_unstale":"2023-07-06T20:18:59.653973+0000","last_undegraded":"2023-07-06T20:18:59.653973+0000","last_fullsized":"2023-07-06T20:18:59.653973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:39:50.216113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575038+0000","last_change":"2023-07-06T20:17:28.568370+0000","last_active":"2023-07-06T20:19:00.575038+0000","last_peered":"2023-07-06T20:19:00.575038+0000","last_clean":"2023-07-06T20:19:00.575038+0000","last_became_active":"2023-07-06T20:17:28.568032+0000","last_became_peered":"2023-07-06T20:17:28.568032+0000","last_unstale":"2023-07-06T20:19:00.575038+0000","last_undegraded":"2023-07-06T20:19:00.575038+0000","last_fullsized":"2023-07-06T20:19:00.575038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:29:40.448529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648913+0000","last_change":"2023-07-06T20:17:28.563386+0000","last_active":"2023-07-06T20:18:59.648913+0000","last_peered":"2023-07-06T20:18:59.648913+0000","last_clean":"2023-07-06T20:18:59.648913+0000","last_became_active":"2023-07-06T20:17:28.562864+0000","last_became_peered":"2023-07-06T20:17:28.562864+0000","last_unstale":"2023-07-06T20:18:59.648913+0000","last_undegraded":"2023-07-06T20:18:59.648913+0000","last_fullsized":"2023-07-06T20:18:59.648913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:17:20.864607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.655052+0000","last_change":"2023-07-06T20:17:28.578016+0000","last_active":"2023-07-06T20:18:59.655052+0000","last_peered":"2023-07-06T20:18:59.655052+0000","last_clean":"2023-07-06T20:18:59.655052+0000","last_became_active":"2023-07-06T20:17:28.577305+0000","last_became_peered":"2023-07-06T20:17:28.577305+0000","last_unstale":"2023-07-06T20:18:59.655052+0000","last_undegraded":"2023-07-06T20:18:59.655052+0000","last_fullsized":"2023-07-06T20:18:59.655052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:04.298499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654451+0000","last_change":"2023-07-06T20:17:28.569683+0000","last_active":"2023-07-06T20:18:59.654451+0000","last_peered":"2023-07-06T20:18:59.654451+0000","last_clean":"2023-07-06T20:18:59.654451+0000","last_became_active":"2023-07-06T20:17:28.569070+0000","last_became_peered":"2023-07-06T20:17:28.569070+0000","last_unstale":"2023-07-06T20:18:59.654451+0000","last_undegraded":"2023-07-06T20:18:59.654451+0000","last_fullsized":"2023-07-06T20:18:59.654451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:32:02.630231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.651730+0000","last_change":"2023-07-06T20:17:28.579463+0000","last_active":"2023-07-06T20:18:59.651730+0000","last_peered":"2023-07-06T20:18:59.651730+0000","last_clean":"2023-07-06T20:18:59.651730+0000","last_became_active":"2023-07-06T20:17:28.579141+0000","last_became_peered":"2023-07-06T20:17:28.579141+0000","last_unstale":"2023-07-06T20:18:59.651730+0000","last_undegraded":"2023-07-06T20:18:59.651730+0000","last_fullsized":"2023-07-06T20:18:59.651730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:35:41.411836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646027+0000","last_change":"2023-07-06T20:17:58.864016+0000","last_active":"2023-07-06T20:18:59.646027+0000","last_peered":"2023-07-06T20:18:59.646027+0000","last_clean":"2023-07-06T20:18:59.646027+0000","last_became_active":"2023-07-06T20:17:58.863583+0000","last_became_peered":"2023-07-06T20:17:58.863583+0000","last_unstale":"2023-07-06T20:18:59.646027+0000","last_undegraded":"2023-07-06T20:18:59.646027+0000","last_fullsized":"2023-07-06T20:18:59.646027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:01:20.046705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575003+0000","last_change":"2023-07-06T20:17:28.572197+0000","last_active":"2023-07-06T20:19:00.575003+0000","last_peered":"2023-07-06T20:19:00.575003+0000","last_clean":"2023-07-06T20:19:00.575003+0000","last_became_active":"2023-07-06T20:17:28.571816+0000","last_became_peered":"2023-07-06T20:17:28.571816+0000","last_unstale":"2023-07-06T20:19:00.575003+0000","last_undegraded":"2023-07-06T20:19:00.575003+0000","last_fullsized":"2023-07-06T20:19:00.575003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:51.605941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574805+0000","last_change":"2023-07-06T20:17:58.746966+0000","last_active":"2023-07-06T20:19:00.574805+0000","last_peered":"2023-07-06T20:19:00.574805+0000","last_clean":"2023-07-06T20:19:00.574805+0000","last_became_active":"2023-07-06T20:17:58.746598+0000","last_became_peered":"2023-07-06T20:17:58.746598+0000","last_unstale":"2023-07-06T20:19:00.574805+0000","last_undegraded":"2023-07-06T20:19:00.574805+0000","last_fullsized":"2023-07-06T20:19:00.574805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:23:21.366915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648494+0000","last_change":"2023-07-06T20:17:28.573552+0000","last_active":"2023-07-06T20:18:59.648494+0000","last_peered":"2023-07-06T20:18:59.648494+0000","last_clean":"2023-07-06T20:18:59.648494+0000","last_became_active":"2023-07-06T20:17:28.573253+0000","last_became_peered":"2023-07-06T20:17:28.573253+0000","last_unstale":"2023-07-06T20:18:59.648494+0000","last_undegraded":"2023-07-06T20:18:59.648494+0000","last_fullsized":"2023-07-06T20:18:59.648494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:20:52.534070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574810+0000","last_change":"2023-07-06T20:17:28.572425+0000","last_active":"2023-07-06T20:19:00.574810+0000","last_peered":"2023-07-06T20:19:00.574810+0000","last_clean":"2023-07-06T20:19:00.574810+0000","last_became_active":"2023-07-06T20:17:28.572104+0000","last_became_peered":"2023-07-06T20:17:28.572104+0000","last_unstale":"2023-07-06T20:19:00.574810+0000","last_undegraded":"2023-07-06T20:19:00.574810+0000","last_fullsized":"2023-07-06T20:19:00.574810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:35:59.178149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574524+0000","last_change":"2023-07-06T20:17:28.573814+0000","last_active":"2023-07-06T20:19:00.574524+0000","last_peered":"2023-07-06T20:19:00.574524+0000","last_clean":"2023-07-06T20:19:00.574524+0000","last_became_active":"2023-07-06T20:17:28.573470+0000","last_became_peered":"2023-07-06T20:17:28.573470+0000","last_unstale":"2023-07-06T20:19:00.574524+0000","last_undegraded":"2023-07-06T20:19:00.574524+0000","last_fullsized":"2023-07-06T20:19:00.574524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:03:51.146800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073530+0000","last_change":"2023-07-06T20:17:58.865802+0000","last_active":"2023-07-06T20:19:00.073530+0000","last_peered":"2023-07-06T20:19:00.073530+0000","last_clean":"2023-07-06T20:19:00.073530+0000","last_became_active":"2023-07-06T20:17:58.865290+0000","last_became_peered":"2023-07-06T20:17:58.865290+0000","last_unstale":"2023-07-06T20:19:00.073530+0000","last_undegraded":"2023-07-06T20:19:00.073530+0000","last_fullsized":"2023-07-06T20:19:00.073530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:07:36.214950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574917+0000","last_change":"2023-07-06T20:17:27.603771+0000","last_active":"2023-07-06T20:19:00.574917+0000","last_peered":"2023-07-06T20:19:00.574917+0000","last_clean":"2023-07-06T20:19:00.574917+0000","last_became_active":"2023-07-06T20:17:27.603404+0000","last_became_peered":"2023-07-06T20:17:27.603404+0000","last_unstale":"2023-07-06T20:19:00.574917+0000","last_undegraded":"2023-07-06T20:19:00.574917+0000","last_fullsized":"2023-07-06T20:19:00.574917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:51:42.385000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575838+0000","last_change":"2023-07-06T20:17:28.568371+0000","last_active":"2023-07-06T20:19:00.575838+0000","last_peered":"2023-07-06T20:19:00.575838+0000","last_clean":"2023-07-06T20:19:00.575838+0000","last_became_active":"2023-07-06T20:17:28.568034+0000","last_became_peered":"2023-07-06T20:17:28.568034+0000","last_unstale":"2023-07-06T20:19:00.575838+0000","last_undegraded":"2023-07-06T20:19:00.575838+0000","last_fullsized":"2023-07-06T20:19:00.575838+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:20:30.073418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575967+0000","last_change":"2023-07-06T20:17:27.609833+0000","last_active":"2023-07-06T20:19:00.575967+0000","last_peered":"2023-07-06T20:19:00.575967+0000","last_clean":"2023-07-06T20:19:00.575967+0000","last_became_active":"2023-07-06T20:17:27.609520+0000","last_became_peered":"2023-07-06T20:17:27.609520+0000","last_unstale":"2023-07-06T20:19:00.575967+0000","last_undegraded":"2023-07-06T20:19:00.575967+0000","last_fullsized":"2023-07-06T20:19:00.575967+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:20:22.478396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575733+0000","last_change":"2023-07-06T20:17:28.574520+0000","last_active":"2023-07-06T20:19:00.575733+0000","last_peered":"2023-07-06T20:19:00.575733+0000","last_clean":"2023-07-06T20:19:00.575733+0000","last_became_active":"2023-07-06T20:17:28.574158+0000","last_became_peered":"2023-07-06T20:17:28.574158+0000","last_unstale":"2023-07-06T20:19:00.575733+0000","last_undegraded":"2023-07-06T20:19:00.575733+0000","last_fullsized":"2023-07-06T20:19:00.575733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:18:06.841394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646514+0000","last_change":"2023-07-06T20:17:27.601612+0000","last_active":"2023-07-06T20:18:59.646514+0000","last_peered":"2023-07-06T20:18:59.646514+0000","last_clean":"2023-07-06T20:18:59.646514+0000","last_became_active":"2023-07-06T20:17:27.601313+0000","last_became_peered":"2023-07-06T20:17:27.601313+0000","last_unstale":"2023-07-06T20:18:59.646514+0000","last_undegraded":"2023-07-06T20:18:59.646514+0000","last_fullsized":"2023-07-06T20:18:59.646514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:42:58.053499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648800+0000","last_change":"2023-07-06T20:17:28.566389+0000","last_active":"2023-07-06T20:18:59.648800+0000","last_peered":"2023-07-06T20:18:59.648800+0000","last_clean":"2023-07-06T20:18:59.648800+0000","last_became_active":"2023-07-06T20:17:28.566094+0000","last_became_peered":"2023-07-06T20:17:28.566094+0000","last_unstale":"2023-07-06T20:18:59.648800+0000","last_undegraded":"2023-07-06T20:18:59.648800+0000","last_fullsized":"2023-07-06T20:18:59.648800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:43:18.213143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646434+0000","last_change":"2023-07-06T20:17:27.599893+0000","last_active":"2023-07-06T20:18:59.646434+0000","last_peered":"2023-07-06T20:18:59.646434+0000","last_clean":"2023-07-06T20:18:59.646434+0000","last_became_active":"2023-07-06T20:17:27.599544+0000","last_became_peered":"2023-07-06T20:17:27.599544+0000","last_unstale":"2023-07-06T20:18:59.646434+0000","last_undegraded":"2023-07-06T20:18:59.646434+0000","last_fullsized":"2023-07-06T20:18:59.646434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:08:58.904898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703267+0000","last_change":"2023-07-06T20:17:28.575794+0000","last_active":"2023-07-06T20:18:59.703267+0000","last_peered":"2023-07-06T20:18:59.703267+0000","last_clean":"2023-07-06T20:18:59.703267+0000","last_became_active":"2023-07-06T20:17:28.575456+0000","last_became_peered":"2023-07-06T20:17:28.575456+0000","last_unstale":"2023-07-06T20:18:59.703267+0000","last_undegraded":"2023-07-06T20:18:59.703267+0000","last_fullsized":"2023-07-06T20:18:59.703267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:47:11.340543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":128,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:05.714486+0000","last_change":"2023-07-06T20:17:27.596351+0000","last_active":"2023-07-06T20:19:05.714486+0000","last_peered":"2023-07-06T20:19:05.714486+0000","last_clean":"2023-07-06T20:19:05.714486+0000","last_became_active":"2023-07-06T20:17:27.595982+0000","last_became_peered":"2023-07-06T20:17:27.595982+0000","last_unstale":"2023-07-06T20:19:05.714486+0000","last_undegraded":"2023-07-06T20:19:05.714486+0000","last_fullsized":"2023-07-06T20:19:05.714486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:33:14.017909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646053+0000","last_change":"2023-07-06T20:17:28.570018+0000","last_active":"2023-07-06T20:18:59.646053+0000","last_peered":"2023-07-06T20:18:59.646053+0000","last_clean":"2023-07-06T20:18:59.646053+0000","last_became_active":"2023-07-06T20:17:28.569674+0000","last_became_peered":"2023-07-06T20:17:28.569674+0000","last_unstale":"2023-07-06T20:18:59.646053+0000","last_undegraded":"2023-07-06T20:18:59.646053+0000","last_fullsized":"2023-07-06T20:18:59.646053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:24:03.335858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654185+0000","last_change":"2023-07-06T20:17:58.866940+0000","last_active":"2023-07-06T20:18:59.654185+0000","last_peered":"2023-07-06T20:18:59.654185+0000","last_clean":"2023-07-06T20:18:59.654185+0000","last_became_active":"2023-07-06T20:17:58.863435+0000","last_became_peered":"2023-07-06T20:17:58.863435+0000","last_unstale":"2023-07-06T20:18:59.654185+0000","last_undegraded":"2023-07-06T20:18:59.654185+0000","last_fullsized":"2023-07-06T20:18:59.654185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:34:45.362921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703189+0000","last_change":"2023-07-06T20:17:28.575249+0000","last_active":"2023-07-06T20:18:59.703189+0000","last_peered":"2023-07-06T20:18:59.703189+0000","last_clean":"2023-07-06T20:18:59.703189+0000","last_became_active":"2023-07-06T20:17:28.574954+0000","last_became_peered":"2023-07-06T20:17:28.574954+0000","last_unstale":"2023-07-06T20:18:59.703189+0000","last_undegraded":"2023-07-06T20:18:59.703189+0000","last_fullsized":"2023-07-06T20:18:59.703189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:10:03.983955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645825+0000","last_change":"2023-07-06T20:17:27.600552+0000","last_active":"2023-07-06T20:18:59.645825+0000","last_peered":"2023-07-06T20:18:59.645825+0000","last_clean":"2023-07-06T20:18:59.645825+0000","last_became_active":"2023-07-06T20:17:27.600016+0000","last_became_peered":"2023-07-06T20:17:27.600016+0000","last_unstale":"2023-07-06T20:18:59.645825+0000","last_undegraded":"2023-07-06T20:18:59.645825+0000","last_fullsized":"2023-07-06T20:18:59.645825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:08:51.272104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575191+0000","last_change":"2023-07-06T20:17:28.566364+0000","last_active":"2023-07-06T20:19:00.575191+0000","last_peered":"2023-07-06T20:19:00.575191+0000","last_clean":"2023-07-06T20:19:00.575191+0000","last_became_active":"2023-07-06T20:17:28.565474+0000","last_became_peered":"2023-07-06T20:17:28.565474+0000","last_unstale":"2023-07-06T20:19:00.575191+0000","last_undegraded":"2023-07-06T20:19:00.575191+0000","last_fullsized":"2023-07-06T20:19:00.575191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:28:42.265878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.702829+0000","last_change":"2023-07-06T20:17:27.598606+0000","last_active":"2023-07-06T20:18:59.702829+0000","last_peered":"2023-07-06T20:18:59.702829+0000","last_clean":"2023-07-06T20:18:59.702829+0000","last_became_active":"2023-07-06T20:17:27.598292+0000","last_became_peered":"2023-07-06T20:17:27.598292+0000","last_unstale":"2023-07-06T20:18:59.702829+0000","last_undegraded":"2023-07-06T20:18:59.702829+0000","last_fullsized":"2023-07-06T20:18:59.702829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:11:39.786934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574951+0000","last_change":"2023-07-06T20:17:28.571246+0000","last_active":"2023-07-06T20:19:00.574951+0000","last_peered":"2023-07-06T20:19:00.574951+0000","last_clean":"2023-07-06T20:19:00.574951+0000","last_became_active":"2023-07-06T20:17:28.570846+0000","last_became_peered":"2023-07-06T20:17:28.570846+0000","last_unstale":"2023-07-06T20:19:00.574951+0000","last_undegraded":"2023-07-06T20:19:00.574951+0000","last_fullsized":"2023-07-06T20:19:00.574951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:31:31.244148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:03.710217+0000","last_change":"2023-07-06T20:17:27.595879+0000","last_active":"2023-07-06T20:19:03.710217+0000","last_peered":"2023-07-06T20:19:03.710217+0000","last_clean":"2023-07-06T20:19:03.710217+0000","last_became_active":"2023-07-06T20:17:27.595553+0000","last_became_peered":"2023-07-06T20:17:27.595553+0000","last_unstale":"2023-07-06T20:19:03.710217+0000","last_undegraded":"2023-07-06T20:19:03.710217+0000","last_fullsized":"2023-07-06T20:19:03.710217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:01:12.807960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648843+0000","last_change":"2023-07-06T20:17:28.566816+0000","last_active":"2023-07-06T20:18:59.648843+0000","last_peered":"2023-07-06T20:18:59.648843+0000","last_clean":"2023-07-06T20:18:59.648843+0000","last_became_active":"2023-07-06T20:17:28.566565+0000","last_became_peered":"2023-07-06T20:17:28.566565+0000","last_unstale":"2023-07-06T20:18:59.648843+0000","last_undegraded":"2023-07-06T20:18:59.648843+0000","last_fullsized":"2023-07-06T20:18:59.648843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:37:37.504246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.653939+0000","last_change":"2023-07-06T20:17:58.868853+0000","last_active":"2023-07-06T20:18:59.653939+0000","last_peered":"2023-07-06T20:18:59.653939+0000","last_clean":"2023-07-06T20:18:59.653939+0000","last_became_active":"2023-07-06T20:17:58.863304+0000","last_became_peered":"2023-07-06T20:17:58.863304+0000","last_unstale":"2023-07-06T20:18:59.653939+0000","last_undegraded":"2023-07-06T20:18:59.653939+0000","last_fullsized":"2023-07-06T20:18:59.653939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:33:48.601790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575721+0000","last_change":"2023-07-06T20:17:28.572400+0000","last_active":"2023-07-06T20:19:00.575721+0000","last_peered":"2023-07-06T20:19:00.575721+0000","last_clean":"2023-07-06T20:19:00.575721+0000","last_became_active":"2023-07-06T20:17:28.572047+0000","last_became_peered":"2023-07-06T20:17:28.572047+0000","last_unstale":"2023-07-06T20:19:00.575721+0000","last_undegraded":"2023-07-06T20:19:00.575721+0000","last_fullsized":"2023-07-06T20:19:00.575721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:17:30.668701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703472+0000","last_change":"2023-07-06T20:17:27.603224+0000","last_active":"2023-07-06T20:18:59.703472+0000","last_peered":"2023-07-06T20:18:59.703472+0000","last_clean":"2023-07-06T20:18:59.703472+0000","last_became_active":"2023-07-06T20:17:27.602864+0000","last_became_peered":"2023-07-06T20:17:27.602864+0000","last_unstale":"2023-07-06T20:18:59.703472+0000","last_undegraded":"2023-07-06T20:18:59.703472+0000","last_fullsized":"2023-07-06T20:18:59.703472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:23:10.000206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648876+0000","last_change":"2023-07-06T20:17:28.572683+0000","last_active":"2023-07-06T20:18:59.648876+0000","last_peered":"2023-07-06T20:18:59.648876+0000","last_clean":"2023-07-06T20:18:59.648876+0000","last_became_active":"2023-07-06T20:17:28.572352+0000","last_became_peered":"2023-07-06T20:17:28.572352+0000","last_unstale":"2023-07-06T20:18:59.648876+0000","last_undegraded":"2023-07-06T20:18:59.648876+0000","last_fullsized":"2023-07-06T20:18:59.648876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:15:00.435753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'30","reported_seq":247,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:04.711908+0000","last_change":"2023-07-06T20:17:27.597285+0000","last_active":"2023-07-06T20:19:04.711908+0000","last_peered":"2023-07-06T20:19:04.711908+0000","last_clean":"2023-07-06T20:19:04.711908+0000","last_became_active":"2023-07-06T20:17:27.596850+0000","last_became_peered":"2023-07-06T20:17:27.596850+0000","last_unstale":"2023-07-06T20:19:04.711908+0000","last_undegraded":"2023-07-06T20:19:04.711908+0000","last_fullsized":"2023-07-06T20:19:04.711908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:22:39.869484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58776,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":195,"num_read_kb":154,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646322+0000","last_change":"2023-07-06T20:17:28.569855+0000","last_active":"2023-07-06T20:18:59.646322+0000","last_peered":"2023-07-06T20:18:59.646322+0000","last_clean":"2023-07-06T20:18:59.646322+0000","last_became_active":"2023-07-06T20:17:28.569495+0000","last_became_peered":"2023-07-06T20:17:28.569495+0000","last_unstale":"2023-07-06T20:18:59.646322+0000","last_undegraded":"2023-07-06T20:18:59.646322+0000","last_fullsized":"2023-07-06T20:18:59.646322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:59:00.755254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073397+0000","last_change":"2023-07-06T20:17:27.605898+0000","last_active":"2023-07-06T20:19:00.073397+0000","last_peered":"2023-07-06T20:19:00.073397+0000","last_clean":"2023-07-06T20:19:00.073397+0000","last_became_active":"2023-07-06T20:17:27.605605+0000","last_became_peered":"2023-07-06T20:17:27.605605+0000","last_unstale":"2023-07-06T20:19:00.073397+0000","last_undegraded":"2023-07-06T20:19:00.073397+0000","last_fullsized":"2023-07-06T20:19:00.073397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:31:31.780838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645727+0000","last_change":"2023-07-06T20:17:28.579236+0000","last_active":"2023-07-06T20:18:59.645727+0000","last_peered":"2023-07-06T20:18:59.645727+0000","last_clean":"2023-07-06T20:18:59.645727+0000","last_became_active":"2023-07-06T20:17:28.578854+0000","last_became_peered":"2023-07-06T20:17:28.578854+0000","last_unstale":"2023-07-06T20:18:59.645727+0000","last_undegraded":"2023-07-06T20:18:59.645727+0000","last_fullsized":"2023-07-06T20:18:59.645727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:20:05.459582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703207+0000","last_change":"2023-07-06T20:17:27.595724+0000","last_active":"2023-07-06T20:18:59.703207+0000","last_peered":"2023-07-06T20:18:59.703207+0000","last_clean":"2023-07-06T20:18:59.703207+0000","last_became_active":"2023-07-06T20:17:27.595360+0000","last_became_peered":"2023-07-06T20:17:27.595360+0000","last_unstale":"2023-07-06T20:18:59.703207+0000","last_undegraded":"2023-07-06T20:18:59.703207+0000","last_fullsized":"2023-07-06T20:18:59.703207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:46:07.776434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654912+0000","last_change":"2023-07-06T20:17:28.573447+0000","last_active":"2023-07-06T20:18:59.654912+0000","last_peered":"2023-07-06T20:18:59.654912+0000","last_clean":"2023-07-06T20:18:59.654912+0000","last_became_active":"2023-07-06T20:17:28.573161+0000","last_became_peered":"2023-07-06T20:17:28.573161+0000","last_unstale":"2023-07-06T20:18:59.654912+0000","last_undegraded":"2023-07-06T20:18:59.654912+0000","last_fullsized":"2023-07-06T20:18:59.654912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:04:10.288520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":137,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:05.587015+0000","last_change":"2023-07-06T20:19:00.579098+0000","last_active":"2023-07-06T20:19:05.587015+0000","last_peered":"2023-07-06T20:19:05.587015+0000","last_clean":"2023-07-06T20:19:05.587015+0000","last_became_active":"2023-07-06T20:19:00.575486+0000","last_became_peered":"2023-07-06T20:19:00.575486+0000","last_unstale":"2023-07-06T20:19:05.587015+0000","last_undegraded":"2023-07-06T20:19:05.587015+0000","last_fullsized":"2023-07-06T20:19:05.587015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:36:44.887142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1071,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.653581+0000","last_change":"2023-07-06T20:17:28.569745+0000","last_active":"2023-07-06T20:18:59.653581+0000","last_peered":"2023-07-06T20:18:59.653581+0000","last_clean":"2023-07-06T20:18:59.653581+0000","last_became_active":"2023-07-06T20:17:28.569468+0000","last_became_peered":"2023-07-06T20:17:28.569468+0000","last_unstale":"2023-07-06T20:18:59.653581+0000","last_undegraded":"2023-07-06T20:18:59.653581+0000","last_fullsized":"2023-07-06T20:18:59.653581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:04:39.612959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574256+0000","last_change":"2023-07-06T20:17:58.771936+0000","last_active":"2023-07-06T20:19:00.574256+0000","last_peered":"2023-07-06T20:19:00.574256+0000","last_clean":"2023-07-06T20:19:00.574256+0000","last_became_active":"2023-07-06T20:17:58.748196+0000","last_became_peered":"2023-07-06T20:17:58.748196+0000","last_unstale":"2023-07-06T20:19:00.574256+0000","last_undegraded":"2023-07-06T20:19:00.574256+0000","last_fullsized":"2023-07-06T20:19:00.574256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:20:15.168480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703012+0000","last_change":"2023-07-06T20:17:28.575390+0000","last_active":"2023-07-06T20:18:59.703012+0000","last_peered":"2023-07-06T20:18:59.703012+0000","last_clean":"2023-07-06T20:18:59.703012+0000","last_became_active":"2023-07-06T20:17:28.575083+0000","last_became_peered":"2023-07-06T20:17:28.575083+0000","last_unstale":"2023-07-06T20:18:59.703012+0000","last_undegraded":"2023-07-06T20:18:59.703012+0000","last_fullsized":"2023-07-06T20:18:59.703012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:05.579725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646574+0000","last_change":"2023-07-06T20:17:27.600891+0000","last_active":"2023-07-06T20:18:59.646574+0000","last_peered":"2023-07-06T20:18:59.646574+0000","last_clean":"2023-07-06T20:18:59.646574+0000","last_became_active":"2023-07-06T20:17:27.600533+0000","last_became_peered":"2023-07-06T20:17:27.600533+0000","last_unstale":"2023-07-06T20:18:59.646574+0000","last_undegraded":"2023-07-06T20:18:59.646574+0000","last_fullsized":"2023-07-06T20:18:59.646574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:44:00.501767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574645+0000","last_change":"2023-07-06T20:17:58.863073+0000","last_active":"2023-07-06T20:19:00.574645+0000","last_peered":"2023-07-06T20:19:00.574645+0000","last_clean":"2023-07-06T20:19:00.574645+0000","last_became_active":"2023-07-06T20:17:58.862613+0000","last_became_peered":"2023-07-06T20:17:58.862613+0000","last_unstale":"2023-07-06T20:19:00.574645+0000","last_undegraded":"2023-07-06T20:19:00.574645+0000","last_fullsized":"2023-07-06T20:19:00.574645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:21:52.224293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574359+0000","last_change":"2023-07-06T20:17:27.604464+0000","last_active":"2023-07-06T20:19:00.574359+0000","last_peered":"2023-07-06T20:19:00.574359+0000","last_clean":"2023-07-06T20:19:00.574359+0000","last_became_active":"2023-07-06T20:17:27.604036+0000","last_became_peered":"2023-07-06T20:17:27.604036+0000","last_unstale":"2023-07-06T20:19:00.574359+0000","last_undegraded":"2023-07-06T20:19:00.574359+0000","last_fullsized":"2023-07-06T20:19:00.574359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:59:41.175216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654818+0000","last_change":"2023-07-06T20:17:28.577302+0000","last_active":"2023-07-06T20:18:59.654818+0000","last_peered":"2023-07-06T20:18:59.654818+0000","last_clean":"2023-07-06T20:18:59.654818+0000","last_became_active":"2023-07-06T20:17:28.576998+0000","last_became_peered":"2023-07-06T20:17:28.576998+0000","last_unstale":"2023-07-06T20:18:59.654818+0000","last_undegraded":"2023-07-06T20:18:59.654818+0000","last_fullsized":"2023-07-06T20:18:59.654818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:30:35.224494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654890+0000","last_change":"2023-07-06T20:17:27.608391+0000","last_active":"2023-07-06T20:18:59.654890+0000","last_peered":"2023-07-06T20:18:59.654890+0000","last_clean":"2023-07-06T20:18:59.654890+0000","last_became_active":"2023-07-06T20:17:27.608073+0000","last_became_peered":"2023-07-06T20:17:27.608073+0000","last_unstale":"2023-07-06T20:18:59.654890+0000","last_undegraded":"2023-07-06T20:18:59.654890+0000","last_fullsized":"2023-07-06T20:18:59.654890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:58:17.469333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654516+0000","last_change":"2023-07-06T20:17:28.577554+0000","last_active":"2023-07-06T20:18:59.654516+0000","last_peered":"2023-07-06T20:18:59.654516+0000","last_clean":"2023-07-06T20:18:59.654516+0000","last_became_active":"2023-07-06T20:17:28.577208+0000","last_became_peered":"2023-07-06T20:17:28.577208+0000","last_unstale":"2023-07-06T20:18:59.654516+0000","last_undegraded":"2023-07-06T20:18:59.654516+0000","last_fullsized":"2023-07-06T20:18:59.654516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:11:56.614831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703008+0000","last_change":"2023-07-06T20:17:28.571678+0000","last_active":"2023-07-06T20:18:59.703008+0000","last_peered":"2023-07-06T20:18:59.703008+0000","last_clean":"2023-07-06T20:18:59.703008+0000","last_became_active":"2023-07-06T20:17:28.570974+0000","last_became_peered":"2023-07-06T20:17:28.570974+0000","last_unstale":"2023-07-06T20:18:59.703008+0000","last_undegraded":"2023-07-06T20:18:59.703008+0000","last_fullsized":"2023-07-06T20:18:59.703008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:53:32.536581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":121,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:07.589485+0000","last_change":"2023-07-06T20:17:27.607172+0000","last_active":"2023-07-06T20:19:07.589485+0000","last_peered":"2023-07-06T20:19:07.589485+0000","last_clean":"2023-07-06T20:19:07.589485+0000","last_became_active":"2023-07-06T20:17:27.606841+0000","last_became_peered":"2023-07-06T20:17:27.606841+0000","last_unstale":"2023-07-06T20:19:07.589485+0000","last_undegraded":"2023-07-06T20:19:07.589485+0000","last_fullsized":"2023-07-06T20:19:07.589485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:48:15.870982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574630+0000","last_change":"2023-07-06T20:17:28.567707+0000","last_active":"2023-07-06T20:19:00.574630+0000","last_peered":"2023-07-06T20:19:00.574630+0000","last_clean":"2023-07-06T20:19:00.574630+0000","last_became_active":"2023-07-06T20:17:28.567394+0000","last_became_peered":"2023-07-06T20:17:28.567394+0000","last_unstale":"2023-07-06T20:19:00.574630+0000","last_undegraded":"2023-07-06T20:19:00.574630+0000","last_fullsized":"2023-07-06T20:19:00.574630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:58:40.539787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.651620+0000","last_change":"2023-07-06T20:17:27.595746+0000","last_active":"2023-07-06T20:18:59.651620+0000","last_peered":"2023-07-06T20:18:59.651620+0000","last_clean":"2023-07-06T20:18:59.651620+0000","last_became_active":"2023-07-06T20:17:27.595039+0000","last_became_peered":"2023-07-06T20:17:27.595039+0000","last_unstale":"2023-07-06T20:18:59.651620+0000","last_undegraded":"2023-07-06T20:18:59.651620+0000","last_fullsized":"2023-07-06T20:18:59.651620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:00:27.347076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654306+0000","last_change":"2023-07-06T20:16:23.542324+0000","last_active":"2023-07-06T20:18:59.654306+0000","last_peered":"2023-07-06T20:18:59.654306+0000","last_clean":"2023-07-06T20:18:59.654306+0000","last_became_active":"2023-07-06T20:16:23.541832+0000","last_became_peered":"2023-07-06T20:16:23.541832+0000","last_unstale":"2023-07-06T20:18:59.654306+0000","last_undegraded":"2023-07-06T20:18:59.654306+0000","last_fullsized":"2023-07-06T20:18:59.654306+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:13:14.072924+0000","last_clean_scrub_stamp":"2023-07-06T20:13:14.072924+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:08:55.801420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645653+0000","last_change":"2023-07-06T20:17:28.577309+0000","last_active":"2023-07-06T20:18:59.645653+0000","last_peered":"2023-07-06T20:18:59.645653+0000","last_clean":"2023-07-06T20:18:59.645653+0000","last_became_active":"2023-07-06T20:17:28.576962+0000","last_became_peered":"2023-07-06T20:17:28.576962+0000","last_unstale":"2023-07-06T20:18:59.645653+0000","last_undegraded":"2023-07-06T20:18:59.645653+0000","last_fullsized":"2023-07-06T20:18:59.645653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:53.192162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654524+0000","last_change":"2023-07-06T20:17:27.601487+0000","last_active":"2023-07-06T20:18:59.654524+0000","last_peered":"2023-07-06T20:18:59.654524+0000","last_clean":"2023-07-06T20:18:59.654524+0000","last_became_active":"2023-07-06T20:17:27.601195+0000","last_became_peered":"2023-07-06T20:17:27.601195+0000","last_unstale":"2023-07-06T20:18:59.654524+0000","last_undegraded":"2023-07-06T20:18:59.654524+0000","last_fullsized":"2023-07-06T20:18:59.654524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:01:12.039199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646358+0000","last_change":"2023-07-06T20:17:27.610374+0000","last_active":"2023-07-06T20:18:59.646358+0000","last_peered":"2023-07-06T20:18:59.646358+0000","last_clean":"2023-07-06T20:18:59.646358+0000","last_became_active":"2023-07-06T20:17:27.610104+0000","last_became_peered":"2023-07-06T20:17:27.610104+0000","last_unstale":"2023-07-06T20:18:59.646358+0000","last_undegraded":"2023-07-06T20:18:59.646358+0000","last_fullsized":"2023-07-06T20:18:59.646358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:51:41.440974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645823+0000","last_change":"2023-07-06T20:17:28.570146+0000","last_active":"2023-07-06T20:18:59.645823+0000","last_peered":"2023-07-06T20:18:59.645823+0000","last_clean":"2023-07-06T20:18:59.645823+0000","last_became_active":"2023-07-06T20:17:28.569341+0000","last_became_peered":"2023-07-06T20:17:28.569341+0000","last_unstale":"2023-07-06T20:18:59.645823+0000","last_undegraded":"2023-07-06T20:18:59.645823+0000","last_fullsized":"2023-07-06T20:18:59.645823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:25:56.302477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:02.380205+0000","last_change":"2023-07-06T20:17:27.604630+0000","last_active":"2023-07-06T20:19:02.380205+0000","last_peered":"2023-07-06T20:19:02.380205+0000","last_clean":"2023-07-06T20:19:02.380205+0000","last_became_active":"2023-07-06T20:17:27.604118+0000","last_became_peered":"2023-07-06T20:17:27.604118+0000","last_unstale":"2023-07-06T20:19:02.380205+0000","last_undegraded":"2023-07-06T20:19:02.380205+0000","last_fullsized":"2023-07-06T20:19:02.380205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:04:38.195720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575728+0000","last_change":"2023-07-06T20:17:28.571753+0000","last_active":"2023-07-06T20:19:00.575728+0000","last_peered":"2023-07-06T20:19:00.575728+0000","last_clean":"2023-07-06T20:19:00.575728+0000","last_became_active":"2023-07-06T20:17:28.571073+0000","last_became_peered":"2023-07-06T20:17:28.571073+0000","last_unstale":"2023-07-06T20:19:00.575728+0000","last_undegraded":"2023-07-06T20:19:00.575728+0000","last_fullsized":"2023-07-06T20:19:00.575728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:49:51.350250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.651893+0000","last_change":"2023-07-06T20:17:27.594354+0000","last_active":"2023-07-06T20:18:59.651893+0000","last_peered":"2023-07-06T20:18:59.651893+0000","last_clean":"2023-07-06T20:18:59.651893+0000","last_became_active":"2023-07-06T20:17:27.594042+0000","last_became_peered":"2023-07-06T20:17:27.594042+0000","last_unstale":"2023-07-06T20:18:59.651893+0000","last_undegraded":"2023-07-06T20:18:59.651893+0000","last_fullsized":"2023-07-06T20:18:59.651893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:33:21.518458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575621+0000","last_change":"2023-07-06T20:17:28.576170+0000","last_active":"2023-07-06T20:19:00.575621+0000","last_peered":"2023-07-06T20:19:00.575621+0000","last_clean":"2023-07-06T20:19:00.575621+0000","last_became_active":"2023-07-06T20:17:28.575493+0000","last_became_peered":"2023-07-06T20:17:28.575493+0000","last_unstale":"2023-07-06T20:19:00.575621+0000","last_undegraded":"2023-07-06T20:19:00.575621+0000","last_fullsized":"2023-07-06T20:19:00.575621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:12:05.377174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648489+0000","last_change":"2023-07-06T20:17:27.602984+0000","last_active":"2023-07-06T20:18:59.648489+0000","last_peered":"2023-07-06T20:18:59.648489+0000","last_clean":"2023-07-06T20:18:59.648489+0000","last_became_active":"2023-07-06T20:17:27.602720+0000","last_became_peered":"2023-07-06T20:17:27.602720+0000","last_unstale":"2023-07-06T20:18:59.648489+0000","last_undegraded":"2023-07-06T20:18:59.648489+0000","last_fullsized":"2023-07-06T20:18:59.648489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:59:43.619260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646022+0000","last_change":"2023-07-06T20:17:28.565954+0000","last_active":"2023-07-06T20:18:59.646022+0000","last_peered":"2023-07-06T20:18:59.646022+0000","last_clean":"2023-07-06T20:18:59.646022+0000","last_became_active":"2023-07-06T20:17:28.565300+0000","last_became_peered":"2023-07-06T20:17:28.565300+0000","last_unstale":"2023-07-06T20:18:59.646022+0000","last_undegraded":"2023-07-06T20:18:59.646022+0000","last_fullsized":"2023-07-06T20:18:59.646022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:57:19.951265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575343+0000","last_change":"2023-07-06T20:17:27.597859+0000","last_active":"2023-07-06T20:19:00.575343+0000","last_peered":"2023-07-06T20:19:00.575343+0000","last_clean":"2023-07-06T20:19:00.575343+0000","last_became_active":"2023-07-06T20:17:27.597479+0000","last_became_peered":"2023-07-06T20:17:27.597479+0000","last_unstale":"2023-07-06T20:19:00.575343+0000","last_undegraded":"2023-07-06T20:19:00.575343+0000","last_fullsized":"2023-07-06T20:19:00.575343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:44:33.032535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574266+0000","last_change":"2023-07-06T20:17:28.573069+0000","last_active":"2023-07-06T20:19:00.574266+0000","last_peered":"2023-07-06T20:19:00.574266+0000","last_clean":"2023-07-06T20:19:00.574266+0000","last_became_active":"2023-07-06T20:17:28.572742+0000","last_became_peered":"2023-07-06T20:17:28.572742+0000","last_unstale":"2023-07-06T20:19:00.574266+0000","last_undegraded":"2023-07-06T20:19:00.574266+0000","last_fullsized":"2023-07-06T20:19:00.574266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:00:57.440359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073240+0000","last_change":"2023-07-06T20:17:27.599259+0000","last_active":"2023-07-06T20:19:00.073240+0000","last_peered":"2023-07-06T20:19:00.073240+0000","last_clean":"2023-07-06T20:19:00.073240+0000","last_became_active":"2023-07-06T20:17:27.598883+0000","last_became_peered":"2023-07-06T20:17:27.598883+0000","last_unstale":"2023-07-06T20:19:00.073240+0000","last_undegraded":"2023-07-06T20:19:00.073240+0000","last_fullsized":"2023-07-06T20:19:00.073240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:06:44.671018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575340+0000","last_change":"2023-07-06T20:17:28.573874+0000","last_active":"2023-07-06T20:19:00.575340+0000","last_peered":"2023-07-06T20:19:00.575340+0000","last_clean":"2023-07-06T20:19:00.575340+0000","last_became_active":"2023-07-06T20:17:28.573379+0000","last_became_peered":"2023-07-06T20:17:28.573379+0000","last_unstale":"2023-07-06T20:19:00.575340+0000","last_undegraded":"2023-07-06T20:19:00.575340+0000","last_fullsized":"2023-07-06T20:19:00.575340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T20:26:16.579909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.646204+0000","last_change":"2023-07-06T20:17:27.601474+0000","last_active":"2023-07-06T20:18:59.646204+0000","last_peered":"2023-07-06T20:18:59.646204+0000","last_clean":"2023-07-06T20:18:59.646204+0000","last_became_active":"2023-07-06T20:17:27.601099+0000","last_became_peered":"2023-07-06T20:17:27.601099+0000","last_unstale":"2023-07-06T20:18:59.646204+0000","last_undegraded":"2023-07-06T20:18:59.646204+0000","last_fullsized":"2023-07-06T20:18:59.646204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:36:33.224709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654754+0000","last_change":"2023-07-06T20:17:28.571170+0000","last_active":"2023-07-06T20:18:59.654754+0000","last_peered":"2023-07-06T20:18:59.654754+0000","last_clean":"2023-07-06T20:18:59.654754+0000","last_became_active":"2023-07-06T20:17:28.570874+0000","last_became_peered":"2023-07-06T20:17:28.570874+0000","last_unstale":"2023-07-06T20:18:59.654754+0000","last_undegraded":"2023-07-06T20:18:59.654754+0000","last_fullsized":"2023-07-06T20:18:59.654754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:05.977773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073444+0000","last_change":"2023-07-06T20:17:27.603604+0000","last_active":"2023-07-06T20:19:00.073444+0000","last_peered":"2023-07-06T20:19:00.073444+0000","last_clean":"2023-07-06T20:19:00.073444+0000","last_became_active":"2023-07-06T20:17:27.603015+0000","last_became_peered":"2023-07-06T20:17:27.603015+0000","last_unstale":"2023-07-06T20:19:00.073444+0000","last_undegraded":"2023-07-06T20:19:00.073444+0000","last_fullsized":"2023-07-06T20:19:00.073444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:26:02.215516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654631+0000","last_change":"2023-07-06T20:17:28.567906+0000","last_active":"2023-07-06T20:18:59.654631+0000","last_peered":"2023-07-06T20:18:59.654631+0000","last_clean":"2023-07-06T20:18:59.654631+0000","last_became_active":"2023-07-06T20:17:28.567509+0000","last_became_peered":"2023-07-06T20:17:28.567509+0000","last_unstale":"2023-07-06T20:18:59.654631+0000","last_undegraded":"2023-07-06T20:18:59.654631+0000","last_fullsized":"2023-07-06T20:18:59.654631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:51:09.883831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.703298+0000","last_change":"2023-07-06T20:17:27.598323+0000","last_active":"2023-07-06T20:18:59.703298+0000","last_peered":"2023-07-06T20:18:59.703298+0000","last_clean":"2023-07-06T20:18:59.703298+0000","last_became_active":"2023-07-06T20:17:27.598032+0000","last_became_peered":"2023-07-06T20:17:27.598032+0000","last_unstale":"2023-07-06T20:18:59.703298+0000","last_undegraded":"2023-07-06T20:18:59.703298+0000","last_fullsized":"2023-07-06T20:18:59.703298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:02:05.424357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654137+0000","last_change":"2023-07-06T20:17:28.571756+0000","last_active":"2023-07-06T20:18:59.654137+0000","last_peered":"2023-07-06T20:18:59.654137+0000","last_clean":"2023-07-06T20:18:59.654137+0000","last_became_active":"2023-07-06T20:17:28.571440+0000","last_became_peered":"2023-07-06T20:17:28.571440+0000","last_unstale":"2023-07-06T20:18:59.654137+0000","last_undegraded":"2023-07-06T20:18:59.654137+0000","last_fullsized":"2023-07-06T20:18:59.654137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:48.120173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645836+0000","last_change":"2023-07-06T20:17:27.602620+0000","last_active":"2023-07-06T20:18:59.645836+0000","last_peered":"2023-07-06T20:18:59.645836+0000","last_clean":"2023-07-06T20:18:59.645836+0000","last_became_active":"2023-07-06T20:17:27.602224+0000","last_became_peered":"2023-07-06T20:17:27.602224+0000","last_unstale":"2023-07-06T20:18:59.645836+0000","last_undegraded":"2023-07-06T20:18:59.645836+0000","last_fullsized":"2023-07-06T20:18:59.645836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:26.573320+0000","last_clean_scrub_stamp":"2023-07-06T20:17:26.573320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:18:17.166344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.654194+0000","last_change":"2023-07-06T20:17:28.569203+0000","last_active":"2023-07-06T20:18:59.654194+0000","last_peered":"2023-07-06T20:18:59.654194+0000","last_clean":"2023-07-06T20:18:59.654194+0000","last_became_active":"2023-07-06T20:17:28.568948+0000","last_became_peered":"2023-07-06T20:17:28.568948+0000","last_unstale":"2023-07-06T20:18:59.654194+0000","last_undegraded":"2023-07-06T20:18:59.654194+0000","last_fullsized":"2023-07-06T20:18:59.654194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:49:16.105664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574493+0000","last_change":"2023-07-06T20:17:28.568829+0000","last_active":"2023-07-06T20:19:00.574493+0000","last_peered":"2023-07-06T20:19:00.574493+0000","last_clean":"2023-07-06T20:19:00.574493+0000","last_became_active":"2023-07-06T20:17:28.568511+0000","last_became_peered":"2023-07-06T20:17:28.568511+0000","last_unstale":"2023-07-06T20:19:00.574493+0000","last_undegraded":"2023-07-06T20:19:00.574493+0000","last_fullsized":"2023-07-06T20:19:00.574493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:08.295954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073259+0000","last_change":"2023-07-06T20:17:58.863188+0000","last_active":"2023-07-06T20:19:00.073259+0000","last_peered":"2023-07-06T20:19:00.073259+0000","last_clean":"2023-07-06T20:19:00.073259+0000","last_became_active":"2023-07-06T20:17:58.862743+0000","last_became_peered":"2023-07-06T20:17:58.862743+0000","last_unstale":"2023-07-06T20:19:00.073259+0000","last_undegraded":"2023-07-06T20:19:00.073259+0000","last_fullsized":"2023-07-06T20:19:00.073259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:07:20.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073191+0000","last_change":"2023-07-06T20:17:28.572264+0000","last_active":"2023-07-06T20:19:00.073191+0000","last_peered":"2023-07-06T20:19:00.073191+0000","last_clean":"2023-07-06T20:19:00.073191+0000","last_became_active":"2023-07-06T20:17:28.571914+0000","last_became_peered":"2023-07-06T20:17:28.571914+0000","last_unstale":"2023-07-06T20:19:00.073191+0000","last_undegraded":"2023-07-06T20:19:00.073191+0000","last_fullsized":"2023-07-06T20:19:00.073191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:01:02.267552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575399+0000","last_change":"2023-07-06T20:17:28.581374+0000","last_active":"2023-07-06T20:19:00.575399+0000","last_peered":"2023-07-06T20:19:00.575399+0000","last_clean":"2023-07-06T20:19:00.575399+0000","last_became_active":"2023-07-06T20:17:28.580995+0000","last_became_peered":"2023-07-06T20:17:28.580995+0000","last_unstale":"2023-07-06T20:19:00.575399+0000","last_undegraded":"2023-07-06T20:19:00.575399+0000","last_fullsized":"2023-07-06T20:19:00.575399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:08:18.718991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645952+0000","last_change":"2023-07-06T20:17:28.578926+0000","last_active":"2023-07-06T20:18:59.645952+0000","last_peered":"2023-07-06T20:18:59.645952+0000","last_clean":"2023-07-06T20:18:59.645952+0000","last_became_active":"2023-07-06T20:17:28.578620+0000","last_became_peered":"2023-07-06T20:17:28.578620+0000","last_unstale":"2023-07-06T20:18:59.645952+0000","last_undegraded":"2023-07-06T20:18:59.645952+0000","last_fullsized":"2023-07-06T20:18:59.645952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:17:06.451679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574477+0000","last_change":"2023-07-06T20:17:58.864937+0000","last_active":"2023-07-06T20:19:00.574477+0000","last_peered":"2023-07-06T20:19:00.574477+0000","last_clean":"2023-07-06T20:19:00.574477+0000","last_became_active":"2023-07-06T20:17:58.864457+0000","last_became_peered":"2023-07-06T20:17:58.864457+0000","last_unstale":"2023-07-06T20:19:00.574477+0000","last_undegraded":"2023-07-06T20:19:00.574477+0000","last_fullsized":"2023-07-06T20:19:00.574477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:53:40.789746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.073661+0000","last_change":"2023-07-06T20:17:28.567965+0000","last_active":"2023-07-06T20:19:00.073661+0000","last_peered":"2023-07-06T20:19:00.073661+0000","last_clean":"2023-07-06T20:19:00.073661+0000","last_became_active":"2023-07-06T20:17:28.567618+0000","last_became_peered":"2023-07-06T20:17:28.567618+0000","last_unstale":"2023-07-06T20:19:00.073661+0000","last_undegraded":"2023-07-06T20:19:00.073661+0000","last_fullsized":"2023-07-06T20:19:00.073661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:20:53.386538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575649+0000","last_change":"2023-07-06T20:17:28.571934+0000","last_active":"2023-07-06T20:19:00.575649+0000","last_peered":"2023-07-06T20:19:00.575649+0000","last_clean":"2023-07-06T20:19:00.575649+0000","last_became_active":"2023-07-06T20:17:28.571472+0000","last_became_peered":"2023-07-06T20:17:28.571472+0000","last_unstale":"2023-07-06T20:19:00.575649+0000","last_undegraded":"2023-07-06T20:19:00.575649+0000","last_fullsized":"2023-07-06T20:19:00.575649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:22.858255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.702897+0000","last_change":"2023-07-06T20:17:28.571904+0000","last_active":"2023-07-06T20:18:59.702897+0000","last_peered":"2023-07-06T20:18:59.702897+0000","last_clean":"2023-07-06T20:18:59.702897+0000","last_became_active":"2023-07-06T20:17:28.571375+0000","last_became_peered":"2023-07-06T20:17:28.571375+0000","last_unstale":"2023-07-06T20:18:59.702897+0000","last_undegraded":"2023-07-06T20:18:59.702897+0000","last_fullsized":"2023-07-06T20:18:59.702897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:32:15.205545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.575210+0000","last_change":"2023-07-06T20:17:28.569139+0000","last_active":"2023-07-06T20:19:00.575210+0000","last_peered":"2023-07-06T20:19:00.575210+0000","last_clean":"2023-07-06T20:19:00.575210+0000","last_became_active":"2023-07-06T20:17:28.568828+0000","last_became_peered":"2023-07-06T20:17:28.568828+0000","last_unstale":"2023-07-06T20:19:00.575210+0000","last_undegraded":"2023-07-06T20:19:00.575210+0000","last_fullsized":"2023-07-06T20:19:00.575210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:32.012178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.648570+0000","last_change":"2023-07-06T20:17:28.562352+0000","last_active":"2023-07-06T20:18:59.648570+0000","last_peered":"2023-07-06T20:18:59.648570+0000","last_clean":"2023-07-06T20:18:59.648570+0000","last_became_active":"2023-07-06T20:17:28.562128+0000","last_became_peered":"2023-07-06T20:17:28.562128+0000","last_unstale":"2023-07-06T20:18:59.648570+0000","last_undegraded":"2023-07-06T20:18:59.648570+0000","last_fullsized":"2023-07-06T20:18:59.648570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:18:12.118763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:18:59.645970+0000","last_change":"2023-07-06T20:17:28.578940+0000","last_active":"2023-07-06T20:18:59.645970+0000","last_peered":"2023-07-06T20:18:59.645970+0000","last_clean":"2023-07-06T20:18:59.645970+0000","last_became_active":"2023-07-06T20:17:28.578660+0000","last_became_peered":"2023-07-06T20:17:28.578660+0000","last_unstale":"2023-07-06T20:18:59.645970+0000","last_undegraded":"2023-07-06T20:18:59.645970+0000","last_fullsized":"2023-07-06T20:18:59.645970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:00:23.290920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-06T20:19:00.574415+0000","last_change":"2023-07-06T20:17:28.573964+0000","last_active":"2023-07-06T20:19:00.574415+0000","last_peered":"2023-07-06T20:19:00.574415+0000","last_clean":"2023-07-06T20:19:00.574415+0000","last_became_active":"2023-07-06T20:17:28.573618+0000","last_became_peered":"2023-07-06T20:17:28.573618+0000","last_unstale":"2023-07-06T20:19:00.574415+0000","last_undegraded":"2023-07-06T20:19:00.574415+0000","last_fullsized":"2023-07-06T20:19:00.574415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T20:17:27.577668+0000","last_clean_scrub_stamp":"2023-07-06T20:17:27.577668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:06:11.474673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63664,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":589,"num_read_kb":362,"num_write":162,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1183,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":202062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449121,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":462935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.72599999999999998,"15min":0.67300000000000004},"min":{"1min":0.45600000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":4.968,"5min":4.968,"15min":4.968},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.04,"5min":0.71899999999999997,"15min":0.66500000000000004},"min":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":5.0019999999999998,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63800000000000001,"15min":0.629},"min":{"1min":0.45300000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.79700000000000004,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.36599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.72399999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69699999999999995,"15min":0.69099999999999995},"min":{"1min":0.48899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.622,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":1.006,"15min":1.006},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66400000000000003,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92800000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.47699999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90400000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.89100000000000001,"5min":1.097,"15min":1.097},"last":0.752}]},{"osd":7,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78900000000000003,"15min":0.79500000000000004},"min":{"1min":0.53700000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.98599999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.76000000000000001},"min":{"1min":0.53000000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68000000000000005}]},{"osd":8,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.80800000000000005}]},{"osd":9,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.60999999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.96799999999999997,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.89800000000000002,"15min":0.86799999999999999},"min":{"1min":0.67000000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":0.84399999999999997}]},{"osd":10,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78100000000000003,"15min":0.78800000000000003},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.88200000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.66200000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97099999999999997,"5min":1.208,"15min":1.208},"last":0.65400000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1616,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1654784,"data_stored":1112186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.66700000000000004}]},{"osd":1,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.434,"15min":0.436},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55000000000000004,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67600000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.499}]},{"osd":2,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86599999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.46200000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.624},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.874,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.32500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67700000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.496,"5min":0.439,"15min":0.439},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67500000000000004,"15min":0.67300000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.751}]},{"osd":7,"last update":"Thu Jul 6 20:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.48099999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.875,"5min":0.998,"15min":0.998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74299999999999999,"15min":0.748},"min":{"1min":0.57099999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.71899999999999997}]},{"osd":9,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.94699999999999995,"15min":0.98099999999999998},"min":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93999999999999995,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.91100000000000003,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.81399999999999995}]},{"osd":11,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73999999999999999,"15min":0.73899999999999999},"min":{"1min":0.57699999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.0049999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.872},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78300000000000003,"15min":0.78200000000000003},"min":{"1min":0.54200000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.85499999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":461676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.49299999999999999,"15min":0.503},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.93600000000000005,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83499999999999996,"5min":5.7919999999999998,"15min":5.7919999999999998},"last":0.435}]},{"osd":1,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.69099999999999995,"15min":0.70999999999999996},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89600000000000002,"5min":1.202,"15min":1.202},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.66800000000000004,"15min":0.68500000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.81999999999999995}]},{"osd":2,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.98499999999999999,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69699999999999995,"15min":0.71099999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.72699999999999998,"15min":0.66900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2729999999999999,"5min":16.628,"15min":16.628},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.73199999999999998,"15min":0.71399999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.083,"5min":11.332000000000001,"15min":11.332000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69399999999999995,"15min":0.70499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.73599999999999999,"15min":0.76100000000000001},"min":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.78900000000000003,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.80700000000000005,"15min":0.83299999999999996},"min":{"1min":0.51400000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91200000000000003,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79000000000000004,"15min":0.81599999999999995},"min":{"1min":0.58999999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.016,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.76500000000000001,"15min":0.78500000000000003},"min":{"1min":0.54100000000000004,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.115,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.77500000000000002,"15min":0.82499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84499999999999997,"15min":0.88},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.85899999999999999,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.86499999999999999,"15min":0.90400000000000003},"min":{"1min":0.625,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.054,"5min":1.339,"15min":1.339},"last":0.76700000000000002}]},{"osd":8,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.88200000000000001},"min":{"1min":0.504,"5min":0.496,"15min":0.496},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.751},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.877,"15min":0.89600000000000002},"min":{"1min":0.60299999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2230000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":0.72799999999999998}]},{"osd":10,"last update":"Thu Jul 6 20:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.85799999999999998,"15min":0.84999999999999998},"min":{"1min":0.68700000000000006,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.80500000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103213,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":461203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.747,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58699999999999997}]},{"osd":1,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.60099999999999998,"15min":0.57599999999999996},"min":{"1min":0.54800000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.48199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81000000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.45900000000000002,"15min":0.44800000000000001},"min":{"1min":0.34300000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.48099999999999998,"15min":0.47199999999999998},"min":{"1min":0.32400000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71299999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.48099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67700000000000005},"min":{"1min":0.54700000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.629,"15min":0.61399999999999999},"min":{"1min":0.48099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.66800000000000004,"15min":0.64700000000000002},"min":{"1min":0.621,"5min":0.433,"15min":0.433},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.89300000000000002}]},{"osd":5,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.622,"15min":0.60099999999999998},"min":{"1min":0.40200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.97999999999999998,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.74299999999999999,"15min":0.72099999999999997},"min":{"1min":0.64700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.73799999999999999,"15min":0.71999999999999997},"min":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.66100000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.0489999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.86799999999999999,"15min":0.90600000000000003},"min":{"1min":0.56499999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.091,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.95999999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.78800000000000003,"15min":0.74399999999999999},"min":{"1min":0.628,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.044},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.80500000000000005,"15min":0.76900000000000002},"min":{"1min":0.64400000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.97199999999999998}]},{"osd":10,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.76400000000000001,"15min":0.73899999999999999},"min":{"1min":0.69999999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.022},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.77300000000000002,"15min":0.75700000000000001},"min":{"1min":0.70399999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.0049999999999999}]},{"osd":11,"last update":"Thu Jul 6 20:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.996}]}]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1528,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1564672,"data_stored":1053036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51300000000000001,"15min":0.504},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85299999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.39400000000000002,"15min":0.38400000000000001},"min":{"1min":0.34699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.41299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.625,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.374}]},{"osd":2,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.40400000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80400000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.68000000000000005,"15min":0.622},"min":{"1min":0.40100000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.034,"5min":0.67300000000000004,"15min":0.59599999999999997},"min":{"1min":0.47899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.048,"5min":6.048,"15min":6.048},"last":0.53700000000000003}]},{"osd":4,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.32300000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.502},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.499,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59299999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.44700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.755,"15min":0.68100000000000005},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":8.1579999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.77000000000000002,"15min":0.70299999999999996},"min":{"1min":0.48799999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":8.1980000000000004,"5min":8.1980000000000004,"15min":8.1980000000000004},"last":0.66800000000000004}]},{"osd":8,"last update":"Thu Jul 6 20:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64600000000000002,"15min":0.63},"min":{"1min":0.47799999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.66600000000000004,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90500000000000003,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65400000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70599999999999996,"15min":0.70299999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71999999999999997,"15min":0.70399999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95099999999999996,"5min":1.419,"15min":1.419},"last":0.64000000000000001}]},{"osd":10,"last update":"Thu Jul 6 20:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.55300000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":1.014,"15min":1.014},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.54200000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54800000000000004}]},{"osd":11,"last update":"Thu Jul 6 20:18:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.755},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.73999999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":461071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49099999999999999,"15min":0.47399999999999998},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.91600000000000004,"5min":1.552,"15min":1.552},"last":0.875},{"interface":"front","average":{"1min":0.439,"5min":0.496,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66400000000000003,"5min":1.389,"15min":1.389},"last":0.53300000000000003}]},{"osd":1,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.71599999999999997,"15min":0.66800000000000004},"min":{"1min":0.55600000000000005,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.66800000000000004,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91000000000000003,"5min":1.089,"15min":1.089},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.68500000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.755},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94699999999999995,"5min":1.04,"15min":1.04},"last":0.93700000000000006}]},{"osd":3,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.60199999999999998,"15min":0.622},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.98999999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.95499999999999996,"5min":1.254,"15min":1.254},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86399999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.623}]},{"osd":5,"last update":"Thu Jul 6 20:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72999999999999998,"15min":0.76000000000000001},"min":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0109999999999999,"5min":1.393,"15min":1.393},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.75600000000000001,"15min":0.76100000000000001},"min":{"1min":0.505,"5min":0.496,"15min":0.496},"max":{"1min":0.90600000000000003,"5min":1.131,"15min":1.131},"last":1.0229999999999999}]},{"osd":7,"last update":"Thu Jul 6 20:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.79400000000000004,"15min":0.77800000000000002},"min":{"1min":0.55100000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.008},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.78700000000000003,"15min":0.78500000000000003},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.95599999999999996,"5min":1.274,"15min":1.274},"last":0.68799999999999994}]},{"osd":8,"last update":"Thu Jul 6 20:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81399999999999995,"15min":0.84399999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.077,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80300000000000005,"15min":0.82299999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0029999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Thu Jul 6 20:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.56299999999999994,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.05},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.89000000000000001,"15min":0.90200000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":1.091}]},{"osd":10,"last update":"Thu Jul 6 20:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92200000000000004,"15min":0.91400000000000003},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":3.8410000000000002,"5min":3.8410000000000002,"15min":3.8410000000000002},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.88100000000000001,"15min":0.89700000000000002},"min":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0269999999999999,"5min":1.244,"15min":1.244},"last":1.0780000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":520058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.495},"min":{"1min":0.32700000000000001,"5min":0.318,"15min":0.248},"max":{"1min":0.82199999999999995,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.53100000000000003},"min":{"1min":0.44600000000000001,"5min":0.34499999999999997,"15min":0.248},"max":{"1min":0.86099999999999999,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56899999999999995,"15min":0.51300000000000001},"min":{"1min":0.41599999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.93600000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.59399999999999997,"15min":0.52000000000000002},"min":{"1min":0.60699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94999999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.997}]},{"osd":3,"last update":"Thu Jul 6 20:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60499999999999998,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.90200000000000002,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.56499999999999995},"min":{"1min":0.42499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89000000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61499999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.42899999999999999,"15min":0.39300000000000002},"min":{"1min":0.33800000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.46999999999999997,"15min":0.42599999999999999},"min":{"1min":0.33500000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jul 6 20:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.60099999999999998,"15min":0.54900000000000004},"min":{"1min":0.55300000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90000000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.64100000000000001,"15min":0.57499999999999996},"min":{"1min":0.58999999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0289999999999999,"5min":1.042,"15min":1.042},"last":0.92900000000000005}]},{"osd":6,"last update":"Thu Jul 6 20:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69699999999999995,"15min":0.66300000000000003},"min":{"1min":0.52400000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.437,"5min":0.372,"15min":0.372},"max":{"1min":0.874,"5min":1.653,"15min":1.653},"last":0.97999999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.624,"15min":0.59199999999999997},"min":{"1min":0.42399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.59599999999999997,"15min":0.55500000000000005},"min":{"1min":0.5,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.0309999999999999}]},{"osd":8,"last update":"Thu Jul 6 20:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.69999999999999996,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96999999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.77000000000000002,"15min":0.73399999999999999},"min":{"1min":0.55100000000000005,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":1.101}]},{"osd":9,"last update":"Thu Jul 6 20:19:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.76600000000000001,"15min":0.746},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0129999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78900000000000003,"15min":0.77100000000000002},"min":{"1min":0.56000000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0189999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.014}]},{"osd":10,"last update":"Thu Jul 6 20:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76900000000000002,"15min":0.76100000000000001},"min":{"1min":0.47999999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.82799999999999996,"15min":0.81599999999999995},"min":{"1min":0.67300000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.86099999999999999}]},{"osd":11,"last update":"Thu Jul 6 20:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":462336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 20:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.317,"15min":0.317},"max":{"1min":4.1130000000000004,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.628,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.6400000000000001,"5min":4.1379999999999999,"15min":4.1379999999999999},"last":0.499}]},{"osd":2,"last update":"Thu Jul 6 20:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.63700000000000001,"15min":0.58999999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91100000000000003,"5min":1.085,"15min":1.085},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.66000000000000003,"15min":0.58599999999999997},"min":{"1min":0.44600000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97199999999999998,"5min":1.258,"15min":1.258},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jul 6 20:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.73799999999999999},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0049999999999999,"5min":4.1239999999999997,"15min":4.1239999999999997},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jul 6 20:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67500000000000004,"15min":0.64500000000000002},"min":{"1min":0.51300000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91000000000000003,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.46800000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95599999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.69299999999999995},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jul 6 20:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.03,"5min":1.131,"15min":1.131},"last":0.505},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79700000000000004,"15min":0.79600000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.179,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.84399999999999997}]},{"osd":8,"last update":"Thu Jul 6 20:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.78900000000000003,"15min":0.78600000000000003},"min":{"1min":0.54800000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.7650000000000001,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79700000000000004,"15min":0.81899999999999995},"min":{"1min":0.54100000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.073,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.65400000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.79000000000000004,"15min":0.82199999999999995},"min":{"1min":0.51100000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97799999999999998,"5min":1.161,"15min":1.161},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77500000000000002,"15min":0.79100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.7130000000000001,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.63100000000000001}]},{"osd":10,"last update":"Thu Jul 6 20:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.877,"15min":0.879},"min":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.681,"5min":2.681,"15min":2.681},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.86699999999999999,"15min":0.85499999999999998},"min":{"1min":0.54400000000000004,"5min":0.498,"15min":0.498},"max":{"1min":3.9630000000000001,"5min":3.9630000000000001,"15min":3.9630000000000001},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Jul 6 20:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.83299999999999996,"15min":0.81399999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.9209999999999998,"5min":3.9209999999999998,"15min":3.9209999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.111,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.89800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":1044480,"data_stored":527046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.33500000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.89400000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.56000000000000005},"min":{"1min":0.38200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84099999999999997,"5min":0.876,"15min":0.876},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.39800000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.87,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":1,"15min":1},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jul 6 20:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83499999999999996,"5min":0.995,"15min":0.995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.64600000000000002,"15min":0.63300000000000001},"min":{"1min":0.50600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jul 6 20:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jul 6 20:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49299999999999999,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77200000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.34499999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78400000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jul 6 20:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66500000000000004,"15min":0.64300000000000002},"min":{"1min":0.53100000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.97599999999999998,"5min":1.056,"15min":1.056},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.70599999999999996,"15min":0.66000000000000003},"min":{"1min":0.60399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0149999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.92700000000000005}]},{"osd":7,"last update":"Thu Jul 6 20:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.624},"min":{"1min":0.435,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81899999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91000000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.432,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.89100000000000001}]},{"osd":9,"last update":"Thu Jul 6 20:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.77300000000000002,"15min":0.76200000000000001},"min":{"1min":0.48099999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.81399999999999995,"15min":0.79600000000000004},"min":{"1min":0.65700000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.98099999999999998}]},{"osd":10,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.746,"15min":0.73499999999999999},"min":{"1min":0.56299999999999994,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.98999999999999999}]},{"osd":11,"last update":"Thu Jul 6 20:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.94499999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":461388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41399999999999998,"15min":0.38500000000000001},"min":{"1min":0.30499999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.51600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.45700000000000002,"15min":0.40999999999999998},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.36699999999999999}]},{"osd":1,"last update":"Thu Jul 6 20:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65800000000000003,"15min":0.58499999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81599999999999995,"5min":3.6360000000000001,"15min":3.6360000000000001},"last":0.5},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59799999999999998,"15min":0.57199999999999995},"min":{"1min":0.45200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95599999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jul 6 20:18:50 2023","interfaces":[{"interface":"back","average":{"1min":2.181,"5min":0.94199999999999995,"15min":0.69599999999999995},"min":{"1min":0.51900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":36.344000000000001,"5min":36.344000000000001,"15min":36.344000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67600000000000005,"15min":0.625},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93700000000000006,"5min":3.6720000000000002,"15min":3.6720000000000002},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jul 6 20:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.73199999999999998,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":6.9420000000000002,"5min":6.9420000000000002,"15min":6.9420000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63700000000000001,"15min":0.58699999999999997},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.442}]},{"osd":5,"last update":"Thu Jul 6 20:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.70599999999999996,"15min":0.66300000000000003},"min":{"1min":0.48899999999999999,"5min":0.441,"15min":0.441},"max":{"1min":4.5389999999999997,"5min":4.5389999999999997,"15min":4.5389999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66800000000000004,"15min":0.63},"min":{"1min":0.51600000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jul 6 20:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64200000000000002,"15min":0.61799999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86599999999999999,"5min":1.052,"15min":1.052},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.56799999999999995},"min":{"1min":0.40799999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95199999999999996,"5min":0.998,"15min":0.998},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.79900000000000004,"15min":0.76500000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.1859999999999999,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":0.73199999999999998}]},{"osd":8,"last update":"Thu Jul 6 20:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.58299999999999996}]},{"osd":9,"last update":"Thu Jul 6 20:19:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.77300000000000002,"15min":0.80200000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92400000000000004,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.71699999999999997,"15min":0.73499999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.71599999999999997}]},{"osd":10,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.046,"5min":1.173,"15min":1.173},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.077,"5min":0.89300000000000002,"15min":0.86199999999999999},"min":{"1min":0.68999999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":5.8040000000000003,"5min":5.8040000000000003,"15min":5.8040000000000003},"last":0.70599999999999996}]},{"osd":11,"last update":"Thu Jul 6 20:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":6.79,"5min":6.79,"15min":6.79},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.752}]}]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":462297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.55200000000000005,"15min":0.501},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":6.1539999999999999,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.51900000000000002,"15min":0.49299999999999999},"min":{"1min":0.45700000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.39000000000000001,"15min":0.35799999999999998},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58399999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.379},{"interface":"front","average":{"1min":0.44,"5min":0.39600000000000002,"15min":0.35799999999999998},"min":{"1min":0.311,"5min":0.219,"15min":0.219},"max":{"1min":0.55900000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.40500000000000003}]},{"osd":2,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.53500000000000003},"min":{"1min":0.47199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.753},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.44400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86699999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jul 6 20:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.53600000000000003},"min":{"1min":0.52200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.099,"5min":5.593,"15min":5.593},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.123,"5min":0.71699999999999997,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.378,"15min":0.378},"max":{"1min":9.7959999999999994,"5min":9.7959999999999994,"15min":9.7959999999999994},"last":0.44}]},{"osd":5,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.61599999999999999,"15min":0.56699999999999995},"min":{"1min":0.57599999999999996,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.63100000000000001,"15min":0.57999999999999996},"min":{"1min":0.47999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90900000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jul 6 20:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.40899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91600000000000004,"5min":1.109,"15min":1.109},"last":0.621},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.004,"5min":1.163,"15min":1.163},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jul 6 20:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.35499999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.97699999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.372,"5min":0.314,"15min":0.314},"max":{"1min":0.91200000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Thu Jul 6 20:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.754,"15min":0.74199999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0780000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.52300000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96599999999999997,"5min":1.194,"15min":1.194},"last":0.76500000000000001}]},{"osd":9,"last update":"Thu Jul 6 20:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71699999999999997,"15min":0.72499999999999998},"min":{"1min":0.54900000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.88100000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.92000000000000004,"15min":0.86899999999999999},"min":{"1min":0.63900000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0329999999999999,"5min":8.3369999999999997,"15min":8.3369999999999997},"last":0.81699999999999995}]},{"osd":10,"last update":"Thu Jul 6 20:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.82999999999999996,"15min":0.83299999999999996},"min":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.054,"5min":1.258,"15min":1.258},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.55300000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89700000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.71499999999999997}]},{"osd":11,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79300000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1576960,"data_stored":1052977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54100000000000004,"15min":0.53000000000000003},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81699999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.79200000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.42899999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.41299999999999998,"15min":0.40100000000000002},"min":{"1min":0.315,"5min":0.253,"15min":0.253},"max":{"1min":0.66100000000000003,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.31900000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.76400000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.58099999999999996},"min":{"1min":0.497,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jul 6 20:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.747},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.44800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.70299999999999996}]},{"osd":6,"last update":"Thu Jul 6 20:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.435,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66400000000000003,"15min":0.64200000000000002},"min":{"1min":0.51700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.85199999999999998,"5min":1.01,"15min":1.01},"last":0.76800000000000002}]},{"osd":7,"last update":"Thu Jul 6 20:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.81799999999999995}]},{"osd":8,"last update":"Thu Jul 6 20:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.46400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.45200000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.78200000000000003}]},{"osd":9,"last update":"Thu Jul 6 20:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.61899999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.59799999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.84199999999999997}]},{"osd":10,"last update":"Thu Jul 6 20:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.53300000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.92600000000000005,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.57699999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.96599999999999997,"5min":1.109,"15min":1.109},"last":0.88700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T20:19:11.490 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:19:11.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.723+0000 7f237b3ff700 1 -- 172.21.15.69:0/2981216451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23741017b0 msgr2=0x7f2374108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:11.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.723+0000 7f237b3ff700 1 --2- 172.21.15.69:0/2981216451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23741017b0 0x7f2374108070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f236c00b0a0 tx=0x7f236c00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.723+0000 7f237b3ff700 1 -- 172.21.15.69:0/2981216451 shutdown_connections 2023-07-06T20:19:11.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.723+0000 7f237b3ff700 1 --2- 172.21.15.69:0/2981216451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23741017b0 0x7f2374108070 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.723+0000 7f237b3ff700 1 --2- 172.21.15.69:0/2981216451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23740ffc30 0x7f2374100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.723+0000 7f237b3ff700 1 --2- 172.21.15.69:0/2981216451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2374100e20 0x7f2374101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.723+0000 7f237b3ff700 1 -- 172.21.15.69:0/2981216451 >> 172.21.15.69:0/2981216451 conn(0x7f23740fb210 msgr2=0x7f23740fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:11.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.723+0000 7f237b3ff700 1 -- 172.21.15.69:0/2981216451 shutdown_connections 2023-07-06T20:19:11.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.723+0000 7f237b3ff700 1 -- 172.21.15.69:0/2981216451 wait complete. 2023-07-06T20:19:11.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.724+0000 7f237b3ff700 1 Processor -- start 2023-07-06T20:19:11.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.724+0000 7f237b3ff700 1 -- start start 2023-07-06T20:19:11.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.724+0000 7f237b3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23740ffc30 0x7f2374195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:11.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.724+0000 7f237b3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2374100e20 0x7f2374195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:11.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.724+0000 7f237b3ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23741017b0 0x7f237419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:11.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.724+0000 7f237b3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237419d5b0 con 0x7f2374100e20 2023-07-06T20:19:11.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.724+0000 7f237b3ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237419d6f0 con 0x7f23741017b0 2023-07-06T20:19:11.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.724+0000 7f237b3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237419d880 con 0x7f23740ffc30 2023-07-06T20:19:11.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.725+0000 7f237899a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2374100e20 0x7f2374195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:11.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.725+0000 7f237919b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23740ffc30 0x7f2374195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:11.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.725+0000 7f237919b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23740ffc30 0x7f2374195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:59740/0 (socket says 172.21.15.69:59740) 2023-07-06T20:19:11.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.725+0000 7f237919b700 1 -- 172.21.15.69:0/934016373 learned_addr learned my addr 172.21.15.69:0/934016373 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:11.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.725+0000 7f237999c700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23741017b0 0x7f237419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:11.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.725+0000 7f237899a700 1 -- 172.21.15.69:0/934016373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23740ffc30 msgr2=0x7f2374195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:11.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.725+0000 7f237899a700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23740ffc30 0x7f2374195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.725+0000 7f237899a700 1 -- 172.21.15.69:0/934016373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23741017b0 msgr2=0x7f237419cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:11.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.725+0000 7f237899a700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23741017b0 0x7f237419cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.725+0000 7f237899a700 1 -- 172.21.15.69:0/934016373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c009d00 con 0x7f2374100e20 2023-07-06T20:19:11.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.725+0000 7f237899a700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2374100e20 0x7f2374195f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f236400d750 tx=0x7f236400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:11.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.725+0000 7f236a7fc700 1 -- 172.21.15.69:0/934016373 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364004020 con 0x7f2374100e20 2023-07-06T20:19:11.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.726+0000 7f236a7fc700 1 -- 172.21.15.69:0/934016373 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2364004830 con 0x7f2374100e20 2023-07-06T20:19:11.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.726+0000 7f236a7fc700 1 -- 172.21.15.69:0/934016373 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640054c0 con 0x7f2374100e20 2023-07-06T20:19:11.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.726+0000 7f237b3ff700 1 -- 172.21.15.69:0/934016373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237419db60 con 0x7f2374100e20 2023-07-06T20:19:11.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.726+0000 7f237b3ff700 1 -- 172.21.15.69:0/934016373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237419e070 con 0x7f2374100e20 2023-07-06T20:19:11.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.727+0000 7f237b3ff700 1 -- 172.21.15.69:0/934016373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23741070d0 con 0x7f2374100e20 2023-07-06T20:19:11.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.729+0000 7f236a7fc700 1 -- 172.21.15.69:0/934016373 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f23640049a0 con 0x7f2374100e20 2023-07-06T20:19:11.731 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.730+0000 7f236a7fc700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f23600a1220 0x7f23600a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:11.731 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.730+0000 7f236a7fc700 1 -- 172.21.15.69:0/934016373 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f23640cc1a0 con 0x7f2374100e20 2023-07-06T20:19:11.731 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.730+0000 7f237919b700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f23600a1220 0x7f23600a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:11.732 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.731+0000 7f237919b700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f23600a1220 0x7f23600a36d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2370007cf0 tx=0x7f2370007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:11.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.734+0000 7f236a7fc700 1 -- 172.21.15.69:0/934016373 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2364095860 con 0x7f2374100e20 2023-07-06T20:19:11.758 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:11 smithi069 ceph-mon[126399]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:19:11.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:11 smithi069 ceph-mon[126399]: from='client.25216 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:19:11.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.933+0000 7f237b3ff700 1 -- 172.21.15.69:0/934016373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f237402c8e0 con 0x7f2374100e20 2023-07-06T20:19:11.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.935+0000 7f236a7fc700 1 -- 172.21.15.69:0/934016373 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7f2364098e80 con 0x7f2374100e20 2023-07-06T20:19:11.936 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:11.936 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":87,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:18:59.638711+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:19:11.938 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.937+0000 7f237b3ff700 1 -- 172.21.15.69:0/934016373 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f23600a1220 msgr2=0x7f23600a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:11.938 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.937+0000 7f237b3ff700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f23600a1220 0x7f23600a36d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2370007cf0 tx=0x7f2370007480 comp rx=0 tx=0).stop 2023-07-06T20:19:11.938 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.937+0000 7f237b3ff700 1 -- 172.21.15.69:0/934016373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2374100e20 msgr2=0x7f2374195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:11.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.937+0000 7f237b3ff700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2374100e20 0x7f2374195f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f236400d750 tx=0x7f236400da60 comp rx=0 tx=0).stop 2023-07-06T20:19:11.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.937+0000 7f237b3ff700 1 -- 172.21.15.69:0/934016373 shutdown_connections 2023-07-06T20:19:11.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.937+0000 7f237b3ff700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f23600a1220 0x7f23600a36d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.937+0000 7f237b3ff700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23741017b0 0x7f237419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.937+0000 7f237b3ff700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23740ffc30 0x7f2374195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.937+0000 7f237b3ff700 1 --2- 172.21.15.69:0/934016373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2374100e20 0x7f2374195f70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:11.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.937+0000 7f237b3ff700 1 -- 172.21.15.69:0/934016373 >> 172.21.15.69:0/934016373 conn(0x7f23740fb210 msgr2=0x7f2374104210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:11.940 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.937+0000 7f237b3ff700 1 -- 172.21.15.69:0/934016373 shutdown_connections 2023-07-06T20:19:11.940 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:11.937+0000 7f237b3ff700 1 -- 172.21.15.69:0/934016373 wait complete. 2023-07-06T20:19:11.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:14.022799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:17:26.523183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:17:27.543386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:19:11.958 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:19:12.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:11 smithi156 ceph-mon[129280]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:19:12.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:11 smithi156 ceph-mon[129280]: from='client.25216 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:19:12.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:11 smithi098 ceph-mon[128968]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:19:12.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:11 smithi098 ceph-mon[128968]: from='client.25216 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:19:12.156 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.155+0000 7f4cbbfff700 1 -- 172.21.15.69:0/2657501168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb41017a0 msgr2=0x7f4cb4108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:12.156 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.155+0000 7f4cbbfff700 1 --2- 172.21.15.69:0/2657501168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb41017a0 0x7f4cb4108060 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00b0a0 tx=0x7f4cac00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.155+0000 7f4cbbfff700 1 -- 172.21.15.69:0/2657501168 shutdown_connections 2023-07-06T20:19:12.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.155+0000 7f4cbbfff700 1 --2- 172.21.15.69:0/2657501168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb41017a0 0x7f4cb4108060 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.155+0000 7f4cbbfff700 1 --2- 172.21.15.69:0/2657501168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cb40ffc20 0x7f4cb4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.155+0000 7f4cbbfff700 1 --2- 172.21.15.69:0/2657501168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb4100e10 0x7f4cb4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.155+0000 7f4cbbfff700 1 -- 172.21.15.69:0/2657501168 >> 172.21.15.69:0/2657501168 conn(0x7f4cb40fb260 msgr2=0x7f4cb40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:12.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.155+0000 7f4cbbfff700 1 -- 172.21.15.69:0/2657501168 shutdown_connections 2023-07-06T20:19:12.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.155+0000 7f4cbbfff700 1 -- 172.21.15.69:0/2657501168 wait complete. 2023-07-06T20:19:12.159 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.156+0000 7f4cbbfff700 1 Processor -- start 2023-07-06T20:19:12.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.156+0000 7f4cbbfff700 1 -- start start 2023-07-06T20:19:12.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cbbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb40ffc20 0x7f4cb4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:12.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cbbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cb4100e10 0x7f4cb41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:12.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cbbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb41017a0 0x7f4cb419f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:12.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cbbfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb419f800 con 0x7f4cb40ffc20 2023-07-06T20:19:12.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cbbfff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb419f970 con 0x7f4cb4100e10 2023-07-06T20:19:12.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cbbfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb419fb30 con 0x7f4cb41017a0 2023-07-06T20:19:12.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cb9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb40ffc20 0x7f4cb4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:12.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cb9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb40ffc20 0x7f4cb4197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58458/0 (socket says 172.21.15.69:58458) 2023-07-06T20:19:12.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cb9d9b700 1 -- 172.21.15.69:0/507303675 learned_addr learned my addr 172.21.15.69:0/507303675 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:12.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cb959a700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cb4100e10 0x7f4cb41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:12.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cba59c700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb41017a0 0x7f4cb419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:12.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cb9d9b700 1 -- 172.21.15.69:0/507303675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb41017a0 msgr2=0x7f4cb419f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:12.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cb9d9b700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb41017a0 0x7f4cb419f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cb9d9b700 1 -- 172.21.15.69:0/507303675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cb4100e10 msgr2=0x7f4cb41981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:12.163 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cb9d9b700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cb4100e10 0x7f4cb41981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.163 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cb9d9b700 1 -- 172.21.15.69:0/507303675 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac009d00 con 0x7f4cb40ffc20 2023-07-06T20:19:12.163 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.157+0000 7f4cba59c700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb41017a0 0x7f4cb419f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:19:12.163 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.158+0000 7f4cb9d9b700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb40ffc20 0x7f4cb4197c90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000e970 tx=0x7f4cb000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:12.164 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.158+0000 7f4caaffd700 1 -- 172.21.15.69:0/507303675 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0009790 con 0x7f4cb40ffc20 2023-07-06T20:19:12.164 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.158+0000 7f4caaffd700 1 -- 172.21.15.69:0/507303675 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb000eec0 con 0x7f4cb40ffc20 2023-07-06T20:19:12.164 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.158+0000 7f4cbbfff700 1 -- 172.21.15.69:0/507303675 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb419fe10 con 0x7f4cb40ffc20 2023-07-06T20:19:12.164 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.158+0000 7f4caaffd700 1 -- 172.21.15.69:0/507303675 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0003cf0 con 0x7f4cb40ffc20 2023-07-06T20:19:12.165 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.158+0000 7f4cbbfff700 1 -- 172.21.15.69:0/507303675 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb41a0290 con 0x7f4cb40ffc20 2023-07-06T20:19:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.159+0000 7f4cbbfff700 1 -- 172.21.15.69:0/507303675 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb410b450 con 0x7f4cb40ffc20 2023-07-06T20:19:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.161+0000 7f4caaffd700 1 -- 172.21.15.69:0/507303675 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4cb000ccd0 con 0x7f4cb40ffc20 2023-07-06T20:19:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.162+0000 7f4caaffd700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4ca00a11d0 0x7f4ca00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.163+0000 7f4caaffd700 1 -- 172.21.15.69:0/507303675 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f4cb0014070 con 0x7f4cb40ffc20 2023-07-06T20:19:12.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.163+0000 7f4cb959a700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4ca00a11d0 0x7f4ca00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.163+0000 7f4cb959a700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4ca00a11d0 0x7f4ca00a3680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4199190 tx=0x7f4ca4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.166+0000 7f4caaffd700 1 -- 172.21.15.69:0/507303675 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4cb0096220 con 0x7f4cb40ffc20 2023-07-06T20:19:12.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.363+0000 7f4cbbfff700 1 -- 172.21.15.69:0/507303675 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cb41a0b30 con 0x7f4cb40ffc20 2023-07-06T20:19:12.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.364+0000 7f4caaffd700 1 -- 172.21.15.69:0/507303675 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4cb0099840 con 0x7f4cb40ffc20 2023-07-06T20:19:12.365 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2023-07-06T20:19:12.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.366+0000 7f4cbbfff700 1 -- 172.21.15.69:0/507303675 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4ca00a11d0 msgr2=0x7f4ca00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:12.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.366+0000 7f4cbbfff700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4ca00a11d0 0x7f4ca00a3680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4199190 tx=0x7f4ca4009260 comp rx=0 tx=0).stop 2023-07-06T20:19:12.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.366+0000 7f4cbbfff700 1 -- 172.21.15.69:0/507303675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb40ffc20 msgr2=0x7f4cb4197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:12.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.366+0000 7f4cbbfff700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb40ffc20 0x7f4cb4197c90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000e970 tx=0x7f4cb000ec80 comp rx=0 tx=0).stop 2023-07-06T20:19:12.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.366+0000 7f4cbbfff700 1 -- 172.21.15.69:0/507303675 shutdown_connections 2023-07-06T20:19:12.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.366+0000 7f4cbbfff700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4ca00a11d0 0x7f4ca00a3680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.366+0000 7f4cbbfff700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb41017a0 0x7f4cb419f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.369 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.366+0000 7f4cbbfff700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb40ffc20 0x7f4cb4197c90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.369 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.366+0000 7f4cbbfff700 1 --2- 172.21.15.69:0/507303675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cb4100e10 0x7f4cb41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.369 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.366+0000 7f4cbbfff700 1 -- 172.21.15.69:0/507303675 >> 172.21.15.69:0/507303675 conn(0x7f4cb40fb260 msgr2=0x7f4cb40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:12.370 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.367+0000 7f4cbbfff700 1 -- 172.21.15.69:0/507303675 shutdown_connections 2023-07-06T20:19:12.370 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.367+0000 7f4cbbfff700 1 -- 172.21.15.69:0/507303675 wait complete. 2023-07-06T20:19:12.386 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:19:12.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.587+0000 7fd7121ff700 1 -- 172.21.15.69:0/3636359352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd70c1017a0 msgr2=0x7fd70c108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:12.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.587+0000 7fd7121ff700 1 --2- 172.21.15.69:0/3636359352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd70c1017a0 0x7fd70c108060 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd70400b0a0 tx=0x7fd70400b3b0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.587+0000 7fd7121ff700 1 -- 172.21.15.69:0/3636359352 shutdown_connections 2023-07-06T20:19:12.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.587+0000 7fd7121ff700 1 --2- 172.21.15.69:0/3636359352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd70c1017a0 0x7fd70c108060 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.587+0000 7fd7121ff700 1 --2- 172.21.15.69:0/3636359352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd70c0ffc20 0x7fd70c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.587+0000 7fd7121ff700 1 --2- 172.21.15.69:0/3636359352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd70c100e10 0x7fd70c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.587+0000 7fd7121ff700 1 -- 172.21.15.69:0/3636359352 >> 172.21.15.69:0/3636359352 conn(0x7fd70c0fb260 msgr2=0x7fd70c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:12.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.587+0000 7fd7121ff700 1 -- 172.21.15.69:0/3636359352 shutdown_connections 2023-07-06T20:19:12.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.587+0000 7fd7121ff700 1 -- 172.21.15.69:0/3636359352 wait complete. 2023-07-06T20:19:12.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.588+0000 7fd7121ff700 1 Processor -- start 2023-07-06T20:19:12.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.588+0000 7fd7121ff700 1 -- start start 2023-07-06T20:19:12.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd7121ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd70c0ffc20 0x7fd70c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:12.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd7121ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd70c100e10 0x7fd70c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:12.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd7121ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd70c1017a0 0x7fd70c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:12.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd7121ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c19f7a0 con 0x7fd70c0ffc20 2023-07-06T20:19:12.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd7121ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c19f910 con 0x7fd70c1017a0 2023-07-06T20:19:12.592 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd7121ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c19fad0 con 0x7fd70c100e10 2023-07-06T20:19:12.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd70b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd70c0ffc20 0x7fd70c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:12.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd70b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd70c0ffc20 0x7fd70c197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58468/0 (socket says 172.21.15.69:58468) 2023-07-06T20:19:12.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd70b7fe700 1 -- 172.21.15.69:0/1298463356 learned_addr learned my addr 172.21.15.69:0/1298463356 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:12.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd70bfff700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd70c1017a0 0x7fd70c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:12.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd70affd700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd70c100e10 0x7fd70c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:12.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd70b7fe700 1 -- 172.21.15.69:0/1298463356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd70c100e10 msgr2=0x7fd70c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:12.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd70b7fe700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd70c100e10 0x7fd70c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd70b7fe700 1 -- 172.21.15.69:0/1298463356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd70c1017a0 msgr2=0x7fd70c19f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:12.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd70b7fe700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd70c1017a0 0x7fd70c19f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.589+0000 7fd70b7fe700 1 -- 172.21.15.69:0/1298463356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd704009d00 con 0x7fd70c0ffc20 2023-07-06T20:19:12.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.590+0000 7fd70b7fe700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd70c0ffc20 0x7fd70c197c60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00e970 tx=0x7fd6fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:12.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.590+0000 7fd708ff9700 1 -- 172.21.15.69:0/1298463356 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc009790 con 0x7fd70c0ffc20 2023-07-06T20:19:12.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.590+0000 7fd708ff9700 1 -- 172.21.15.69:0/1298463356 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6fc00c7e0 con 0x7fd70c0ffc20 2023-07-06T20:19:12.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.590+0000 7fd7121ff700 1 -- 172.21.15.69:0/1298463356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70c19fdb0 con 0x7fd70c0ffc20 2023-07-06T20:19:12.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.590+0000 7fd708ff9700 1 -- 172.21.15.69:0/1298463356 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc003c70 con 0x7fd70c0ffc20 2023-07-06T20:19:12.595 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.590+0000 7fd7121ff700 1 -- 172.21.15.69:0/1298463356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70c1a0220 con 0x7fd70c0ffc20 2023-07-06T20:19:12.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.591+0000 7fd7121ff700 1 -- 172.21.15.69:0/1298463356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70c10b400 con 0x7fd70c0ffc20 2023-07-06T20:19:12.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.593+0000 7fd708ff9700 1 -- 172.21.15.69:0/1298463356 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd6fc00cc50 con 0x7fd70c0ffc20 2023-07-06T20:19:12.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.594+0000 7fd708ff9700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd6f40a11d0 0x7fd6f40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:12.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.594+0000 7fd708ff9700 1 -- 172.21.15.69:0/1298463356 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd6fc014070 con 0x7fd70c0ffc20 2023-07-06T20:19:12.596 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.595+0000 7fd70affd700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd6f40a11d0 0x7fd6f40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:12.597 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.595+0000 7fd70affd700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd6f40a11d0 0x7fd6f40a3680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd700009c60 tx=0x7fd700009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:12.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.599+0000 7fd708ff9700 1 -- 172.21.15.69:0/1298463356 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd6fc096190 con 0x7fd70c0ffc20 2023-07-06T20:19:12.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.799+0000 7fd7121ff700 1 -- 172.21.15.69:0/1298463356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd70c1a0890 con 0x7fd70c0ffc20 2023-07-06T20:19:12.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.800+0000 7fd708ff9700 1 -- 172.21.15.69:0/1298463356 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd6fc003dd0 con 0x7fd70c0ffc20 2023-07-06T20:19:12.801 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2023-07-06T20:19:12.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.802+0000 7fd7121ff700 1 -- 172.21.15.69:0/1298463356 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd6f40a11d0 msgr2=0x7fd6f40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:12.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.802+0000 7fd7121ff700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd6f40a11d0 0x7fd6f40a3680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd700009c60 tx=0x7fd700009260 comp rx=0 tx=0).stop 2023-07-06T20:19:12.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.802+0000 7fd7121ff700 1 -- 172.21.15.69:0/1298463356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd70c0ffc20 msgr2=0x7fd70c197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:12.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.802+0000 7fd7121ff700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd70c0ffc20 0x7fd70c197c60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00e970 tx=0x7fd6fc00ec80 comp rx=0 tx=0).stop 2023-07-06T20:19:12.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.802+0000 7fd7121ff700 1 -- 172.21.15.69:0/1298463356 shutdown_connections 2023-07-06T20:19:12.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.802+0000 7fd7121ff700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd6f40a11d0 0x7fd6f40a3680 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.802+0000 7fd7121ff700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd70c1017a0 0x7fd70c19f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.802+0000 7fd7121ff700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd70c0ffc20 0x7fd70c197c60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.802+0000 7fd7121ff700 1 --2- 172.21.15.69:0/1298463356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd70c100e10 0x7fd70c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:12.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.802+0000 7fd7121ff700 1 -- 172.21.15.69:0/1298463356 >> 172.21.15.69:0/1298463356 conn(0x7fd70c0fb260 msgr2=0x7fd70c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:12.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.802+0000 7fd7121ff700 1 -- 172.21.15.69:0/1298463356 shutdown_connections 2023-07-06T20:19:12.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:12.802+0000 7fd7121ff700 1 -- 172.21.15.69:0/1298463356 wait complete. 2023-07-06T20:19:12.822 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:19:13.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:12 smithi069 ceph-mon[126399]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:19:13.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:12 smithi069 ceph-mon[126399]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s; 89 B/s, 0 objects/s recovering 2023-07-06T20:19:13.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:12 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/934016373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:13.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:12 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/507303675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:19:13.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.024+0000 7f4e74b9d700 1 -- 172.21.15.69:0/3488436416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e70105860 msgr2=0x7f4e70107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.024+0000 7f4e74b9d700 1 --2- 172.21.15.69:0/3488436416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e70105860 0x7f4e70107c40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800b0a0 tx=0x7f4e6800b3b0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.024+0000 7f4e74b9d700 1 -- 172.21.15.69:0/3488436416 shutdown_connections 2023-07-06T20:19:13.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.024+0000 7f4e74b9d700 1 --2- 172.21.15.69:0/3488436416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e70105860 0x7f4e70107c40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.024+0000 7f4e74b9d700 1 --2- 172.21.15.69:0/3488436416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e70100620 0x7f4e70102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.024+0000 7f4e74b9d700 1 --2- 172.21.15.69:0/3488436416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e70102f40 0x7f4e70105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.024+0000 7f4e74b9d700 1 -- 172.21.15.69:0/3488436416 >> 172.21.15.69:0/3488436416 conn(0x7f4e700fa000 msgr2=0x7f4e700fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:13.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.024+0000 7f4e74b9d700 1 -- 172.21.15.69:0/3488436416 shutdown_connections 2023-07-06T20:19:13.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.024+0000 7f4e74b9d700 1 -- 172.21.15.69:0/3488436416 wait complete. 2023-07-06T20:19:13.027 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.025+0000 7f4e74b9d700 1 Processor -- start 2023-07-06T20:19:13.028 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.025+0000 7f4e74b9d700 1 -- start start 2023-07-06T20:19:13.028 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.025+0000 7f4e74b9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e70100620 0x7f4e700fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:13.028 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.025+0000 7f4e74b9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e70102f40 0x7f4e700fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:13.028 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.025+0000 7f4e74b9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e70105860 0x7f4e700feb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:13.028 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.025+0000 7f4e74b9d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700ff2b0 con 0x7f4e70102f40 2023-07-06T20:19:13.029 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.025+0000 7f4e74b9d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700ff420 con 0x7f4e70105860 2023-07-06T20:19:13.029 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.025+0000 7f4e74b9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7019f810 con 0x7f4e70100620 2023-07-06T20:19:13.029 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.025+0000 7f4e6effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e70102f40 0x7f4e700fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:13.029 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.025+0000 7f4e6ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e70105860 0x7f4e700feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:13.029 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e6ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e70105860 0x7f4e700feb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:47650/0 (socket says 172.21.15.69:47650) 2023-07-06T20:19:13.029 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e6ffff700 1 -- 172.21.15.69:0/1148764245 learned_addr learned my addr 172.21.15.69:0/1148764245 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:13.030 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e6f7fe700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e70100620 0x7f4e700fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:13.030 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e6ffff700 1 -- 172.21.15.69:0/1148764245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e70100620 msgr2=0x7f4e700fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.030 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e6ffff700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e70100620 0x7f4e700fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.030 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e6ffff700 1 -- 172.21.15.69:0/1148764245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e70102f40 msgr2=0x7f4e700fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.030 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e6ffff700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e70102f40 0x7f4e700fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.030 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e6ffff700 1 -- 172.21.15.69:0/1148764245 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e68009d00 con 0x7f4e70105860 2023-07-06T20:19:13.030 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e6ffff700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e70105860 0x7f4e700feb10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009fd0 tx=0x7f4e6800bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:13.030 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e6cff9700 1 -- 172.21.15.69:0/1148764245 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800e040 con 0x7f4e70105860 2023-07-06T20:19:13.031 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e6cff9700 1 -- 172.21.15.69:0/1148764245 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e680094a0 con 0x7f4e70105860 2023-07-06T20:19:13.031 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e6cff9700 1 -- 172.21.15.69:0/1148764245 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6801b6c0 con 0x7f4e70105860 2023-07-06T20:19:13.031 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.026+0000 7f4e74b9d700 1 -- 172.21.15.69:0/1148764245 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7019fa90 con 0x7f4e70105860 2023-07-06T20:19:13.031 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.027+0000 7f4e74b9d700 1 -- 172.21.15.69:0/1148764245 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7019fe60 con 0x7f4e70105860 2023-07-06T20:19:13.032 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.028+0000 7f4e74b9d700 1 -- 172.21.15.69:0/1148764245 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e700fbb70 con 0x7f4e70105860 2023-07-06T20:19:13.032 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.030+0000 7f4e6cff9700 1 -- 172.21.15.69:0/1148764245 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4e6801b820 con 0x7f4e70105860 2023-07-06T20:19:13.032 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.031+0000 7f4e6cff9700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4e580a1270 0x7f4e580a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:13.032 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.031+0000 7f4e6cff9700 1 -- 172.21.15.69:0/1148764245 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f4e680cdad0 con 0x7f4e70105860 2023-07-06T20:19:13.033 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.032+0000 7f4e6f7fe700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4e580a1270 0x7f4e580a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:13.033 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.032+0000 7f4e6f7fe700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4e580a1270 0x7f4e580a3720 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009c60 tx=0x7f4e60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:13.034 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.033+0000 7f4e6cff9700 1 -- 172.21.15.69:0/1148764245 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4e68097220 con 0x7f4e70105860 2023-07-06T20:19:13.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:12 smithi156 ceph-mon[129280]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:19:13.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:12 smithi156 ceph-mon[129280]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s; 89 B/s, 0 objects/s recovering 2023-07-06T20:19:13.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:12 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/934016373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:13.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:12 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/507303675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:19:13.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:12 smithi098 ceph-mon[128968]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-06T20:19:13.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:12 smithi098 ceph-mon[128968]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s; 89 B/s, 0 objects/s recovering 2023-07-06T20:19:13.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:12 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/934016373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:13.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:12 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/507303675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:19:13.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.230+0000 7f4e74b9d700 1 -- 172.21.15.69:0/1148764245 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e7002ca70 con 0x7f4e70105860 2023-07-06T20:19:13.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.231+0000 7f4e6cff9700 1 -- 172.21.15.69:0/1148764245 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e680a00a0 con 0x7f4e70105860 2023-07-06T20:19:13.233 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2023-07-06T20:19:13.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.234+0000 7f4e74b9d700 1 -- 172.21.15.69:0/1148764245 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4e580a1270 msgr2=0x7f4e580a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.234+0000 7f4e74b9d700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4e580a1270 0x7f4e580a3720 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009c60 tx=0x7f4e60009260 comp rx=0 tx=0).stop 2023-07-06T20:19:13.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.234+0000 7f4e74b9d700 1 -- 172.21.15.69:0/1148764245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e70105860 msgr2=0x7f4e700feb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.234+0000 7f4e74b9d700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e70105860 0x7f4e700feb10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009fd0 tx=0x7f4e6800bce0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.234+0000 7f4e74b9d700 1 -- 172.21.15.69:0/1148764245 shutdown_connections 2023-07-06T20:19:13.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.234+0000 7f4e74b9d700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f4e580a1270 0x7f4e580a3720 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.234+0000 7f4e74b9d700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e70105860 0x7f4e700feb10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.234+0000 7f4e74b9d700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e70100620 0x7f4e700fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.234+0000 7f4e74b9d700 1 --2- 172.21.15.69:0/1148764245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e70102f40 0x7f4e700fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.234+0000 7f4e74b9d700 1 -- 172.21.15.69:0/1148764245 >> 172.21.15.69:0/1148764245 conn(0x7f4e700fa000 msgr2=0x7f4e70101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:13.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.234+0000 7f4e74b9d700 1 -- 172.21.15.69:0/1148764245 shutdown_connections 2023-07-06T20:19:13.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.234+0000 7f4e74b9d700 1 -- 172.21.15.69:0/1148764245 wait complete. 2023-07-06T20:19:13.253 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:19:13.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.450+0000 7f9f19fff700 1 -- 172.21.15.69:0/3931394086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f140ffc30 msgr2=0x7f9f14100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.450+0000 7f9f19fff700 1 --2- 172.21.15.69:0/3931394086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f140ffc30 0x7f9f14100040 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009990 tx=0x7f9f04009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.450+0000 7f9f19fff700 1 -- 172.21.15.69:0/3931394086 shutdown_connections 2023-07-06T20:19:13.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.450+0000 7f9f19fff700 1 --2- 172.21.15.69:0/3931394086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f141017b0 0x7f9f14108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.450+0000 7f9f19fff700 1 --2- 172.21.15.69:0/3931394086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f140ffc30 0x7f9f14100040 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.450+0000 7f9f19fff700 1 --2- 172.21.15.69:0/3931394086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f14100e20 0x7f9f14101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.450+0000 7f9f19fff700 1 -- 172.21.15.69:0/3931394086 >> 172.21.15.69:0/3931394086 conn(0x7f9f140fb210 msgr2=0x7f9f140fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:13.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.450+0000 7f9f19fff700 1 -- 172.21.15.69:0/3931394086 shutdown_connections 2023-07-06T20:19:13.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.450+0000 7f9f19fff700 1 -- 172.21.15.69:0/3931394086 wait complete. 2023-07-06T20:19:13.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.451+0000 7f9f19fff700 1 Processor -- start 2023-07-06T20:19:13.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.451+0000 7f9f19fff700 1 -- start start 2023-07-06T20:19:13.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.451+0000 7f9f19fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f140ffc30 0x7f9f140743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:13.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.451+0000 7f9f19fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f14100e20 0x7f9f14072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:13.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f19fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f141017b0 0x7f9f14072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:13.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f19fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f14073710 con 0x7f9f14100e20 2023-07-06T20:19:13.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f19fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f14073880 con 0x7f9f141017b0 2023-07-06T20:19:13.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f19fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1419f9f0 con 0x7f9f140ffc30 2023-07-06T20:19:13.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f12ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f14100e20 0x7f9f14072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:13.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f12ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f14100e20 0x7f9f14072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53098/0 (socket says 172.21.15.69:53098) 2023-07-06T20:19:13.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f12ffd700 1 -- 172.21.15.69:0/3685124082 learned_addr learned my addr 172.21.15.69:0/3685124082 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:13.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f137fe700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f140ffc30 0x7f9f140743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f13fff700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f141017b0 0x7f9f14072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f137fe700 1 -- 172.21.15.69:0/3685124082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f141017b0 msgr2=0x7f9f14072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f137fe700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f141017b0 0x7f9f14072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f137fe700 1 -- 172.21.15.69:0/3685124082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f14100e20 msgr2=0x7f9f14072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f137fe700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f14100e20 0x7f9f14072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f137fe700 1 -- 172.21.15.69:0/3685124082 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f040095f0 con 0x7f9f140ffc30 2023-07-06T20:19:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f12ffd700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f14100e20 0x7f9f14072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:13.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.452+0000 7f9f137fe700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f140ffc30 0x7f9f140743f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9f04006010 tx=0x7f9f04005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:13.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.453+0000 7f9f10ff9700 1 -- 172.21.15.69:0/3685124082 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0401d070 con 0x7f9f140ffc30 2023-07-06T20:19:13.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.453+0000 7f9f19fff700 1 -- 172.21.15.69:0/3685124082 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1419fc70 con 0x7f9f140ffc30 2023-07-06T20:19:13.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.453+0000 7f9f10ff9700 1 -- 172.21.15.69:0/3685124082 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f04003f60 con 0x7f9f140ffc30 2023-07-06T20:19:13.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.453+0000 7f9f10ff9700 1 -- 172.21.15.69:0/3685124082 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f04022d90 con 0x7f9f140ffc30 2023-07-06T20:19:13.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.453+0000 7f9f19fff700 1 -- 172.21.15.69:0/3685124082 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f141a00a0 con 0x7f9f140ffc30 2023-07-06T20:19:13.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.454+0000 7f9f19fff700 1 -- 172.21.15.69:0/3685124082 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1410b480 con 0x7f9f140ffc30 2023-07-06T20:19:13.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.456+0000 7f9f10ff9700 1 -- 172.21.15.69:0/3685124082 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9f04022470 con 0x7f9f140ffc30 2023-07-06T20:19:13.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.457+0000 7f9f10ff9700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9efc0a11a0 0x7f9efc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:13.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.457+0000 7f9f10ff9700 1 -- 172.21.15.69:0/3685124082 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f9f040ce4d0 con 0x7f9f140ffc30 2023-07-06T20:19:13.460 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.458+0000 7f9f12ffd700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9efc0a11a0 0x7f9efc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:13.460 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.458+0000 7f9f12ffd700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9efc0a11a0 0x7f9efc0a3650 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009ea0 tx=0x7f9f08009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:13.460 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.459+0000 7f9f10ff9700 1 -- 172.21.15.69:0/3685124082 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9f04097c80 con 0x7f9f140ffc30 2023-07-06T20:19:13.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.706+0000 7f9f19fff700 1 -- 172.21.15.69:0/3685124082 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f14106ac0 con 0x7f9f140ffc30 2023-07-06T20:19:13.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.707+0000 7f9f10ff9700 1 -- 172.21.15.69:0/3685124082 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10795 (secure 0 0 0) 0x7f9f0409b2a0 con 0x7f9f140ffc30 2023-07-06T20:19:13.708 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:13.709 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-06T20:17:28.144582+0000","modified":"2023-07-06T20:17:33.557511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24539,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/3235525224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3235525224},{"type":"v1","addr":"172.21.15.98:6839","nonce":3235525224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:19:13.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.710+0000 7f9f19fff700 1 -- 172.21.15.69:0/3685124082 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9efc0a11a0 msgr2=0x7f9efc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.710+0000 7f9f19fff700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9efc0a11a0 0x7f9efc0a3650 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009ea0 tx=0x7f9f08009290 comp rx=0 tx=0).stop 2023-07-06T20:19:13.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.710+0000 7f9f19fff700 1 -- 172.21.15.69:0/3685124082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f140ffc30 msgr2=0x7f9f140743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.710+0000 7f9f19fff700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f140ffc30 0x7f9f140743f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9f04006010 tx=0x7f9f04005740 comp rx=0 tx=0).stop 2023-07-06T20:19:13.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.710+0000 7f9f19fff700 1 -- 172.21.15.69:0/3685124082 shutdown_connections 2023-07-06T20:19:13.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.710+0000 7f9f19fff700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9efc0a11a0 0x7f9efc0a3650 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.710+0000 7f9f19fff700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f141017b0 0x7f9f14072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.710+0000 7f9f19fff700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f140ffc30 0x7f9f140743f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.710+0000 7f9f19fff700 1 --2- 172.21.15.69:0/3685124082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f14100e20 0x7f9f14072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.710+0000 7f9f19fff700 1 -- 172.21.15.69:0/3685124082 >> 172.21.15.69:0/3685124082 conn(0x7f9f140fb210 msgr2=0x7f9f140fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:13.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.710+0000 7f9f19fff700 1 -- 172.21.15.69:0/3685124082 shutdown_connections 2023-07-06T20:19:13.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.710+0000 7f9f19fff700 1 -- 172.21.15.69:0/3685124082 wait complete. 2023-07-06T20:19:13.713 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2023-07-06T20:19:13.729 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:19:13.924 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.923+0000 7f32031ff700 1 -- 172.21.15.69:0/4123028584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31fc100e20 msgr2=0x7f31fc101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.923+0000 7f32031ff700 1 --2- 172.21.15.69:0/4123028584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31fc100e20 0x7f31fc101270 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009990 tx=0x7f31ec009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.923+0000 7f32031ff700 1 -- 172.21.15.69:0/4123028584 shutdown_connections 2023-07-06T20:19:13.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.923+0000 7f32031ff700 1 --2- 172.21.15.69:0/4123028584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31fc1017b0 0x7f31fc108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.923+0000 7f32031ff700 1 --2- 172.21.15.69:0/4123028584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31fc0ffc30 0x7f31fc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.923+0000 7f32031ff700 1 --2- 172.21.15.69:0/4123028584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31fc100e20 0x7f31fc101270 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.923+0000 7f32031ff700 1 -- 172.21.15.69:0/4123028584 >> 172.21.15.69:0/4123028584 conn(0x7f31fc0fb210 msgr2=0x7f31fc0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:13.926 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.923+0000 7f32031ff700 1 -- 172.21.15.69:0/4123028584 shutdown_connections 2023-07-06T20:19:13.926 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.923+0000 7f32031ff700 1 -- 172.21.15.69:0/4123028584 wait complete. 2023-07-06T20:19:13.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.924+0000 7f32031ff700 1 Processor -- start 2023-07-06T20:19:13.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.924+0000 7f32031ff700 1 -- start start 2023-07-06T20:19:13.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f32031ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31fc0ffc30 0x7f31fc197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:13.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f32031ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31fc100e20 0x7f31fc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:13.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f32031ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31fc1017b0 0x7f31fc19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:13.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f32031ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc19f6c0 con 0x7f31fc100e20 2023-07-06T20:19:13.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f32031ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc19f800 con 0x7f31fc1017b0 2023-07-06T20:19:13.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f32031ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc19f990 con 0x7f31fc0ffc30 2023-07-06T20:19:13.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f31f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31fc100e20 0x7f31fc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:13.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f3200f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31fc0ffc30 0x7f31fc197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:13.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f31f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31fc100e20 0x7f31fc198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53116/0 (socket says 172.21.15.69:53116) 2023-07-06T20:19:13.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f31f3fff700 1 -- 172.21.15.69:0/3361644603 learned_addr learned my addr 172.21.15.69:0/3361644603 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:13.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f320179c700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31fc1017b0 0x7f31fc19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:13.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f31f3fff700 1 -- 172.21.15.69:0/3361644603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31fc0ffc30 msgr2=0x7f31fc197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f31f3fff700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31fc0ffc30 0x7f31fc197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f31f3fff700 1 -- 172.21.15.69:0/3361644603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31fc1017b0 msgr2=0x7f31fc19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:13.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f31f3fff700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31fc1017b0 0x7f31fc19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:13.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.925+0000 7f31f3fff700 1 -- 172.21.15.69:0/3361644603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec0095f0 con 0x7f31fc100e20 2023-07-06T20:19:13.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.926+0000 7f31f3fff700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31fc100e20 0x7f31fc198080 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00b620 tx=0x7f31ec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:13.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.926+0000 7f31f1ffb700 1 -- 172.21.15.69:0/3361644603 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec01d070 con 0x7f31fc100e20 2023-07-06T20:19:13.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.926+0000 7f31f1ffb700 1 -- 172.21.15.69:0/3361644603 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31ec022470 con 0x7f31fc100e20 2023-07-06T20:19:13.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.926+0000 7f32031ff700 1 -- 172.21.15.69:0/3361644603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc19fc10 con 0x7f31fc100e20 2023-07-06T20:19:13.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.926+0000 7f31f1ffb700 1 -- 172.21.15.69:0/3361644603 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec017650 con 0x7f31fc100e20 2023-07-06T20:19:13.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.926+0000 7f32031ff700 1 -- 172.21.15.69:0/3361644603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc1a0030 con 0x7f31fc100e20 2023-07-06T20:19:13.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.927+0000 7f32031ff700 1 -- 172.21.15.69:0/3361644603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31fc10b450 con 0x7f31fc100e20 2023-07-06T20:19:13.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.929+0000 7f31f1ffb700 1 -- 172.21.15.69:0/3361644603 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f31ec0225e0 con 0x7f31fc100e20 2023-07-06T20:19:13.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.930+0000 7f31f1ffb700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f31e40a11a0 0x7f31e40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:13.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.930+0000 7f31f1ffb700 1 -- 172.21.15.69:0/3361644603 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f31ec0cde50 con 0x7f31fc100e20 2023-07-06T20:19:13.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.930+0000 7f3200f9b700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f31e40a11a0 0x7f31e40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:13.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.931+0000 7f3200f9b700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f31e40a11a0 0x7f31e40a3650 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f31fc100c80 tx=0x7f31f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:13.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:13.934+0000 7f31f1ffb700 1 -- 172.21.15.69:0/3361644603 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f31ec097600 con 0x7f31fc100e20 2023-07-06T20:19:14.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:13 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1298463356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:19:14.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:13 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1148764245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:19:14.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:13 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1298463356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:19:14.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:13 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1148764245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:19:14.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:13 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1298463356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:19:14.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:13 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1148764245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:19:14.135 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.134+0000 7f32031ff700 1 -- 172.21.15.69:0/3361644603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31fc1a05e0 con 0x7f31fc100e20 2023-07-06T20:19:14.136 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.136+0000 7f31f1ffb700 1 -- 172.21.15.69:0/3361644603 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7f31ec011110 con 0x7f31fc100e20 2023-07-06T20:19:14.137 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:14.137 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":87,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:18:59.638711+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:19:14.139 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.138+0000 7f32031ff700 1 -- 172.21.15.69:0/3361644603 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f31e40a11a0 msgr2=0x7f31e40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.139 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.138+0000 7f32031ff700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f31e40a11a0 0x7f31e40a3650 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f31fc100c80 tx=0x7f31f8009260 comp rx=0 tx=0).stop 2023-07-06T20:19:14.139 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.138+0000 7f32031ff700 1 -- 172.21.15.69:0/3361644603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31fc100e20 msgr2=0x7f31fc198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.139 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.138+0000 7f32031ff700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31fc100e20 0x7f31fc198080 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00b620 tx=0x7f31ec005740 comp rx=0 tx=0).stop 2023-07-06T20:19:14.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.138+0000 7f32031ff700 1 -- 172.21.15.69:0/3361644603 shutdown_connections 2023-07-06T20:19:14.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.138+0000 7f32031ff700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f31e40a11a0 0x7f31e40a3650 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.138+0000 7f32031ff700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31fc1017b0 0x7f31fc19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.138+0000 7f32031ff700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31fc0ffc30 0x7f31fc197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.138+0000 7f32031ff700 1 --2- 172.21.15.69:0/3361644603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31fc100e20 0x7f31fc198080 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.138+0000 7f32031ff700 1 -- 172.21.15.69:0/3361644603 >> 172.21.15.69:0/3361644603 conn(0x7f31fc0fb210 msgr2=0x7f31fc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.140 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.138+0000 7f32031ff700 1 -- 172.21.15.69:0/3361644603 shutdown_connections 2023-07-06T20:19:14.141 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.138+0000 7f32031ff700 1 -- 172.21.15.69:0/3361644603 wait complete. 2023-07-06T20:19:14.158 DEBUG:tasks.fwd_scrub:Yielding 2023-07-06T20:19:14.159 INFO:teuthology.run_tasks:Running task exec... 2023-07-06T20:19:14.169 INFO:teuthology.task.exec:Executing custom commands... 2023-07-06T20:19:14.170 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi069.front.sepia.ceph.com 2023-07-06T20:19:14.170 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-07-06T20:19:14.171 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-06T20:19:14.171 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-06T20:19:14.171 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-06T20:19:14.266 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:14.307 INFO:teuthology.orchestra.run.smithi069.stdout:Filesystem Size Used Avail Use% Mounted on 2023-07-06T20:19:14.307 INFO:teuthology.orchestra.run.smithi069.stdout:0@e3e8ec0c-1c38-11ee-9b2f-001a4aab830c.cephfs=/volumes/_nogroup/sv_1/439e7144-7b1d-4988-b061-fb50b19330d0 340G 0 340G 0% /tmp/cephfs 2023-07-06T20:19:14.312 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-07-06T20:19:14.382 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.381+0000 7fd45e1ff700 1 -- 172.21.15.69:0/3341798397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4580ffc20 msgr2=0x7fd458100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.382 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.381+0000 7fd45e1ff700 1 --2- 172.21.15.69:0/3341798397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4580ffc20 0x7fd458100030 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd448009990 tx=0x7fd448009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.382 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.381+0000 7fd45e1ff700 1 -- 172.21.15.69:0/3341798397 shutdown_connections 2023-07-06T20:19:14.383 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.381+0000 7fd45e1ff700 1 --2- 172.21.15.69:0/3341798397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4581017a0 0x7fd458108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.383 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.381+0000 7fd45e1ff700 1 --2- 172.21.15.69:0/3341798397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4580ffc20 0x7fd458100030 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.383 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.381+0000 7fd45e1ff700 1 --2- 172.21.15.69:0/3341798397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd458100e10 0x7fd458101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.383 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.381+0000 7fd45e1ff700 1 -- 172.21.15.69:0/3341798397 >> 172.21.15.69:0/3341798397 conn(0x7fd4580fb260 msgr2=0x7fd4580fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.383 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.381+0000 7fd45e1ff700 1 -- 172.21.15.69:0/3341798397 shutdown_connections 2023-07-06T20:19:14.383 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.381+0000 7fd45e1ff700 1 -- 172.21.15.69:0/3341798397 wait complete. 2023-07-06T20:19:14.385 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.382+0000 7fd45e1ff700 1 Processor -- start 2023-07-06T20:19:14.385 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.382+0000 7fd45e1ff700 1 -- start start 2023-07-06T20:19:14.385 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.382+0000 7fd45e1ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4580ffc20 0x7fd458197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.385 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.382+0000 7fd45e1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd458100e10 0x7fd4581980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.385 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.382+0000 7fd45e1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4581017a0 0x7fd45819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.385 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.382+0000 7fd45e1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819f710 con 0x7fd458100e10 2023-07-06T20:19:14.386 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.382+0000 7fd45e1ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819f850 con 0x7fd4580ffc20 2023-07-06T20:19:14.386 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.382+0000 7fd45e1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819f9e0 con 0x7fd4581017a0 2023-07-06T20:19:14.386 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.383+0000 7fd4577fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4580ffc20 0x7fd458197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.386 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.383+0000 7fd456ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd458100e10 0x7fd4581980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.386 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.383+0000 7fd4577fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4580ffc20 0x7fd458197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:47702/0 (socket says 172.21.15.69:47702) 2023-07-06T20:19:14.386 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.383+0000 7fd4577fe700 1 -- 172.21.15.69:0/304836875 learned_addr learned my addr 172.21.15.69:0/304836875 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:14.386 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.383+0000 7fd457fff700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4581017a0 0x7fd45819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.387 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.383+0000 7fd4577fe700 1 -- 172.21.15.69:0/304836875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4581017a0 msgr2=0x7fd45819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.387 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.383+0000 7fd4577fe700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4581017a0 0x7fd45819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.387 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.383+0000 7fd4577fe700 1 -- 172.21.15.69:0/304836875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd458100e10 msgr2=0x7fd4581980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.387 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.383+0000 7fd4577fe700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd458100e10 0x7fd4581980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.387 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.383+0000 7fd4577fe700 1 -- 172.21.15.69:0/304836875 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4480095f0 con 0x7fd4580ffc20 2023-07-06T20:19:14.387 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.383+0000 7fd4577fe700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4580ffc20 0x7fd458197b90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd4480098e0 tx=0x7fd448004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:14.387 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.384+0000 7fd454ff9700 1 -- 172.21.15.69:0/304836875 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44801d070 con 0x7fd4580ffc20 2023-07-06T20:19:14.387 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.384+0000 7fd45e1ff700 1 -- 172.21.15.69:0/304836875 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45819fc60 con 0x7fd4580ffc20 2023-07-06T20:19:14.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.384+0000 7fd45e1ff700 1 -- 172.21.15.69:0/304836875 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4581a0080 con 0x7fd4580ffc20 2023-07-06T20:19:14.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.384+0000 7fd454ff9700 1 -- 172.21.15.69:0/304836875 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd448005030 con 0x7fd4580ffc20 2023-07-06T20:19:14.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.384+0000 7fd454ff9700 1 -- 172.21.15.69:0/304836875 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448004b20 con 0x7fd4580ffc20 2023-07-06T20:19:14.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.387+0000 7fd454ff9700 1 -- 172.21.15.69:0/304836875 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd448005430 con 0x7fd4580ffc20 2023-07-06T20:19:14.389 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.388+0000 7fd471ba8b80 1 Processor -- start 2023-07-06T20:19:14.389 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd471ba8b80 1 -- start start 2023-07-06T20:19:14.390 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd454ff9700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd4400a1150 0x7fd4400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.390 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd471ba8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 0x55c75dbf93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd471ba8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf98f0 0x55c75dbf9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd471ba8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbfa410 0x55c75dbfa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd471ba8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75dafdd30 con 0x55c75dbfa410 2023-07-06T20:19:14.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd471ba8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75db50ac0 con 0x55c75dbf8fa0 2023-07-06T20:19:14.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd471ba8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75db49f50 con 0x55c75dbf98f0 2023-07-06T20:19:14.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd454ff9700 1 -- 172.21.15.69:0/304836875 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd4480cdc40 con 0x7fd4580ffc20 2023-07-06T20:19:14.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd4577fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 0x55c75dbf93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd4577fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 0x55c75dbf93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:47718/0 (socket says 172.21.15.69:47718) 2023-07-06T20:19:14.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd4577fe700 1 -- 172.21.15.69:0/436631998 learned_addr learned my addr 172.21.15.69:0/436631998 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:14.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd457fff700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd4400a1150 0x7fd4400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.389+0000 7fd456ffd700 1 --2- 172.21.15.69:0/436631998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf98f0 0x55c75dbf9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.393 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.390+0000 7fd457fff700 1 --2- 172.21.15.69:0/436631998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbfa410 0x55c75dbfa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.390+0000 7fd4577fe700 1 -- 172.21.15.69:0/436631998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf98f0 msgr2=0x55c75dbf9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.390+0000 7fd4577fe700 1 --2- 172.21.15.69:0/436631998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf98f0 0x55c75dbf9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.390+0000 7fd4577fe700 1 -- 172.21.15.69:0/436631998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbfa410 msgr2=0x55c75dbfa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.390+0000 7fd4577fe700 1 --2- 172.21.15.69:0/436631998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbfa410 0x55c75dbfa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.390+0000 7fd4577fe700 1 -- 172.21.15.69:0/436631998 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c75d977e60 con 0x55c75dbf8fa0 2023-07-06T20:19:14.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.390+0000 7fd456ffd700 1 --2- 172.21.15.69:0/436631998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf98f0 0x55c75dbf9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:14.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.390+0000 7fd4577fe700 1 --2- 172.21.15.69:0/436631998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 0x55c75dbf93b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd4480cb620 tx=0x7fd44809aae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d0c1f46999e2f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:14.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.390+0000 7fd457fff700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd4400a1150 0x7fd4400a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd4500078d0 tx=0x7fd45000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:14.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.390+0000 7fd43dffb700 1 -- 172.21.15.69:0/436631998 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480b6120 con 0x55c75dbf8fa0 2023-07-06T20:19:14.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.390+0000 7fd43dffb700 1 -- 172.21.15.69:0/436631998 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd44809f080 con 0x55c75dbf8fa0 2023-07-06T20:19:14.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.390+0000 7fd43dffb700 1 -- 172.21.15.69:0/436631998 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480b9200 con 0x55c75dbf8fa0 2023-07-06T20:19:14.396 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.391+0000 7fd471ba8b80 1 -- 172.21.15.69:0/436631998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 msgr2=0x55c75dbf93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.391+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/436631998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 0x55c75dbf93b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd4480cb620 tx=0x7fd44809aae0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.391+0000 7fd471ba8b80 1 -- 172.21.15.69:0/436631998 shutdown_connections 2023-07-06T20:19:14.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.391+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/436631998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbfa410 0x55c75dbfa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.391+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/436631998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 0x55c75dbf93b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.391+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/436631998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf98f0 0x55c75dbf9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.391+0000 7fd471ba8b80 1 -- 172.21.15.69:0/436631998 >> 172.21.15.69:0/436631998 conn(0x55c75dbee6c0 msgr2=0x55c75dbf3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.391+0000 7fd471ba8b80 1 -- 172.21.15.69:0/436631998 shutdown_connections 2023-07-06T20:19:14.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.391+0000 7fd471ba8b80 1 -- 172.21.15.69:0/436631998 wait complete. 2023-07-06T20:19:14.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.392+0000 7fd471ba8b80 1 -- start start 2023-07-06T20:19:14.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.392+0000 7fd471ba8b80 1 Processor -- start 2023-07-06T20:19:14.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd471ba8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 0x55c75dc02550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd471ba8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf98f0 0x55c75dc02a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.398 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd471ba8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbfa410 0x55c75dc09ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd471ba8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75dae9b40 con 0x55c75dbfa410 2023-07-06T20:19:14.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd471ba8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75dae9840 con 0x55c75dbf8fa0 2023-07-06T20:19:14.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd471ba8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75d9cd9a0 con 0x55c75dbf98f0 2023-07-06T20:19:14.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd471ba8b80 10 client.0 _ll_register_callbacks cb 0x55c75dae77e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:14.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd457fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbfa410 0x55c75dc09ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd4577fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 0x55c75dc02550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd456ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf98f0 0x55c75dc02a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.399 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd457fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbfa410 0x55c75dc09ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53142/0 (socket says 172.21.15.69:53142) 2023-07-06T20:19:14.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd457fff700 1 -- 172.21.15.69:0/2186654745 learned_addr learned my addr 172.21.15.69:0/2186654745 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:14.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd4577fe700 1 -- 172.21.15.69:0/2186654745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf98f0 msgr2=0x55c75dc02a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd4577fe700 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf98f0 0x55c75dc02a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd4577fe700 1 -- 172.21.15.69:0/2186654745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbfa410 msgr2=0x55c75dc09ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd4577fe700 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbfa410 0x55c75dc09ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.400 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd4577fe700 1 -- 172.21.15.69:0/2186654745 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c0098e0 con 0x55c75dbf8fa0 2023-07-06T20:19:14.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd457fff700 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbfa410 0x55c75dc09ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:14.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.393+0000 7fd456ffd700 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf98f0 0x55c75dc02a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:14.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.394+0000 7fd4577fe700 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 0x55c75dc02550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd448097ac0 tx=0x7fd4480b61a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d73934057c2b4758 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:14.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.394+0000 7fd436ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:19:14.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.394+0000 7fd436ffd700 1 -- 172.21.15.69:0/2186654745 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44809b4b0 con 0x55c75dbf8fa0 2023-07-06T20:19:14.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.394+0000 7fd436ffd700 1 -- 172.21.15.69:0/2186654745 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd44809a540 con 0x55c75dbf8fa0 2023-07-06T20:19:14.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.394+0000 7fd471ba8b80 1 -- 172.21.15.69:0/2186654745 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c75daa3570 con 0x55c75dbf8fa0 2023-07-06T20:19:14.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.394+0000 7fd436ffd700 1 -- 172.21.15.69:0/2186654745 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4480b93e0 con 0x55c75dbf8fa0 2023-07-06T20:19:14.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.394+0000 7fd471ba8b80 10 client.25307 fetch_fsmap learned FSMap version 23 2023-07-06T20:19:14.402 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.394+0000 7fd471ba8b80 1 -- 172.21.15.69:0/2186654745 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fd4581a0080 con 0x55c75dbf8fa0 2023-07-06T20:19:14.403 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.394+0000 7fd436ffd700 1 -- 172.21.15.69:0/2186654745 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c75daa3570 con 0x55c75dbf8fa0 2023-07-06T20:19:14.403 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.395+0000 7fd436ffd700 1 -- 172.21.15.69:0/2186654745 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd4480b7020 con 0x55c75dbf8fa0 2023-07-06T20:19:14.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.395+0000 7fd471ba8b80 10 client.25307 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:19:14.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.395+0000 7fd471ba8b80 10 client.25307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:19:14.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.395+0000 7fd471ba8b80 20 client.25307 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:14.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.395+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c75dc0de50 0x55c75dc10260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.395+0000 7fd471ba8b80 4 client.25307 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:19:14.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.395+0000 7fd471ba8b80 1 -- 172.21.15.69:0/2186654745 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c75dbc7de0 con 0x55c75dc0de50 2023-07-06T20:19:14.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.395+0000 7fd456ffd700 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c75dc0de50 0x55c75dc10260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.396+0000 7fd456ffd700 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c75dc0de50 0x55c75dc10260 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0516fb923d00e9 server_cookie=cf037e5fa0b0caf0 in_seq=0 out_seq=0 2023-07-06T20:19:14.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.396+0000 7fd436ffd700 10 client.25307 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:14.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.396+0000 7fd436ffd700 1 -- 172.21.15.69:0/2186654745 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c75dbc7de0 con 0x55c75dc0de50 2023-07-06T20:19:14.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.396+0000 7fd436ffd700 10 client.25307 handle_command_reply: tid=0 2023-07-06T20:19:14.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.396+0000 7fd471ba8b80 1 client.25307 shutdown 2023-07-06T20:19:14.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.396+0000 7fd471ba8b80 1 -- 172.21.15.69:0/2186654745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 msgr2=0x55c75dc02550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.396+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 0x55c75dc02550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd448097ac0 tx=0x7fd4480b61a0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.397+0000 7fd471ba8b80 1 -- 172.21.15.69:0/2186654745 shutdown_connections 2023-07-06T20:19:14.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.397+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c75dc0de50 0x55c75dc10260 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.397+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbfa410 0x55c75dc09ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.397+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf8fa0 0x55c75dc02550 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.397+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/2186654745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf98f0 0x55c75dc02a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.397+0000 7fd436ffd700 0 client.25307 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:14.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.397+0000 7fd471ba8b80 1 -- 172.21.15.69:0/2186654745 >> 172.21.15.69:0/2186654745 conn(0x55c75dbee6c0 msgr2=0x55c75dbf3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.397+0000 7fd471ba8b80 1 -- 172.21.15.69:0/2186654745 shutdown_connections 2023-07-06T20:19:14.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.397+0000 7fd471ba8b80 1 -- 172.21.15.69:0/2186654745 wait complete. 2023-07-06T20:19:14.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.397+0000 7fd471ba8b80 20 client.25307 trim_cache size 0 max 16384 2023-07-06T20:19:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.415+0000 7fd471ba8b80 1 Processor -- start 2023-07-06T20:19:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd471ba8b80 1 -- start start 2023-07-06T20:19:14.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd471ba8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbee6c0 0x55c75dc00810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd471ba8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbf8fa0 0x55c75dc09a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd471ba8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf98f0 0x55c75dbfd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd471ba8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75db49f50 con 0x55c75dbf8fa0 2023-07-06T20:19:14.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd471ba8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75dafdd30 con 0x55c75dbf98f0 2023-07-06T20:19:14.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd471ba8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75dbf4e90 con 0x55c75dbee6c0 2023-07-06T20:19:14.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd4577fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbee6c0 0x55c75dc00810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd4577fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbee6c0 0x55c75dc00810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:36622/0 (socket says 172.21.15.69:36622) 2023-07-06T20:19:14.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd4577fe700 1 -- 172.21.15.69:0/3369337425 learned_addr learned my addr 172.21.15.69:0/3369337425 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:14.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd457fff700 1 --2- 172.21.15.69:0/3369337425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf98f0 0x55c75dbfd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd456ffd700 1 --2- 172.21.15.69:0/3369337425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbf8fa0 0x55c75dc09a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd4577fe700 1 -- 172.21.15.69:0/3369337425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf98f0 msgr2=0x55c75dbfd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.416+0000 7fd4577fe700 1 --2- 172.21.15.69:0/3369337425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf98f0 0x55c75dbfd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.417+0000 7fd4577fe700 1 -- 172.21.15.69:0/3369337425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbf8fa0 msgr2=0x55c75dc09a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.417+0000 7fd4577fe700 1 --2- 172.21.15.69:0/3369337425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbf8fa0 0x55c75dc09a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.420 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.417+0000 7fd4577fe700 1 -- 172.21.15.69:0/3369337425 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c75dbefe60 con 0x55c75dbee6c0 2023-07-06T20:19:14.420 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.417+0000 7fd4577fe700 1 --2- 172.21.15.69:0/3369337425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbee6c0 0x55c75dc00810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd4480095c0 tx=0x7fd44809aae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a860a31d9f69f8ee server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.417+0000 7fd43e7fc700 1 -- 172.21.15.69:0/3369337425 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480b93e0 con 0x55c75dbee6c0 2023-07-06T20:19:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.417+0000 7fd43e7fc700 1 -- 172.21.15.69:0/3369337425 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd448004e90 con 0x55c75dbee6c0 2023-07-06T20:19:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.417+0000 7fd43e7fc700 1 -- 172.21.15.69:0/3369337425 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44809b510 con 0x55c75dbee6c0 2023-07-06T20:19:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.418+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3369337425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbee6c0 msgr2=0x55c75dc00810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.418+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/3369337425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbee6c0 0x55c75dc00810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd4480095c0 tx=0x7fd44809aae0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.418+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3369337425 shutdown_connections 2023-07-06T20:19:14.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.418+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/3369337425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbf98f0 0x55c75dbfd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.418+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/3369337425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbee6c0 0x55c75dc00810 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.418+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/3369337425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbf8fa0 0x55c75dc09a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.418+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3369337425 >> 172.21.15.69:0/3369337425 conn(0x55c75dbf4980 msgr2=0x55c75dc0a300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.418+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3369337425 shutdown_connections 2023-07-06T20:19:14.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.418+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3369337425 wait complete. 2023-07-06T20:19:14.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.418+0000 7fd471ba8b80 1 -- start start 2023-07-06T20:19:14.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd471ba8b80 1 Processor -- start 2023-07-06T20:19:14.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd471ba8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbee6c0 0x55c75dc09a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd471ba8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf8fa0 0x55c75dc06570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd471ba8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbf98f0 0x55c75dc06b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd471ba8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75dc61cc0 con 0x55c75dbf98f0 2023-07-06T20:19:14.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd471ba8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75dc61e00 con 0x55c75dbee6c0 2023-07-06T20:19:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd471ba8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c75dc61f70 con 0x55c75dbf8fa0 2023-07-06T20:19:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd471ba8b80 10 client.0 _ll_register_callbacks cb 0x7fd44c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd4577fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbee6c0 0x55c75dc09a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd4577fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbee6c0 0x55c75dc09a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:47744/0 (socket says 172.21.15.69:47744) 2023-07-06T20:19:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd4577fe700 1 -- 172.21.15.69:0/3607499803 learned_addr learned my addr 172.21.15.69:0/3607499803 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd457fff700 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbf98f0 0x55c75dc06b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.419+0000 7fd456ffd700 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf8fa0 0x55c75dc06570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd4577fe700 1 -- 172.21.15.69:0/3607499803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf8fa0 msgr2=0x55c75dc06570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd4577fe700 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf8fa0 0x55c75dc06570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd4577fe700 1 -- 172.21.15.69:0/3607499803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbf98f0 msgr2=0x55c75dc06b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd4577fe700 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbf98f0 0x55c75dc06b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd4577fe700 1 -- 172.21.15.69:0/3607499803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4581a0080 con 0x55c75dbee6c0 2023-07-06T20:19:14.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd4577fe700 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbee6c0 0x55c75dc09a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd448097500 tx=0x7fd44809b4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5e3c05a5f47558c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:14.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd436ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:19:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd436ffd700 1 -- 172.21.15.69:0/3607499803 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480b93e0 con 0x55c75dbee6c0 2023-07-06T20:19:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd436ffd700 1 -- 172.21.15.69:0/3607499803 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd4480a3410 con 0x55c75dbee6c0 2023-07-06T20:19:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3607499803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd44809f080 con 0x55c75dbee6c0 2023-07-06T20:19:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd436ffd700 1 -- 172.21.15.69:0/3607499803 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4480b3070 con 0x55c75dbee6c0 2023-07-06T20:19:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd471ba8b80 10 client.25313 fetch_fsmap learned FSMap version 23 2023-07-06T20:19:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3607499803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c75d960ce0 con 0x55c75dbee6c0 2023-07-06T20:19:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.420+0000 7fd436ffd700 1 -- 172.21.15.69:0/3607499803 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4480c9020 con 0x55c75dbee6c0 2023-07-06T20:19:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.421+0000 7fd436ffd700 1 -- 172.21.15.69:0/3607499803 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd44800fad0 con 0x55c75dbee6c0 2023-07-06T20:19:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.421+0000 7fd471ba8b80 10 client.25313 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:19:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.421+0000 7fd471ba8b80 10 client.25313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:19:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.421+0000 7fd471ba8b80 20 client.25313 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.421+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c75dbfa410 0x55c75dc652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.421+0000 7fd471ba8b80 4 client.25313 mds_command: new command op to 24584 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-06T20:19:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.421+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3607499803 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c75db42f30 con 0x55c75dbfa410 2023-07-06T20:19:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.421+0000 7fd456ffd700 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c75dbfa410 0x55c75dc652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.421+0000 7fd456ffd700 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c75dbfa410 0x55c75dc652e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf12fc0e92948c8 server_cookie=b0a6c9c9697b3323 in_seq=0 out_seq=0 2023-07-06T20:19:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.421+0000 7fd436ffd700 10 client.25313 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.422+0000 7fd436ffd700 1 -- 172.21.15.69:0/3607499803 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c75db42f30 con 0x55c75dbfa410 2023-07-06T20:19:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd436ffd700 10 client.25313 handle_command_reply: tid=0 2023-07-06T20:19:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd471ba8b80 1 client.25313 shutdown 2023-07-06T20:19:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3607499803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbee6c0 msgr2=0x55c75dc09a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbee6c0 0x55c75dc09a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd448097500 tx=0x7fd44809b4e0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3607499803 shutdown_connections 2023-07-06T20:19:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c75dbfa410 0x55c75dc652e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c75dbf98f0 0x55c75dc06b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd436ffd700 0 client.25313 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c75dbee6c0 0x55c75dc09a60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd471ba8b80 1 --2- 172.21.15.69:0/3607499803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c75dbf8fa0 0x55c75dc06570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3607499803 >> 172.21.15.69:0/3607499803 conn(0x55c75dbf4980 msgr2=0x55c75dc0a300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3607499803 shutdown_connections 2023-07-06T20:19:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd471ba8b80 1 -- 172.21.15.69:0/3607499803 wait complete. 2023-07-06T20:19:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.423+0000 7fd471ba8b80 20 client.25313 trim_cache size 0 max 16384 2023-07-06T20:19:14.432 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:19:14.432 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2023-07-06T20:19:14.432 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "e38fc1e2-d0e3-4033-a66d-d56d28cb1270", 2023-07-06T20:19:14.433 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2023-07-06T20:19:14.433 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:19:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.424+0000 7fd45e1ff700 1 -- 172.21.15.69:0/304836875 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd4400a1150 msgr2=0x7fd4400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.424+0000 7fd45e1ff700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd4400a1150 0x7fd4400a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd4500078d0 tx=0x7fd45000b040 comp rx=0 tx=0).stop 2023-07-06T20:19:14.434 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.424+0000 7fd45e1ff700 1 -- 172.21.15.69:0/304836875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4580ffc20 msgr2=0x7fd458197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.434 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.424+0000 7fd45e1ff700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4580ffc20 0x7fd458197b90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd4480098e0 tx=0x7fd448004730 comp rx=0 tx=0).stop 2023-07-06T20:19:14.434 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.424+0000 7fd45e1ff700 1 -- 172.21.15.69:0/304836875 shutdown_connections 2023-07-06T20:19:14.434 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.424+0000 7fd45e1ff700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd4400a1150 0x7fd4400a3600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.434 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.424+0000 7fd45e1ff700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4581017a0 0x7fd45819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.434 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.424+0000 7fd45e1ff700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4580ffc20 0x7fd458197b90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.434 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.424+0000 7fd45e1ff700 1 --2- 172.21.15.69:0/304836875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd458100e10 0x7fd4581980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.435 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.424+0000 7fd45e1ff700 1 -- 172.21.15.69:0/304836875 >> 172.21.15.69:0/304836875 conn(0x7fd4580fb260 msgr2=0x7fd458106710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.435 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.424+0000 7fd45e1ff700 1 -- 172.21.15.69:0/304836875 shutdown_connections 2023-07-06T20:19:14.435 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.424+0000 7fd45e1ff700 1 -- 172.21.15.69:0/304836875 wait complete. 2023-07-06T20:19:14.444 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:19:14.449 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-07-06T20:19:14.591 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-07-06T20:19:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.651+0000 7f96179ff700 1 -- 172.21.15.69:0/2802149715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610107ab0 msgr2=0x7f9610109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.651+0000 7f96179ff700 1 --2- 172.21.15.69:0/2802149715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610107ab0 0x7f9610109f30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9608009f90 tx=0x7f960800b330 comp rx=0 tx=0).stop 2023-07-06T20:19:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.651+0000 7f96179ff700 1 -- 172.21.15.69:0/2802149715 shutdown_connections 2023-07-06T20:19:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.651+0000 7f96179ff700 1 --2- 172.21.15.69:0/2802149715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610107ab0 0x7f9610109f30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.651+0000 7f96179ff700 1 --2- 172.21.15.69:0/2802149715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610074620 0x7f9610072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.651+0000 7f96179ff700 1 --2- 172.21.15.69:0/2802149715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610073040 0x7f96100734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.651+0000 7f96179ff700 1 -- 172.21.15.69:0/2802149715 >> 172.21.15.69:0/2802149715 conn(0x7f96100fb260 msgr2=0x7f96100fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.651+0000 7f96179ff700 1 -- 172.21.15.69:0/2802149715 shutdown_connections 2023-07-06T20:19:14.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.651+0000 7f96179ff700 1 -- 172.21.15.69:0/2802149715 wait complete. 2023-07-06T20:19:14.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.652+0000 7f96179ff700 1 Processor -- start 2023-07-06T20:19:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.652+0000 7f96179ff700 1 -- start start 2023-07-06T20:19:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f96179ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610073040 0x7f9610197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f96179ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610074620 0x7f9610198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f96179ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610107ab0 0x7f961019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f96179ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961019f9d0 con 0x7f9610073040 2023-07-06T20:19:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f96179ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961019fb10 con 0x7f9610107ab0 2023-07-06T20:19:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f96179ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961019fca0 con 0x7f9610074620 2023-07-06T20:19:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f961579b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610073040 0x7f9610197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f961579b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610073040 0x7f9610197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53178/0 (socket says 172.21.15.69:53178) 2023-07-06T20:19:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f961579b700 1 -- 172.21.15.69:0/2606421145 learned_addr learned my addr 172.21.15.69:0/2606421145 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f9615f9c700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610107ab0 0x7f961019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f9614f9a700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610074620 0x7f9610198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.658 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f961579b700 1 -- 172.21.15.69:0/2606421145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610074620 msgr2=0x7f9610198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.658 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f961579b700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610074620 0x7f9610198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.658 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f961579b700 1 -- 172.21.15.69:0/2606421145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610107ab0 msgr2=0x7f961019f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.658 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.653+0000 7f961579b700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610107ab0 0x7f961019f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.659 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.654+0000 7f961579b700 1 -- 172.21.15.69:0/2606421145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9608009c40 con 0x7f9610073040 2023-07-06T20:19:14.659 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.654+0000 7f9615f9c700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610107ab0 0x7f961019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:14.659 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.654+0000 7f961579b700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610073040 0x7f9610197e50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f960c00e9c0 tx=0x7f960c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:14.659 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.654+0000 7f96067fc700 1 -- 172.21.15.69:0/2606421145 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c004a30 con 0x7f9610073040 2023-07-06T20:19:14.660 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.654+0000 7f96067fc700 1 -- 172.21.15.69:0/2606421145 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f960c00cc80 con 0x7f9610073040 2023-07-06T20:19:14.660 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.654+0000 7f96179ff700 1 -- 172.21.15.69:0/2606421145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961019ff80 con 0x7f9610073040 2023-07-06T20:19:14.660 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.654+0000 7f96067fc700 1 -- 172.21.15.69:0/2606421145 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0054e0 con 0x7f9610073040 2023-07-06T20:19:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.654+0000 7f96179ff700 1 -- 172.21.15.69:0/2606421145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96101a03b0 con 0x7f9610073040 2023-07-06T20:19:14.662 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.657+0000 7f96067fc700 1 -- 172.21.15.69:0/2606421145 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f960c004b90 con 0x7f9610073040 2023-07-06T20:19:14.663 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.658+0000 7f962b2cbb80 1 Processor -- start 2023-07-06T20:19:14.663 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.659+0000 7f962b2cbb80 1 -- start start 2023-07-06T20:19:14.663 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.659+0000 7f962b2cbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f63c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.663 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.659+0000 7f962b2cbb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 0x56244f63ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.663 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.659+0000 7f962b2cbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63d390 0x56244f63d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.659+0000 7f962b2cbb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244f540cb0 con 0x56244f63c870 2023-07-06T20:19:14.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.659+0000 7f962b2cbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244f593a40 con 0x56244f63d390 2023-07-06T20:19:14.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.659+0000 7f962b2cbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244f58ced0 con 0x56244f63bf20 2023-07-06T20:19:14.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.659+0000 7f96067fc700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f95fc0a11d0 0x7f95fc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.659+0000 7f96067fc700 1 -- 172.21.15.69:0/2606421145 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f960c014070 con 0x7f9610073040 2023-07-06T20:19:14.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.659+0000 7f961579b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 0x56244f63ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f961579b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 0x56244f63ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53194/0 (socket says 172.21.15.69:53194) 2023-07-06T20:19:14.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f961579b700 1 -- 172.21.15.69:0/1804211172 learned_addr learned my addr 172.21.15.69:0/1804211172 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:14.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f9615f9c700 1 --2- 172.21.15.69:0/1804211172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f63c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f9614f9a700 1 --2- 172.21.15.69:0/1804211172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63d390 0x56244f63d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f9615f9c700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f95fc0a11d0 0x7f95fc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f961579b700 1 -- 172.21.15.69:0/1804211172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 msgr2=0x56244f63c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f961579b700 1 --2- 172.21.15.69:0/1804211172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f63c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f961579b700 1 -- 172.21.15.69:0/1804211172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63d390 msgr2=0x56244f63d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f961579b700 1 --2- 172.21.15.69:0/1804211172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63d390 0x56244f63d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f961579b700 1 -- 172.21.15.69:0/1804211172 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56244f3bade0 con 0x56244f63c870 2023-07-06T20:19:14.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f961579b700 1 --2- 172.21.15.69:0/1804211172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 0x56244f63ce00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f960c00cfd0 tx=0x7f960c099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77d32e23b3db871 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:14.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f9615f9c700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f95fc0a11d0 0x7f95fc0a3680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9608007900 tx=0x7f96080096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:14.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.660+0000 7f95fb7fe700 1 -- 172.21.15.69:0/1804211172 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c099d80 con 0x56244f63c870 2023-07-06T20:19:14.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.661+0000 7f95fb7fe700 1 -- 172.21.15.69:0/1804211172 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f960c09ddb0 con 0x56244f63c870 2023-07-06T20:19:14.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.661+0000 7f95fb7fe700 1 -- 172.21.15.69:0/1804211172 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c099fe0 con 0x56244f63c870 2023-07-06T20:19:14.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.661+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1804211172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 msgr2=0x56244f63ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.661+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/1804211172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 0x56244f63ce00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f960c00cfd0 tx=0x7f960c099bd0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.662+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1804211172 shutdown_connections 2023-07-06T20:19:14.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.662+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/1804211172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63d390 0x56244f63d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.662+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/1804211172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f63c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.662+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/1804211172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 0x56244f63ce00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.662+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1804211172 >> 172.21.15.69:0/1804211172 conn(0x56244f631640 msgr2=0x56244f6369f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.662+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1804211172 shutdown_connections 2023-07-06T20:19:14.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.662+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1804211172 wait complete. 2023-07-06T20:19:14.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.662+0000 7f962b2cbb80 1 -- start start 2023-07-06T20:19:14.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.663+0000 7f962b2cbb80 1 Processor -- start 2023-07-06T20:19:14.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.663+0000 7f962b2cbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f645880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.663+0000 7f962b2cbb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 0x56244f645dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.663+0000 7f962b2cbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63d390 0x56244f64ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.663+0000 7f962b2cbb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244f52cac0 con 0x56244f63c870 2023-07-06T20:19:14.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.663+0000 7f962b2cbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244f52c7c0 con 0x56244f63d390 2023-07-06T20:19:14.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.663+0000 7f962b2cbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244f410920 con 0x56244f63bf20 2023-07-06T20:19:14.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f962b2cbb80 10 client.0 _ll_register_callbacks cb 0x56244f52a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:14.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f9614f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 0x56244f645dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f9614f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 0x56244f645dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53210/0 (socket says 172.21.15.69:53210) 2023-07-06T20:19:14.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f9614f9a700 1 -- 172.21.15.69:0/1369453026 learned_addr learned my addr 172.21.15.69:0/1369453026 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:14.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f961579b700 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f645880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f9615f9c700 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63d390 0x56244f64ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f9614f9a700 1 -- 172.21.15.69:0/1369453026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 msgr2=0x56244f645880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f9614f9a700 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f645880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f9614f9a700 1 -- 172.21.15.69:0/1369453026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63d390 msgr2=0x56244f64ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f9614f9a700 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63d390 0x56244f64ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f9614f9a700 1 -- 172.21.15.69:0/1369453026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96101a03b0 con 0x56244f63c870 2023-07-06T20:19:14.676 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f9615f9c700 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63d390 0x56244f64ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:19:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f9614f9a700 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 0x56244f645dc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9600011830 tx=0x7f9600011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ca7fc5f5ebfcbc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.664+0000 7f95f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:19:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.665+0000 7f95f8ff9700 1 -- 172.21.15.69:0/1369453026 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600016070 con 0x56244f63c870 2023-07-06T20:19:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.665+0000 7f95f8ff9700 1 -- 172.21.15.69:0/1369453026 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f960000f890 con 0x56244f63c870 2023-07-06T20:19:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.665+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1369453026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56244f4e64f0 con 0x56244f63c870 2023-07-06T20:19:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.665+0000 7f95f8ff9700 1 -- 172.21.15.69:0/1369453026 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f960000fa50 con 0x56244f63c870 2023-07-06T20:19:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.665+0000 7f962b2cbb80 10 client.15276 fetch_fsmap learned FSMap version 23 2023-07-06T20:19:14.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.665+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1369453026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56244f3a3c60 con 0x56244f63c870 2023-07-06T20:19:14.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.665+0000 7f95f8ff9700 1 -- 172.21.15.69:0/1369453026 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9600029020 con 0x56244f63c870 2023-07-06T20:19:14.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.665+0000 7f95f8ff9700 1 -- 172.21.15.69:0/1369453026 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9600025cc0 con 0x56244f63c870 2023-07-06T20:19:14.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.666+0000 7f962b2cbb80 10 client.15276 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:19:14.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.666+0000 7f962b2cbb80 10 client.15276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:19:14.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.666+0000 7f962b2cbb80 20 client.15276 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:14.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.666+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56244f651760 0x56244f653b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.666+0000 7f962b2cbb80 4 client.15276 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:19:14.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.666+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1369453026 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56244f60ad60 con 0x56244f651760 2023-07-06T20:19:14.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.666+0000 7f961579b700 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56244f651760 0x56244f653b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.667+0000 7f961579b700 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56244f651760 0x56244f653b70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58408c6f8a8707da server_cookie=83671927bdaece3f in_seq=0 out_seq=0 2023-07-06T20:19:14.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.667+0000 7f95f8ff9700 10 client.15276 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:14.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.667+0000 7f95f8ff9700 1 -- 172.21.15.69:0/1369453026 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56244f60ad60 con 0x56244f651760 2023-07-06T20:19:14.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.667+0000 7f95f8ff9700 10 client.15276 handle_command_reply: tid=0 2023-07-06T20:19:14.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.667+0000 7f962b2cbb80 1 client.15276 shutdown 2023-07-06T20:19:14.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.668+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1369453026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 msgr2=0x56244f645dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.668+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 0x56244f645dc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9600011830 tx=0x7f9600011b40 comp rx=0 tx=0).stop 2023-07-06T20:19:14.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.668+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1369453026 shutdown_connections 2023-07-06T20:19:14.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.668+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56244f651760 0x56244f653b70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.668+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63d390 0x56244f64ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.668+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f645880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.668+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/1369453026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63c870 0x56244f645dc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.668+0000 7f95f8ff9700 0 client.15276 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:14.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.668+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1369453026 >> 172.21.15.69:0/1369453026 conn(0x56244f631640 msgr2=0x56244f6360b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.668+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1369453026 shutdown_connections 2023-07-06T20:19:14.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.668+0000 7f962b2cbb80 1 -- 172.21.15.69:0/1369453026 wait complete. 2023-07-06T20:19:14.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.668+0000 7f962b2cbb80 20 client.15276 trim_cache size 0 max 16384 2023-07-06T20:19:14.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.692+0000 7f962b2cbb80 1 Processor -- start 2023-07-06T20:19:14.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.692+0000 7f962b2cbb80 1 -- start start 2023-07-06T20:19:14.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.692+0000 7f962b2cbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f6445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.692+0000 7f962b2cbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63c870 0x56244f64cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.692+0000 7f962b2cbb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 0x56244f64d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.692+0000 7f962b2cbb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244f6384a0 con 0x56244f63d390 2023-07-06T20:19:14.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.692+0000 7f962b2cbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244f643540 con 0x56244f63c870 2023-07-06T20:19:14.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.692+0000 7f962b2cbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244f6442d0 con 0x56244f63bf20 2023-07-06T20:19:14.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f9615f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 0x56244f64d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f9614f9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63c870 0x56244f64cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f961579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f6445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f9615f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 0x56244f64d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53216/0 (socket says 172.21.15.69:53216) 2023-07-06T20:19:14.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f9614f9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63c870 0x56244f64cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:47776/0 (socket says 172.21.15.69:47776) 2023-07-06T20:19:14.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f9615f9c700 1 -- 172.21.15.69:0/3548622153 learned_addr learned my addr 172.21.15.69:0/3548622153 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:14.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f961579b700 1 -- 172.21.15.69:0/3548622153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63c870 msgr2=0x56244f64cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f961579b700 1 --2- 172.21.15.69:0/3548622153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63c870 0x56244f64cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f961579b700 1 -- 172.21.15.69:0/3548622153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 msgr2=0x56244f64d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f961579b700 1 --2- 172.21.15.69:0/3548622153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 0x56244f64d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f961579b700 1 -- 172.21.15.69:0/3548622153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56244f3e1250 con 0x56244f63bf20 2023-07-06T20:19:14.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f9614f9a700 1 --2- 172.21.15.69:0/3548622153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63c870 0x56244f64cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:14.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f9615f9c700 1 --2- 172.21.15.69:0/3548622153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 0x56244f64d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:14.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f961579b700 1 --2- 172.21.15.69:0/3548622153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f6445a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f960c096280 tx=0x7f960c004f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d36c1ed1f109c80f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:14.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f95fbfff700 1 -- 172.21.15.69:0/3548622153 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0b6070 con 0x56244f63bf20 2023-07-06T20:19:14.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f95fbfff700 1 -- 172.21.15.69:0/3548622153 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f960c0c8070 con 0x56244f63bf20 2023-07-06T20:19:14.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.693+0000 7f95fbfff700 1 -- 172.21.15.69:0/3548622153 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0981b0 con 0x56244f63bf20 2023-07-06T20:19:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.694+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3548622153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 msgr2=0x56244f6445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.694+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/3548622153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f6445a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f960c096280 tx=0x7f960c004f30 comp rx=0 tx=0).stop 2023-07-06T20:19:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.694+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3548622153 shutdown_connections 2023-07-06T20:19:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.694+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/3548622153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 0x56244f64d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.694+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/3548622153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63bf20 0x56244f6445a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.694+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/3548622153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63c870 0x56244f64cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.694+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3548622153 >> 172.21.15.69:0/3548622153 conn(0x56244f631640 msgr2=0x56244f6584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.694+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3548622153 shutdown_connections 2023-07-06T20:19:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.694+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3548622153 wait complete. 2023-07-06T20:19:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f962b2cbb80 1 -- start start 2023-07-06T20:19:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f962b2cbb80 1 Processor -- start 2023-07-06T20:19:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f962b2cbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63bf20 0x56244f648e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f962b2cbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63c870 0x56244f649340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f962b2cbb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 0x56244f64c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f962b2cbb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244f64c8a0 con 0x56244f63d390 2023-07-06T20:19:14.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f962b2cbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244f64ca10 con 0x56244f63bf20 2023-07-06T20:19:14.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f962b2cbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244f64cbd0 con 0x56244f63c870 2023-07-06T20:19:14.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f962b2cbb80 10 client.0 _ll_register_callbacks cb 0x7f9600009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:14.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f9615f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 0x56244f64c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f961579b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63bf20 0x56244f648e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f9615f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 0x56244f64c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53226/0 (socket says 172.21.15.69:53226) 2023-07-06T20:19:14.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.695+0000 7f9615f9c700 1 -- 172.21.15.69:0/3768376081 learned_addr learned my addr 172.21.15.69:0/3768376081 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:14.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.696+0000 7f9614f9a700 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63c870 0x56244f649340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.696+0000 7f961579b700 1 -- 172.21.15.69:0/3768376081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63c870 msgr2=0x56244f649340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.696+0000 7f961579b700 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63c870 0x56244f649340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.696+0000 7f961579b700 1 -- 172.21.15.69:0/3768376081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 msgr2=0x56244f64c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:14.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.696+0000 7f961579b700 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 0x56244f64c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.696+0000 7f961579b700 1 -- 172.21.15.69:0/3768376081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56244f3a3c60 con 0x56244f63bf20 2023-07-06T20:19:14.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.696+0000 7f9615f9c700 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 0x56244f64c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.696+0000 7f961579b700 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63bf20 0x56244f648e00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x56244f5fd220 tx=0x7f960c00ceb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa102721ecb47176 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.696+0000 7f95f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:19:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.696+0000 7f95f8ff9700 1 -- 172.21.15.69:0/3768376081 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0b6070 con 0x56244f63bf20 2023-07-06T20:19:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.696+0000 7f95f8ff9700 1 -- 172.21.15.69:0/3768376081 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f960c099d80 con 0x56244f63bf20 2023-07-06T20:19:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.696+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3768376081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f960c09ddb0 con 0x56244f63bf20 2023-07-06T20:19:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.697+0000 7f95f8ff9700 1 -- 172.21.15.69:0/3768376081 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f960c0ca580 con 0x56244f63bf20 2023-07-06T20:19:14.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.697+0000 7f962b2cbb80 10 client.25331 fetch_fsmap learned FSMap version 23 2023-07-06T20:19:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.697+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3768376081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f961010ab50 con 0x56244f63bf20 2023-07-06T20:19:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.697+0000 7f95f8ff9700 1 -- 172.21.15.69:0/3768376081 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f960c09ddb0 con 0x56244f63bf20 2023-07-06T20:19:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.697+0000 7f95f8ff9700 1 -- 172.21.15.69:0/3768376081 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f960c099fe0 con 0x56244f63bf20 2023-07-06T20:19:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.697+0000 7f962b2cbb80 10 client.25331 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T20:19:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.697+0000 7f962b2cbb80 10 client.25331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:19:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.697+0000 7f962b2cbb80 20 client.25331 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.697+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56244f6a63d0 0x56244f6a87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:14.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.697+0000 7f962b2cbb80 4 client.25331 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:19:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.697+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3768376081 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56244f585eb0 con 0x56244f6a63d0 2023-07-06T20:19:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.697+0000 7f9614f9a700 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56244f6a63d0 0x56244f6a87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f9614f9a700 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56244f6a63d0 0x56244f6a87b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be77054b761ffae2 server_cookie=a8541b26323704d6 in_seq=0 out_seq=0 2023-07-06T20:19:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f95f8ff9700 10 client.25331 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f95f8ff9700 1 -- 172.21.15.69:0/3768376081 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56244f585eb0 con 0x56244f6a63d0 2023-07-06T20:19:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f95f8ff9700 10 client.25331 handle_command_reply: tid=0 2023-07-06T20:19:14.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f962b2cbb80 1 client.25331 shutdown 2023-07-06T20:19:14.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3768376081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63bf20 msgr2=0x56244f648e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63bf20 0x56244f648e00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x56244f5fd220 tx=0x7f960c00ceb0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3768376081 shutdown_connections 2023-07-06T20:19:14.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56244f6a63d0 0x56244f6a87b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56244f63d390 0x56244f64c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f95f8ff9700 0 client.25331 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:14.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56244f63bf20 0x56244f648e00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f962b2cbb80 1 --2- 172.21.15.69:0/3768376081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56244f63c870 0x56244f649340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.698+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3768376081 >> 172.21.15.69:0/3768376081 conn(0x56244f631640 msgr2=0x56244f64d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.699+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3768376081 shutdown_connections 2023-07-06T20:19:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.699+0000 7f962b2cbb80 1 -- 172.21.15.69:0/3768376081 wait complete. 2023-07-06T20:19:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.699+0000 7f962b2cbb80 20 client.25331 trim_cache size 0 max 16384 2023-07-06T20:19:14.710 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:19:14.711 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-06T20:19:14.711 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:19:14.711 INFO:teuthology.orchestra.run.smithi069.stdout: "e38fc1e2-d0e3-4033-a66d-d56d28cb1270": { 2023-07-06T20:19:14.711 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:19:14.711 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "e38fc1e2-d0e3-4033-a66d-d56d28cb1270", 2023-07-06T20:19:14.711 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:19:14.712 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:19:14.712 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:19:14.712 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:19:14.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.699+0000 7f96179ff700 1 -- 172.21.15.69:0/2606421145 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f95fc0a11d0 msgr2=0x7f95fc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.699+0000 7f96179ff700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f95fc0a11d0 0x7f95fc0a3680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9608007900 tx=0x7f96080096c0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.699+0000 7f96179ff700 1 -- 172.21.15.69:0/2606421145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610073040 msgr2=0x7f9610197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.699+0000 7f96179ff700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610073040 0x7f9610197e50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f960c00e9c0 tx=0x7f960c00ecd0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.700+0000 7f96179ff700 1 -- 172.21.15.69:0/2606421145 shutdown_connections 2023-07-06T20:19:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.700+0000 7f96179ff700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f95fc0a11d0 0x7f95fc0a3680 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.700+0000 7f96179ff700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9610107ab0 0x7f961019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.700+0000 7f96179ff700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9610073040 0x7f9610197e50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.700+0000 7f96179ff700 1 --2- 172.21.15.69:0/2606421145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9610074620 0x7f9610198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.700+0000 7f96179ff700 1 -- 172.21.15.69:0/2606421145 >> 172.21.15.69:0/2606421145 conn(0x7f96100fb260 msgr2=0x7f9610109c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.700+0000 7f96179ff700 1 -- 172.21.15.69:0/2606421145 shutdown_connections 2023-07-06T20:19:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:14.700+0000 7f96179ff700 1 -- 172.21.15.69:0/2606421145 wait complete. 2023-07-06T20:19:14.727 INFO:tasks.cephfs.filesystem:scrub status for tag:e38fc1e2-d0e3-4033-a66d-d56d28cb1270 - {'path': '/', 'tag': 'e38fc1e2-d0e3-4033-a66d-d56d28cb1270', 'options': 'recursive,force'} 2023-07-06T20:19:15.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:14 smithi069 ceph-mon[126399]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 6 op/s; 89 B/s, 0 objects/s recovering 2023-07-06T20:19:15.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:14 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3685124082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:19:15.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:14 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3361644603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:15.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:14 smithi156 ceph-mon[129280]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 6 op/s; 89 B/s, 0 objects/s recovering 2023-07-06T20:19:15.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:14 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3685124082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:19:15.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:14 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3361644603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:15.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:14 smithi098 ceph-mon[128968]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 6 op/s; 89 B/s, 0 objects/s recovering 2023-07-06T20:19:15.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:14 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3685124082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:19:15.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:14 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3361644603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:19:16.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:15 smithi069 ceph-mon[126399]: scrub queued for path: / 2023-07-06T20:19:16.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:15 smithi069 ceph-mon[126399]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:16.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:15 smithi069 ceph-mon[126399]: scrub summary: active paths [/] 2023-07-06T20:19:16.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:15 smithi069 ceph-mon[126399]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:16.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:15 smithi156 ceph-mon[129280]: scrub queued for path: / 2023-07-06T20:19:16.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:15 smithi156 ceph-mon[129280]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:16.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:15 smithi156 ceph-mon[129280]: scrub summary: active paths [/] 2023-07-06T20:19:16.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:15 smithi156 ceph-mon[129280]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:16.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:15 smithi098 ceph-mon[128968]: scrub queued for path: / 2023-07-06T20:19:16.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:15 smithi098 ceph-mon[128968]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:16.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:15 smithi098 ceph-mon[128968]: scrub summary: active paths [/] 2023-07-06T20:19:16.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:15 smithi098 ceph-mon[128968]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:17.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:16 smithi069 ceph-mon[126399]: pgmap v326: 97 pgs: 97 active+clean; 652 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-07-06T20:19:17.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:16 smithi156 ceph-mon[129280]: pgmap v326: 97 pgs: 97 active+clean; 652 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-07-06T20:19:17.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:16 smithi098 ceph-mon[128968]: pgmap v326: 97 pgs: 97 active+clean; 652 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-07-06T20:19:17.753 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-07-06T20:19:17.848 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-07-06T20:19:17.852 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-07-06T20:19:17.951 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-07-06T20:19:18.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:17 smithi069 ceph-mon[126399]: scrub summary: idle 2023-07-06T20:19:18.044 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/: 2023-07-06T20:19:18.045 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:18.045 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 . 2023-07-06T20:19:18.045 INFO:teuthology.orchestra.run.smithi069.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 .. 2023-07-06T20:19:18.045 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 data 2023-07-06T20:19:18.045 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:18.045 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data: 2023-07-06T20:19:18.046 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:18.046 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:18.046 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 .. 2023-07-06T20:19:18.049 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-07-06T20:19:18.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:17 smithi156 ceph-mon[129280]: scrub summary: idle 2023-07-06T20:19:18.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:17 smithi098 ceph-mon[128968]: scrub summary: idle 2023-07-06T20:19:18.167 INFO:teuthology.orchestra.run.smithi069.stdout:[Service] 2023-07-06T20:19:18.168 INFO:teuthology.orchestra.run.smithi069.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-07-06T20:19:18.168 INFO:teuthology.orchestra.run.smithi069.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-07-06T20:19:18.173 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-07-06T20:19:18.354 INFO:teuthology.orchestra.run.smithi069.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-07-06T20:19:19.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:18 smithi069 ceph-mon[126399]: pgmap v327: 97 pgs: 97 active+clean; 652 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2023-07-06T20:19:19.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:18 smithi156 ceph-mon[129280]: pgmap v327: 97 pgs: 97 active+clean; 652 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2023-07-06T20:19:19.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:18 smithi098 ceph-mon[128968]: pgmap v327: 97 pgs: 97 active+clean; 652 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2023-07-06T20:19:21.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:20 smithi069 ceph-mon[126399]: pgmap v328: 97 pgs: 97 active+clean; 652 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-07-06T20:19:21.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:20 smithi156 ceph-mon[129280]: pgmap v328: 97 pgs: 97 active+clean; 652 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-07-06T20:19:21.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:20 smithi098 ceph-mon[128968]: pgmap v328: 97 pgs: 97 active+clean; 652 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-07-06T20:19:22.268 INFO:teuthology.orchestra.run.smithi069.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-07-06T20:19:22.274 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-07-06T20:19:22.383 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/: 2023-07-06T20:19:22.383 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.383 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 . 2023-07-06T20:19:22.384 INFO:teuthology.orchestra.run.smithi069.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 .. 2023-07-06T20:19:22.384 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 data 2023-07-06T20:19:22.384 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.384 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data: 2023-07-06T20:19:22.384 INFO:teuthology.orchestra.run.smithi069.stdout:total 30 2023-07-06T20:19:22.384 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 . 2023-07-06T20:19:22.384 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 .. 2023-07-06T20:19:22.385 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 base 2023-07-06T20:19:22.385 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Jul 6 20:19 global 2023-07-06T20:19:22.385 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 log 2023-07-06T20:19:22.385 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_commit_ts 2023-07-06T20:19:22.385 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_dynshmem 2023-07-06T20:19:22.385 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Jul 6 20:19 pg_hba.conf 2023-07-06T20:19:22.385 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jul 6 20:19 pg_ident.conf 2023-07-06T20:19:22.386 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 pg_logical 2023-07-06T20:19:22.386 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:19 pg_multixact 2023-07-06T20:19:22.386 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 pg_notify 2023-07-06T20:19:22.386 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_replslot 2023-07-06T20:19:22.386 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_serial 2023-07-06T20:19:22.386 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_snapshots 2023-07-06T20:19:22.386 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_stat 2023-07-06T20:19:22.386 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_stat_tmp 2023-07-06T20:19:22.387 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 pg_subtrans 2023-07-06T20:19:22.387 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_tblspc 2023-07-06T20:19:22.387 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_twophase 2023-07-06T20:19:22.387 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 PG_VERSION 2023-07-06T20:19:22.387 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:19 pg_wal 2023-07-06T20:19:22.387 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 pg_xact 2023-07-06T20:19:22.387 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jul 6 20:19 postgresql.auto.conf 2023-07-06T20:19:22.387 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Jul 6 20:19 postgresql.conf 2023-07-06T20:19:22.388 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.388 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base: 2023-07-06T20:19:22.388 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.388 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 . 2023-07-06T20:19:22.388 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.388 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 6 20:19 1 2023-07-06T20:19:22.388 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 6 20:19 12346 2023-07-06T20:19:22.389 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 6 20:19 12347 2023-07-06T20:19:22.389 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.389 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/1: 2023-07-06T20:19:22.389 INFO:teuthology.orchestra.run.smithi069.stdout:total 7217 2023-07-06T20:19:22.389 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 6 20:19 . 2023-07-06T20:19:22.389 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 .. 2023-07-06T20:19:22.389 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 112 2023-07-06T20:19:22.389 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 113 2023-07-06T20:19:22.390 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 6 20:19 12182 2023-07-06T20:19:22.390 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12182_fsm 2023-07-06T20:19:22.390 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12182_vm 2023-07-06T20:19:22.390 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12184 2023-07-06T20:19:22.390 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12186 2023-07-06T20:19:22.390 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12187 2023-07-06T20:19:22.390 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12187_fsm 2023-07-06T20:19:22.390 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12187_vm 2023-07-06T20:19:22.391 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12189 2023-07-06T20:19:22.391 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12191 2023-07-06T20:19:22.391 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12192 2023-07-06T20:19:22.391 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12192_fsm 2023-07-06T20:19:22.391 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12192_vm 2023-07-06T20:19:22.391 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12194 2023-07-06T20:19:22.391 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12196 2023-07-06T20:19:22.391 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12197 2023-07-06T20:19:22.392 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12197_fsm 2023-07-06T20:19:22.392 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12197_vm 2023-07-06T20:19:22.392 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12199 2023-07-06T20:19:22.392 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12201 2023-07-06T20:19:22.392 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12202 2023-07-06T20:19:22.392 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12202_fsm 2023-07-06T20:19:22.392 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12202_vm 2023-07-06T20:19:22.393 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12204 2023-07-06T20:19:22.393 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12206 2023-07-06T20:19:22.393 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12207 2023-07-06T20:19:22.393 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12207_fsm 2023-07-06T20:19:22.393 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12207_vm 2023-07-06T20:19:22.393 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12209 2023-07-06T20:19:22.393 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12211 2023-07-06T20:19:22.393 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12212 2023-07-06T20:19:22.394 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12214 2023-07-06T20:19:22.394 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12216 2023-07-06T20:19:22.394 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 6 20:19 1247 2023-07-06T20:19:22.394 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1247_fsm 2023-07-06T20:19:22.394 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1247_vm 2023-07-06T20:19:22.394 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 6 20:19 1249 2023-07-06T20:19:22.394 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1249_fsm 2023-07-06T20:19:22.394 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1249_vm 2023-07-06T20:19:22.395 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 6 20:19 1255 2023-07-06T20:19:22.395 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1255_fsm 2023-07-06T20:19:22.395 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1255_vm 2023-07-06T20:19:22.395 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 6 20:19 1259 2023-07-06T20:19:22.395 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1259_fsm 2023-07-06T20:19:22.395 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1259_vm 2023-07-06T20:19:22.396 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1417 2023-07-06T20:19:22.396 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1417_vm 2023-07-06T20:19:22.396 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1418 2023-07-06T20:19:22.396 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1418_vm 2023-07-06T20:19:22.396 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 174 2023-07-06T20:19:22.396 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 175 2023-07-06T20:19:22.397 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2187 2023-07-06T20:19:22.397 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2224 2023-07-06T20:19:22.397 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2224_vm 2023-07-06T20:19:22.397 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2328 2023-07-06T20:19:22.397 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2328_vm 2023-07-06T20:19:22.397 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2336 2023-07-06T20:19:22.397 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2336_vm 2023-07-06T20:19:22.398 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2337 2023-07-06T20:19:22.398 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2600 2023-07-06T20:19:22.398 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2600_fsm 2023-07-06T20:19:22.398 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2600_vm 2023-07-06T20:19:22.398 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2601 2023-07-06T20:19:22.398 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2601_fsm 2023-07-06T20:19:22.398 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2601_vm 2023-07-06T20:19:22.398 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2602 2023-07-06T20:19:22.399 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2602_fsm 2023-07-06T20:19:22.399 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2602_vm 2023-07-06T20:19:22.399 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2603 2023-07-06T20:19:22.399 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2603_fsm 2023-07-06T20:19:22.399 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2603_vm 2023-07-06T20:19:22.399 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2604 2023-07-06T20:19:22.400 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2604_vm 2023-07-06T20:19:22.400 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2605 2023-07-06T20:19:22.400 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2605_fsm 2023-07-06T20:19:22.400 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2605_vm 2023-07-06T20:19:22.400 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2606 2023-07-06T20:19:22.400 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2606_fsm 2023-07-06T20:19:22.400 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2606_vm 2023-07-06T20:19:22.401 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2607 2023-07-06T20:19:22.401 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2607_fsm 2023-07-06T20:19:22.401 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2607_vm 2023-07-06T20:19:22.401 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 6 20:19 2608 2023-07-06T20:19:22.401 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2608_fsm 2023-07-06T20:19:22.401 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2608_vm 2023-07-06T20:19:22.401 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 6 20:19 2609 2023-07-06T20:19:22.401 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2609_fsm 2023-07-06T20:19:22.402 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2609_vm 2023-07-06T20:19:22.402 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2610 2023-07-06T20:19:22.402 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2610_fsm 2023-07-06T20:19:22.402 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2610_vm 2023-07-06T20:19:22.402 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2611 2023-07-06T20:19:22.402 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2611_vm 2023-07-06T20:19:22.402 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2612 2023-07-06T20:19:22.403 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2612_fsm 2023-07-06T20:19:22.403 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2612_vm 2023-07-06T20:19:22.403 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2613 2023-07-06T20:19:22.403 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2613_vm 2023-07-06T20:19:22.403 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2615 2023-07-06T20:19:22.403 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2615_fsm 2023-07-06T20:19:22.403 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2615_vm 2023-07-06T20:19:22.404 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2616 2023-07-06T20:19:22.404 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2616_fsm 2023-07-06T20:19:22.404 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2616_vm 2023-07-06T20:19:22.404 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 6 20:19 2617 2023-07-06T20:19:22.404 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2617_fsm 2023-07-06T20:19:22.404 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2617_vm 2023-07-06T20:19:22.404 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 6 20:19 2618 2023-07-06T20:19:22.404 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2618_fsm 2023-07-06T20:19:22.405 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2618_vm 2023-07-06T20:19:22.405 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 6 20:19 2619 2023-07-06T20:19:22.405 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2619_fsm 2023-07-06T20:19:22.405 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2619_vm 2023-07-06T20:19:22.405 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2620 2023-07-06T20:19:22.405 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2620_vm 2023-07-06T20:19:22.405 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2650 2023-07-06T20:19:22.406 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2651 2023-07-06T20:19:22.406 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2652 2023-07-06T20:19:22.406 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2653 2023-07-06T20:19:22.406 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2654 2023-07-06T20:19:22.406 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2655 2023-07-06T20:19:22.406 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2656 2023-07-06T20:19:22.406 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2657 2023-07-06T20:19:22.407 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 6 20:19 2658 2023-07-06T20:19:22.407 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 6 20:19 2659 2023-07-06T20:19:22.407 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2660 2023-07-06T20:19:22.407 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2661 2023-07-06T20:19:22.407 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2662 2023-07-06T20:19:22.407 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2663 2023-07-06T20:19:22.407 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2664 2023-07-06T20:19:22.407 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2665 2023-07-06T20:19:22.408 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2666 2023-07-06T20:19:22.408 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2667 2023-07-06T20:19:22.408 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2668 2023-07-06T20:19:22.408 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2669 2023-07-06T20:19:22.408 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2670 2023-07-06T20:19:22.409 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 335872 Jul 6 20:19 2673 2023-07-06T20:19:22.409 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Jul 6 20:19 2674 2023-07-06T20:19:22.409 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 6 20:19 2675 2023-07-06T20:19:22.409 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2678 2023-07-06T20:19:22.409 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2679 2023-07-06T20:19:22.409 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2680 2023-07-06T20:19:22.410 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2681 2023-07-06T20:19:22.410 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2682 2023-07-06T20:19:22.410 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2683 2023-07-06T20:19:22.410 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2684 2023-07-06T20:19:22.410 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2685 2023-07-06T20:19:22.410 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2686 2023-07-06T20:19:22.410 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2687 2023-07-06T20:19:22.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2688 2023-07-06T20:19:22.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2689 2023-07-06T20:19:22.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 6 20:19 2690 2023-07-06T20:19:22.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 6 20:19 2691 2023-07-06T20:19:22.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2692 2023-07-06T20:19:22.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2693 2023-07-06T20:19:22.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2696 2023-07-06T20:19:22.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2699 2023-07-06T20:19:22.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2701 2023-07-06T20:19:22.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2702 2023-07-06T20:19:22.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2703 2023-07-06T20:19:22.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2704 2023-07-06T20:19:22.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2753 2023-07-06T20:19:22.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2753_fsm 2023-07-06T20:19:22.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2753_vm 2023-07-06T20:19:22.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2754 2023-07-06T20:19:22.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2755 2023-07-06T20:19:22.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2756 2023-07-06T20:19:22.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2757 2023-07-06T20:19:22.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2830 2023-07-06T20:19:22.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2830_vm 2023-07-06T20:19:22.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2831 2023-07-06T20:19:22.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2832 2023-07-06T20:19:22.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2832_vm 2023-07-06T20:19:22.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2833 2023-07-06T20:19:22.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2834 2023-07-06T20:19:22.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2834_vm 2023-07-06T20:19:22.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2835 2023-07-06T20:19:22.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2836 2023-07-06T20:19:22.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2836_vm 2023-07-06T20:19:22.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2837 2023-07-06T20:19:22.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 6 20:19 2838 2023-07-06T20:19:22.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2838_fsm 2023-07-06T20:19:22.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2838_vm 2023-07-06T20:19:22.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2839 2023-07-06T20:19:22.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2840 2023-07-06T20:19:22.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2840_fsm 2023-07-06T20:19:22.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2840_vm 2023-07-06T20:19:22.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2841 2023-07-06T20:19:22.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2995 2023-07-06T20:19:22.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2995_vm 2023-07-06T20:19:22.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2996 2023-07-06T20:19:22.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3079 2023-07-06T20:19:22.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3079_fsm 2023-07-06T20:19:22.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3079_vm 2023-07-06T20:19:22.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3080 2023-07-06T20:19:22.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3081 2023-07-06T20:19:22.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3085 2023-07-06T20:19:22.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3118 2023-07-06T20:19:22.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3118_vm 2023-07-06T20:19:22.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3119 2023-07-06T20:19:22.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3164 2023-07-06T20:19:22.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3256 2023-07-06T20:19:22.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3256_vm 2023-07-06T20:19:22.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3257 2023-07-06T20:19:22.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3258 2023-07-06T20:19:22.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3350 2023-07-06T20:19:22.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3350_vm 2023-07-06T20:19:22.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3351 2023-07-06T20:19:22.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3379 2023-07-06T20:19:22.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3380 2023-07-06T20:19:22.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3381 2023-07-06T20:19:22.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3381_vm 2023-07-06T20:19:22.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3394 2023-07-06T20:19:22.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3394_fsm 2023-07-06T20:19:22.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3394_vm 2023-07-06T20:19:22.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3395 2023-07-06T20:19:22.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3439 2023-07-06T20:19:22.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3439_vm 2023-07-06T20:19:22.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3440 2023-07-06T20:19:22.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 3455 2023-07-06T20:19:22.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3456 2023-07-06T20:19:22.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3456_fsm 2023-07-06T20:19:22.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3456_vm 2023-07-06T20:19:22.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3466 2023-07-06T20:19:22.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3466_vm 2023-07-06T20:19:22.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3467 2023-07-06T20:19:22.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3468 2023-07-06T20:19:22.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3501 2023-07-06T20:19:22.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3501_vm 2023-07-06T20:19:22.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3502 2023-07-06T20:19:22.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3503 2023-07-06T20:19:22.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3534 2023-07-06T20:19:22.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3541 2023-07-06T20:19:22.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3541_fsm 2023-07-06T20:19:22.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3541_vm 2023-07-06T20:19:22.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3542 2023-07-06T20:19:22.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3574 2023-07-06T20:19:22.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3575 2023-07-06T20:19:22.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3576 2023-07-06T20:19:22.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3576_vm 2023-07-06T20:19:22.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3596 2023-07-06T20:19:22.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3596_vm 2023-07-06T20:19:22.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3597 2023-07-06T20:19:22.430 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3598 2023-07-06T20:19:22.430 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3598_vm 2023-07-06T20:19:22.430 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3599 2023-07-06T20:19:22.430 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3600 2023-07-06T20:19:22.430 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3600_fsm 2023-07-06T20:19:22.430 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3600_vm 2023-07-06T20:19:22.430 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3601 2023-07-06T20:19:22.430 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3601_fsm 2023-07-06T20:19:22.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3601_vm 2023-07-06T20:19:22.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3602 2023-07-06T20:19:22.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3602_fsm 2023-07-06T20:19:22.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3602_vm 2023-07-06T20:19:22.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3603 2023-07-06T20:19:22.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3603_fsm 2023-07-06T20:19:22.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3603_vm 2023-07-06T20:19:22.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3604 2023-07-06T20:19:22.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3605 2023-07-06T20:19:22.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3606 2023-07-06T20:19:22.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3607 2023-07-06T20:19:22.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3608 2023-07-06T20:19:22.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 3609 2023-07-06T20:19:22.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3712 2023-07-06T20:19:22.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3764 2023-07-06T20:19:22.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3764_fsm 2023-07-06T20:19:22.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3764_vm 2023-07-06T20:19:22.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3766 2023-07-06T20:19:22.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3767 2023-07-06T20:19:22.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3997 2023-07-06T20:19:22.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 5002 2023-07-06T20:19:22.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 548 2023-07-06T20:19:22.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 549 2023-07-06T20:19:22.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6102 2023-07-06T20:19:22.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6102_vm 2023-07-06T20:19:22.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6104 2023-07-06T20:19:22.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6104_vm 2023-07-06T20:19:22.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6106 2023-07-06T20:19:22.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6106_vm 2023-07-06T20:19:22.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6110 2023-07-06T20:19:22.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6111 2023-07-06T20:19:22.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6112 2023-07-06T20:19:22.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6113 2023-07-06T20:19:22.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6117 2023-07-06T20:19:22.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 826 2023-07-06T20:19:22.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 826_vm 2023-07-06T20:19:22.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 827 2023-07-06T20:19:22.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 828 2023-07-06T20:19:22.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 6 20:19 pg_filenode.map 2023-07-06T20:19:22.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 PG_VERSION 2023-07-06T20:19:22.438 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.438 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-07-06T20:19:22.438 INFO:teuthology.orchestra.run.smithi069.stdout:total 7217 2023-07-06T20:19:22.438 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 6 20:19 . 2023-07-06T20:19:22.439 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 .. 2023-07-06T20:19:22.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 112 2023-07-06T20:19:22.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 113 2023-07-06T20:19:22.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 6 20:19 12182 2023-07-06T20:19:22.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12182_fsm 2023-07-06T20:19:22.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12182_vm 2023-07-06T20:19:22.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12184 2023-07-06T20:19:22.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12186 2023-07-06T20:19:22.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12187 2023-07-06T20:19:22.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12187_fsm 2023-07-06T20:19:22.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12187_vm 2023-07-06T20:19:22.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12189 2023-07-06T20:19:22.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12191 2023-07-06T20:19:22.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12192 2023-07-06T20:19:22.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12192_fsm 2023-07-06T20:19:22.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12192_vm 2023-07-06T20:19:22.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12194 2023-07-06T20:19:22.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12196 2023-07-06T20:19:22.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12197 2023-07-06T20:19:22.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12197_fsm 2023-07-06T20:19:22.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12197_vm 2023-07-06T20:19:22.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12199 2023-07-06T20:19:22.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12201 2023-07-06T20:19:22.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12202 2023-07-06T20:19:22.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12202_fsm 2023-07-06T20:19:22.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12202_vm 2023-07-06T20:19:22.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12204 2023-07-06T20:19:22.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12206 2023-07-06T20:19:22.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12207 2023-07-06T20:19:22.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12207_fsm 2023-07-06T20:19:22.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12207_vm 2023-07-06T20:19:22.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12209 2023-07-06T20:19:22.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12211 2023-07-06T20:19:22.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12212 2023-07-06T20:19:22.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12214 2023-07-06T20:19:22.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12216 2023-07-06T20:19:22.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 6 20:19 1247 2023-07-06T20:19:22.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1247_fsm 2023-07-06T20:19:22.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1247_vm 2023-07-06T20:19:22.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 6 20:19 1249 2023-07-06T20:19:22.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1249_fsm 2023-07-06T20:19:22.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1249_vm 2023-07-06T20:19:22.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 6 20:19 1255 2023-07-06T20:19:22.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1255_fsm 2023-07-06T20:19:22.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1255_vm 2023-07-06T20:19:22.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 6 20:19 1259 2023-07-06T20:19:22.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1259_fsm 2023-07-06T20:19:22.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1259_vm 2023-07-06T20:19:22.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1417 2023-07-06T20:19:22.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1417_vm 2023-07-06T20:19:22.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1418 2023-07-06T20:19:22.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1418_vm 2023-07-06T20:19:22.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 174 2023-07-06T20:19:22.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 175 2023-07-06T20:19:22.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2187 2023-07-06T20:19:22.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2224 2023-07-06T20:19:22.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2224_vm 2023-07-06T20:19:22.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2328 2023-07-06T20:19:22.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2328_vm 2023-07-06T20:19:22.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2336 2023-07-06T20:19:22.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2336_vm 2023-07-06T20:19:22.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2337 2023-07-06T20:19:22.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2600 2023-07-06T20:19:22.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2600_fsm 2023-07-06T20:19:22.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2600_vm 2023-07-06T20:19:22.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2601 2023-07-06T20:19:22.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2601_fsm 2023-07-06T20:19:22.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2601_vm 2023-07-06T20:19:22.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2602 2023-07-06T20:19:22.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2602_fsm 2023-07-06T20:19:22.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2602_vm 2023-07-06T20:19:22.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2603 2023-07-06T20:19:22.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2603_fsm 2023-07-06T20:19:22.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2603_vm 2023-07-06T20:19:22.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2604 2023-07-06T20:19:22.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2604_vm 2023-07-06T20:19:22.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2605 2023-07-06T20:19:22.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2605_fsm 2023-07-06T20:19:22.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2605_vm 2023-07-06T20:19:22.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2606 2023-07-06T20:19:22.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2606_fsm 2023-07-06T20:19:22.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2606_vm 2023-07-06T20:19:22.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2607 2023-07-06T20:19:22.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2607_fsm 2023-07-06T20:19:22.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2607_vm 2023-07-06T20:19:22.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 6 20:19 2608 2023-07-06T20:19:22.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2608_fsm 2023-07-06T20:19:22.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2608_vm 2023-07-06T20:19:22.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 6 20:19 2609 2023-07-06T20:19:22.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2609_fsm 2023-07-06T20:19:22.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2609_vm 2023-07-06T20:19:22.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2610 2023-07-06T20:19:22.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2610_fsm 2023-07-06T20:19:22.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2610_vm 2023-07-06T20:19:22.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2611 2023-07-06T20:19:22.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2611_vm 2023-07-06T20:19:22.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2612 2023-07-06T20:19:22.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2612_fsm 2023-07-06T20:19:22.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2612_vm 2023-07-06T20:19:22.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2613 2023-07-06T20:19:22.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2613_vm 2023-07-06T20:19:22.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2615 2023-07-06T20:19:22.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2615_fsm 2023-07-06T20:19:22.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2615_vm 2023-07-06T20:19:22.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2616 2023-07-06T20:19:22.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2616_fsm 2023-07-06T20:19:22.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2616_vm 2023-07-06T20:19:22.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 6 20:19 2617 2023-07-06T20:19:22.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2617_fsm 2023-07-06T20:19:22.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2617_vm 2023-07-06T20:19:22.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 6 20:19 2618 2023-07-06T20:19:22.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2618_fsm 2023-07-06T20:19:22.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2618_vm 2023-07-06T20:19:22.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 6 20:19 2619 2023-07-06T20:19:22.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2619_fsm 2023-07-06T20:19:22.454 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2619_vm 2023-07-06T20:19:22.454 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2620 2023-07-06T20:19:22.454 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2620_vm 2023-07-06T20:19:22.454 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2650 2023-07-06T20:19:22.454 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2651 2023-07-06T20:19:22.454 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2652 2023-07-06T20:19:22.454 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2653 2023-07-06T20:19:22.454 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2654 2023-07-06T20:19:22.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2655 2023-07-06T20:19:22.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2656 2023-07-06T20:19:22.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2657 2023-07-06T20:19:22.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 6 20:19 2658 2023-07-06T20:19:22.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 6 20:19 2659 2023-07-06T20:19:22.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2660 2023-07-06T20:19:22.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2661 2023-07-06T20:19:22.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2662 2023-07-06T20:19:22.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2663 2023-07-06T20:19:22.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2664 2023-07-06T20:19:22.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2665 2023-07-06T20:19:22.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2666 2023-07-06T20:19:22.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2667 2023-07-06T20:19:22.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2668 2023-07-06T20:19:22.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2669 2023-07-06T20:19:22.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2670 2023-07-06T20:19:22.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 335872 Jul 6 20:19 2673 2023-07-06T20:19:22.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Jul 6 20:19 2674 2023-07-06T20:19:22.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 6 20:19 2675 2023-07-06T20:19:22.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2678 2023-07-06T20:19:22.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2679 2023-07-06T20:19:22.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2680 2023-07-06T20:19:22.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2681 2023-07-06T20:19:22.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2682 2023-07-06T20:19:22.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2683 2023-07-06T20:19:22.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2684 2023-07-06T20:19:22.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2685 2023-07-06T20:19:22.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2686 2023-07-06T20:19:22.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2687 2023-07-06T20:19:22.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2688 2023-07-06T20:19:22.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2689 2023-07-06T20:19:22.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 6 20:19 2690 2023-07-06T20:19:22.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 6 20:19 2691 2023-07-06T20:19:22.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2692 2023-07-06T20:19:22.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2693 2023-07-06T20:19:22.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2696 2023-07-06T20:19:22.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2699 2023-07-06T20:19:22.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2701 2023-07-06T20:19:22.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2702 2023-07-06T20:19:22.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2703 2023-07-06T20:19:22.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2704 2023-07-06T20:19:22.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2753 2023-07-06T20:19:22.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2753_fsm 2023-07-06T20:19:22.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2753_vm 2023-07-06T20:19:22.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2754 2023-07-06T20:19:22.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2755 2023-07-06T20:19:22.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2756 2023-07-06T20:19:22.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2757 2023-07-06T20:19:22.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2830 2023-07-06T20:19:22.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2830_vm 2023-07-06T20:19:22.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2831 2023-07-06T20:19:22.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2832 2023-07-06T20:19:22.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2832_vm 2023-07-06T20:19:22.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2833 2023-07-06T20:19:22.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2834 2023-07-06T20:19:22.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2834_vm 2023-07-06T20:19:22.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2835 2023-07-06T20:19:22.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2836 2023-07-06T20:19:22.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2836_vm 2023-07-06T20:19:22.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2837 2023-07-06T20:19:22.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 6 20:19 2838 2023-07-06T20:19:22.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2838_fsm 2023-07-06T20:19:22.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2838_vm 2023-07-06T20:19:22.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2839 2023-07-06T20:19:22.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2840 2023-07-06T20:19:22.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2840_fsm 2023-07-06T20:19:22.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2840_vm 2023-07-06T20:19:22.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2841 2023-07-06T20:19:22.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2995 2023-07-06T20:19:22.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2995_vm 2023-07-06T20:19:22.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2996 2023-07-06T20:19:22.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3079 2023-07-06T20:19:22.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3079_fsm 2023-07-06T20:19:22.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3079_vm 2023-07-06T20:19:22.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3080 2023-07-06T20:19:22.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3081 2023-07-06T20:19:22.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3085 2023-07-06T20:19:22.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3118 2023-07-06T20:19:22.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3118_vm 2023-07-06T20:19:22.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3119 2023-07-06T20:19:22.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3164 2023-07-06T20:19:22.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3256 2023-07-06T20:19:22.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3256_vm 2023-07-06T20:19:22.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3257 2023-07-06T20:19:22.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3258 2023-07-06T20:19:22.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3350 2023-07-06T20:19:22.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3350_vm 2023-07-06T20:19:22.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3351 2023-07-06T20:19:22.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3379 2023-07-06T20:19:22.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3380 2023-07-06T20:19:22.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3381 2023-07-06T20:19:22.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3381_vm 2023-07-06T20:19:22.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3394 2023-07-06T20:19:22.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3394_fsm 2023-07-06T20:19:22.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3394_vm 2023-07-06T20:19:22.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3395 2023-07-06T20:19:22.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3439 2023-07-06T20:19:22.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3439_vm 2023-07-06T20:19:22.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3440 2023-07-06T20:19:22.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 3455 2023-07-06T20:19:22.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3456 2023-07-06T20:19:22.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3456_fsm 2023-07-06T20:19:22.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3456_vm 2023-07-06T20:19:22.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3466 2023-07-06T20:19:22.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3466_vm 2023-07-06T20:19:22.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3467 2023-07-06T20:19:22.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3468 2023-07-06T20:19:22.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3501 2023-07-06T20:19:22.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3501_vm 2023-07-06T20:19:22.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3502 2023-07-06T20:19:22.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3503 2023-07-06T20:19:22.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3534 2023-07-06T20:19:22.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3541 2023-07-06T20:19:22.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3541_fsm 2023-07-06T20:19:22.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3541_vm 2023-07-06T20:19:22.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3542 2023-07-06T20:19:22.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3574 2023-07-06T20:19:22.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3575 2023-07-06T20:19:22.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3576 2023-07-06T20:19:22.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3576_vm 2023-07-06T20:19:22.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3596 2023-07-06T20:19:22.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3596_vm 2023-07-06T20:19:22.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3597 2023-07-06T20:19:22.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3598 2023-07-06T20:19:22.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3598_vm 2023-07-06T20:19:22.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3599 2023-07-06T20:19:22.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3600 2023-07-06T20:19:22.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3600_fsm 2023-07-06T20:19:22.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3600_vm 2023-07-06T20:19:22.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3601 2023-07-06T20:19:22.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3601_fsm 2023-07-06T20:19:22.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3601_vm 2023-07-06T20:19:22.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3602 2023-07-06T20:19:22.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3602_fsm 2023-07-06T20:19:22.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3602_vm 2023-07-06T20:19:22.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3603 2023-07-06T20:19:22.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3603_fsm 2023-07-06T20:19:22.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3603_vm 2023-07-06T20:19:22.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3604 2023-07-06T20:19:22.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3605 2023-07-06T20:19:22.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3606 2023-07-06T20:19:22.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3607 2023-07-06T20:19:22.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3608 2023-07-06T20:19:22.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 3609 2023-07-06T20:19:22.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3712 2023-07-06T20:19:22.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3764 2023-07-06T20:19:22.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3764_fsm 2023-07-06T20:19:22.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3764_vm 2023-07-06T20:19:22.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3766 2023-07-06T20:19:22.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3767 2023-07-06T20:19:22.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3997 2023-07-06T20:19:22.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 5002 2023-07-06T20:19:22.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 548 2023-07-06T20:19:22.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 549 2023-07-06T20:19:22.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6102 2023-07-06T20:19:22.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6102_vm 2023-07-06T20:19:22.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6104 2023-07-06T20:19:22.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6104_vm 2023-07-06T20:19:22.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6106 2023-07-06T20:19:22.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6106_vm 2023-07-06T20:19:22.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6110 2023-07-06T20:19:22.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6111 2023-07-06T20:19:22.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6112 2023-07-06T20:19:22.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6113 2023-07-06T20:19:22.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6117 2023-07-06T20:19:22.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 826 2023-07-06T20:19:22.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 826_vm 2023-07-06T20:19:22.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 827 2023-07-06T20:19:22.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 828 2023-07-06T20:19:22.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 6 20:19 pg_filenode.map 2023-07-06T20:19:22.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 PG_VERSION 2023-07-06T20:19:22.478 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.478 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-07-06T20:19:22.478 INFO:teuthology.orchestra.run.smithi069.stdout:total 7217 2023-07-06T20:19:22.478 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 6 20:19 . 2023-07-06T20:19:22.478 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 .. 2023-07-06T20:19:22.479 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 112 2023-07-06T20:19:22.479 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 113 2023-07-06T20:19:22.479 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 6 20:19 12182 2023-07-06T20:19:22.479 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12182_fsm 2023-07-06T20:19:22.479 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12182_vm 2023-07-06T20:19:22.479 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12184 2023-07-06T20:19:22.479 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12186 2023-07-06T20:19:22.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12187 2023-07-06T20:19:22.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12187_fsm 2023-07-06T20:19:22.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12187_vm 2023-07-06T20:19:22.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12189 2023-07-06T20:19:22.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12191 2023-07-06T20:19:22.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12192 2023-07-06T20:19:22.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12192_fsm 2023-07-06T20:19:22.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12192_vm 2023-07-06T20:19:22.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12194 2023-07-06T20:19:22.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12196 2023-07-06T20:19:22.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12197 2023-07-06T20:19:22.483 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12197_fsm 2023-07-06T20:19:22.483 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12197_vm 2023-07-06T20:19:22.483 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12199 2023-07-06T20:19:22.483 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12201 2023-07-06T20:19:22.484 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12202 2023-07-06T20:19:22.484 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12202_fsm 2023-07-06T20:19:22.484 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12202_vm 2023-07-06T20:19:22.484 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12204 2023-07-06T20:19:22.485 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12206 2023-07-06T20:19:22.485 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12207 2023-07-06T20:19:22.485 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12207_fsm 2023-07-06T20:19:22.485 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12207_vm 2023-07-06T20:19:22.485 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12209 2023-07-06T20:19:22.486 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12211 2023-07-06T20:19:22.486 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12212 2023-07-06T20:19:22.486 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12214 2023-07-06T20:19:22.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12216 2023-07-06T20:19:22.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 6 20:19 1247 2023-07-06T20:19:22.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1247_fsm 2023-07-06T20:19:22.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1247_vm 2023-07-06T20:19:22.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 6 20:19 1249 2023-07-06T20:19:22.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1249_fsm 2023-07-06T20:19:22.488 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1249_vm 2023-07-06T20:19:22.488 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 6 20:19 1255 2023-07-06T20:19:22.488 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1255_fsm 2023-07-06T20:19:22.488 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1255_vm 2023-07-06T20:19:22.488 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 6 20:19 1259 2023-07-06T20:19:22.488 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1259_fsm 2023-07-06T20:19:22.488 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1259_vm 2023-07-06T20:19:22.489 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1417 2023-07-06T20:19:22.489 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1417_vm 2023-07-06T20:19:22.489 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1418 2023-07-06T20:19:22.489 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1418_vm 2023-07-06T20:19:22.489 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 174 2023-07-06T20:19:22.489 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 175 2023-07-06T20:19:22.489 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2187 2023-07-06T20:19:22.489 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2224 2023-07-06T20:19:22.490 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2224_vm 2023-07-06T20:19:22.490 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2328 2023-07-06T20:19:22.490 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2328_vm 2023-07-06T20:19:22.490 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2336 2023-07-06T20:19:22.490 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2336_vm 2023-07-06T20:19:22.490 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2337 2023-07-06T20:19:22.490 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2600 2023-07-06T20:19:22.490 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2600_fsm 2023-07-06T20:19:22.491 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2600_vm 2023-07-06T20:19:22.491 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2601 2023-07-06T20:19:22.491 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2601_fsm 2023-07-06T20:19:22.491 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2601_vm 2023-07-06T20:19:22.491 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2602 2023-07-06T20:19:22.491 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2602_fsm 2023-07-06T20:19:22.491 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2602_vm 2023-07-06T20:19:22.492 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2603 2023-07-06T20:19:22.492 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2603_fsm 2023-07-06T20:19:22.492 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2603_vm 2023-07-06T20:19:22.492 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2604 2023-07-06T20:19:22.492 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2604_vm 2023-07-06T20:19:22.492 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2605 2023-07-06T20:19:22.492 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2605_fsm 2023-07-06T20:19:22.492 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2605_vm 2023-07-06T20:19:22.493 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2606 2023-07-06T20:19:22.493 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2606_fsm 2023-07-06T20:19:22.493 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2606_vm 2023-07-06T20:19:22.493 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2607 2023-07-06T20:19:22.493 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2607_fsm 2023-07-06T20:19:22.493 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2607_vm 2023-07-06T20:19:22.493 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 6 20:19 2608 2023-07-06T20:19:22.494 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2608_fsm 2023-07-06T20:19:22.494 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2608_vm 2023-07-06T20:19:22.494 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 6 20:19 2609 2023-07-06T20:19:22.494 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2609_fsm 2023-07-06T20:19:22.494 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2609_vm 2023-07-06T20:19:22.494 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2610 2023-07-06T20:19:22.494 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2610_fsm 2023-07-06T20:19:22.494 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2610_vm 2023-07-06T20:19:22.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2611 2023-07-06T20:19:22.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2611_vm 2023-07-06T20:19:22.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2612 2023-07-06T20:19:22.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2612_fsm 2023-07-06T20:19:22.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2612_vm 2023-07-06T20:19:22.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2613 2023-07-06T20:19:22.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2613_vm 2023-07-06T20:19:22.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2615 2023-07-06T20:19:22.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2615_fsm 2023-07-06T20:19:22.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2615_vm 2023-07-06T20:19:22.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2616 2023-07-06T20:19:22.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2616_fsm 2023-07-06T20:19:22.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2616_vm 2023-07-06T20:19:22.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 6 20:19 2617 2023-07-06T20:19:22.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2617_fsm 2023-07-06T20:19:22.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2617_vm 2023-07-06T20:19:22.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 6 20:19 2618 2023-07-06T20:19:22.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2618_fsm 2023-07-06T20:19:22.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2618_vm 2023-07-06T20:19:22.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 6 20:19 2619 2023-07-06T20:19:22.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2619_fsm 2023-07-06T20:19:22.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2619_vm 2023-07-06T20:19:22.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2620 2023-07-06T20:19:22.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2620_vm 2023-07-06T20:19:22.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2650 2023-07-06T20:19:22.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2651 2023-07-06T20:19:22.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2652 2023-07-06T20:19:22.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2653 2023-07-06T20:19:22.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2654 2023-07-06T20:19:22.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2655 2023-07-06T20:19:22.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2656 2023-07-06T20:19:22.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2657 2023-07-06T20:19:22.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 6 20:19 2658 2023-07-06T20:19:22.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 6 20:19 2659 2023-07-06T20:19:22.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2660 2023-07-06T20:19:22.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2661 2023-07-06T20:19:22.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2662 2023-07-06T20:19:22.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2663 2023-07-06T20:19:22.500 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2664 2023-07-06T20:19:22.500 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2665 2023-07-06T20:19:22.500 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2666 2023-07-06T20:19:22.500 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2667 2023-07-06T20:19:22.500 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2668 2023-07-06T20:19:22.500 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2669 2023-07-06T20:19:22.500 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2670 2023-07-06T20:19:22.501 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 335872 Jul 6 20:19 2673 2023-07-06T20:19:22.501 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Jul 6 20:19 2674 2023-07-06T20:19:22.501 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 6 20:19 2675 2023-07-06T20:19:22.501 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2678 2023-07-06T20:19:22.501 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2679 2023-07-06T20:19:22.501 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2680 2023-07-06T20:19:22.501 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2681 2023-07-06T20:19:22.501 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2682 2023-07-06T20:19:22.502 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2683 2023-07-06T20:19:22.502 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2684 2023-07-06T20:19:22.502 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2685 2023-07-06T20:19:22.502 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2686 2023-07-06T20:19:22.502 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2687 2023-07-06T20:19:22.502 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2688 2023-07-06T20:19:22.502 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2689 2023-07-06T20:19:22.502 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 6 20:19 2690 2023-07-06T20:19:22.503 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 6 20:19 2691 2023-07-06T20:19:22.503 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2692 2023-07-06T20:19:22.503 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2693 2023-07-06T20:19:22.503 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2696 2023-07-06T20:19:22.503 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2699 2023-07-06T20:19:22.503 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2701 2023-07-06T20:19:22.503 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2702 2023-07-06T20:19:22.504 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2703 2023-07-06T20:19:22.504 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2704 2023-07-06T20:19:22.504 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2753 2023-07-06T20:19:22.504 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2753_fsm 2023-07-06T20:19:22.504 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2753_vm 2023-07-06T20:19:22.504 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2754 2023-07-06T20:19:22.504 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2755 2023-07-06T20:19:22.505 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2756 2023-07-06T20:19:22.505 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2757 2023-07-06T20:19:22.505 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2830 2023-07-06T20:19:22.505 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2830_vm 2023-07-06T20:19:22.505 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2831 2023-07-06T20:19:22.505 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2832 2023-07-06T20:19:22.505 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2832_vm 2023-07-06T20:19:22.505 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2833 2023-07-06T20:19:22.506 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2834 2023-07-06T20:19:22.506 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2834_vm 2023-07-06T20:19:22.506 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2835 2023-07-06T20:19:22.506 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2836 2023-07-06T20:19:22.506 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2836_vm 2023-07-06T20:19:22.506 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2837 2023-07-06T20:19:22.506 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 6 20:19 2838 2023-07-06T20:19:22.506 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2838_fsm 2023-07-06T20:19:22.507 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2838_vm 2023-07-06T20:19:22.507 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2839 2023-07-06T20:19:22.507 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2840 2023-07-06T20:19:22.507 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2840_fsm 2023-07-06T20:19:22.507 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2840_vm 2023-07-06T20:19:22.507 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2841 2023-07-06T20:19:22.508 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2995 2023-07-06T20:19:22.508 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2995_vm 2023-07-06T20:19:22.508 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2996 2023-07-06T20:19:22.508 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3079 2023-07-06T20:19:22.508 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3079_fsm 2023-07-06T20:19:22.508 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3079_vm 2023-07-06T20:19:22.508 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3080 2023-07-06T20:19:22.508 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3081 2023-07-06T20:19:22.509 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3085 2023-07-06T20:19:22.509 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3118 2023-07-06T20:19:22.509 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3118_vm 2023-07-06T20:19:22.509 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3119 2023-07-06T20:19:22.509 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3164 2023-07-06T20:19:22.509 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3256 2023-07-06T20:19:22.509 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3256_vm 2023-07-06T20:19:22.510 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3257 2023-07-06T20:19:22.510 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3258 2023-07-06T20:19:22.510 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3350 2023-07-06T20:19:22.510 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3350_vm 2023-07-06T20:19:22.510 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3351 2023-07-06T20:19:22.510 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3379 2023-07-06T20:19:22.510 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3380 2023-07-06T20:19:22.511 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3381 2023-07-06T20:19:22.511 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3381_vm 2023-07-06T20:19:22.511 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3394 2023-07-06T20:19:22.511 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3394_fsm 2023-07-06T20:19:22.511 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3394_vm 2023-07-06T20:19:22.511 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3395 2023-07-06T20:19:22.511 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3439 2023-07-06T20:19:22.511 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3439_vm 2023-07-06T20:19:22.512 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3440 2023-07-06T20:19:22.512 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 3455 2023-07-06T20:19:22.512 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3456 2023-07-06T20:19:22.512 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3456_fsm 2023-07-06T20:19:22.512 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3456_vm 2023-07-06T20:19:22.512 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3466 2023-07-06T20:19:22.512 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3466_vm 2023-07-06T20:19:22.512 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3467 2023-07-06T20:19:22.513 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3468 2023-07-06T20:19:22.513 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3501 2023-07-06T20:19:22.513 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3501_vm 2023-07-06T20:19:22.513 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3502 2023-07-06T20:19:22.513 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3503 2023-07-06T20:19:22.513 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3534 2023-07-06T20:19:22.513 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3541 2023-07-06T20:19:22.513 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3541_fsm 2023-07-06T20:19:22.514 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3541_vm 2023-07-06T20:19:22.514 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3542 2023-07-06T20:19:22.514 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3574 2023-07-06T20:19:22.514 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3575 2023-07-06T20:19:22.514 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3576 2023-07-06T20:19:22.514 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3576_vm 2023-07-06T20:19:22.514 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3596 2023-07-06T20:19:22.514 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3596_vm 2023-07-06T20:19:22.515 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3597 2023-07-06T20:19:22.515 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3598 2023-07-06T20:19:22.515 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3598_vm 2023-07-06T20:19:22.515 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3599 2023-07-06T20:19:22.515 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3600 2023-07-06T20:19:22.515 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3600_fsm 2023-07-06T20:19:22.515 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3600_vm 2023-07-06T20:19:22.515 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3601 2023-07-06T20:19:22.516 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3601_fsm 2023-07-06T20:19:22.516 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3601_vm 2023-07-06T20:19:22.516 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3602 2023-07-06T20:19:22.516 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3602_fsm 2023-07-06T20:19:22.516 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3602_vm 2023-07-06T20:19:22.516 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3603 2023-07-06T20:19:22.516 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3603_fsm 2023-07-06T20:19:22.517 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3603_vm 2023-07-06T20:19:22.517 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3604 2023-07-06T20:19:22.517 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3605 2023-07-06T20:19:22.517 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3606 2023-07-06T20:19:22.517 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3607 2023-07-06T20:19:22.517 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3608 2023-07-06T20:19:22.517 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 3609 2023-07-06T20:19:22.517 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3712 2023-07-06T20:19:22.518 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3764 2023-07-06T20:19:22.518 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3764_fsm 2023-07-06T20:19:22.518 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3764_vm 2023-07-06T20:19:22.518 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3766 2023-07-06T20:19:22.518 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3767 2023-07-06T20:19:22.518 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3997 2023-07-06T20:19:22.518 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 5002 2023-07-06T20:19:22.518 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 548 2023-07-06T20:19:22.519 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 549 2023-07-06T20:19:22.519 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6102 2023-07-06T20:19:22.519 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6102_vm 2023-07-06T20:19:22.519 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6104 2023-07-06T20:19:22.519 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6104_vm 2023-07-06T20:19:22.519 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6106 2023-07-06T20:19:22.519 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6106_vm 2023-07-06T20:19:22.519 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6110 2023-07-06T20:19:22.520 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6111 2023-07-06T20:19:22.520 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6112 2023-07-06T20:19:22.520 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6113 2023-07-06T20:19:22.520 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6117 2023-07-06T20:19:22.520 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 826 2023-07-06T20:19:22.520 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 826_vm 2023-07-06T20:19:22.520 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 827 2023-07-06T20:19:22.521 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 828 2023-07-06T20:19:22.521 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 6 20:19 pg_filenode.map 2023-07-06T20:19:22.521 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 PG_VERSION 2023-07-06T20:19:22.521 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.521 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/global: 2023-07-06T20:19:22.521 INFO:teuthology.orchestra.run.smithi069.stdout:total 553 2023-07-06T20:19:22.521 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Jul 6 20:19 . 2023-07-06T20:19:22.521 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.522 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1136 2023-07-06T20:19:22.522 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1136_fsm 2023-07-06T20:19:22.522 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1136_vm 2023-07-06T20:19:22.522 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 1137 2023-07-06T20:19:22.522 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1213 2023-07-06T20:19:22.522 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1213_fsm 2023-07-06T20:19:22.522 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1213_vm 2023-07-06T20:19:22.522 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1214 2023-07-06T20:19:22.523 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1214_fsm 2023-07-06T20:19:22.523 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1214_vm 2023-07-06T20:19:22.523 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 1232 2023-07-06T20:19:22.523 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 1233 2023-07-06T20:19:22.523 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1260 2023-07-06T20:19:22.523 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1260_fsm 2023-07-06T20:19:22.523 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1260_vm 2023-07-06T20:19:22.523 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1261 2023-07-06T20:19:22.524 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1261_fsm 2023-07-06T20:19:22.524 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1261_vm 2023-07-06T20:19:22.524 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1262 2023-07-06T20:19:22.524 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1262_fsm 2023-07-06T20:19:22.524 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1262_vm 2023-07-06T20:19:22.524 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2396 2023-07-06T20:19:22.524 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2396_fsm 2023-07-06T20:19:22.524 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2396_vm 2023-07-06T20:19:22.525 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2397 2023-07-06T20:19:22.525 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2671 2023-07-06T20:19:22.525 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2672 2023-07-06T20:19:22.525 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2676 2023-07-06T20:19:22.525 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2677 2023-07-06T20:19:22.525 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2694 2023-07-06T20:19:22.525 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2695 2023-07-06T20:19:22.526 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2697 2023-07-06T20:19:22.526 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2698 2023-07-06T20:19:22.526 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2846 2023-07-06T20:19:22.526 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2846_vm 2023-07-06T20:19:22.526 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2847 2023-07-06T20:19:22.526 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2964 2023-07-06T20:19:22.526 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2964_vm 2023-07-06T20:19:22.526 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2965 2023-07-06T20:19:22.527 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2966 2023-07-06T20:19:22.527 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2966_vm 2023-07-06T20:19:22.527 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2967 2023-07-06T20:19:22.527 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3592 2023-07-06T20:19:22.527 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3592_vm 2023-07-06T20:19:22.527 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3593 2023-07-06T20:19:22.527 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 4060 2023-07-06T20:19:22.527 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 4060_vm 2023-07-06T20:19:22.528 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 4061 2023-07-06T20:19:22.528 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6000 2023-07-06T20:19:22.528 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6000_vm 2023-07-06T20:19:22.528 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6001 2023-07-06T20:19:22.528 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6002 2023-07-06T20:19:22.528 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6100 2023-07-06T20:19:22.528 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6100_vm 2023-07-06T20:19:22.529 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6114 2023-07-06T20:19:22.529 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6115 2023-07-06T20:19:22.529 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 pg_control 2023-07-06T20:19:22.529 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 6 20:19 pg_filenode.map 2023-07-06T20:19:22.529 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.529 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/log: 2023-07-06T20:19:22.529 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.529 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.530 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.530 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.530 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-07-06T20:19:22.530 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.530 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.530 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.530 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.530 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-07-06T20:19:22.531 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.531 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.531 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.531 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.531 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-07-06T20:19:22.531 INFO:teuthology.orchestra.run.smithi069.stdout:total 1 2023-07-06T20:19:22.531 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 . 2023-07-06T20:19:22.531 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.532 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 mappings 2023-07-06T20:19:22.532 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 6 20:19 replorigin_checkpoint 2023-07-06T20:19:22.532 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 snapshots 2023-07-06T20:19:22.532 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.532 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-07-06T20:19:22.532 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.532 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.533 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 .. 2023-07-06T20:19:22.533 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.533 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-07-06T20:19:22.533 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.533 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.533 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 .. 2023-07-06T20:19:22.533 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.533 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-07-06T20:19:22.534 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.534 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:19 . 2023-07-06T20:19:22.534 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.534 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 members 2023-07-06T20:19:22.534 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 offsets 2023-07-06T20:19:22.534 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.534 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-07-06T20:19:22.534 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2023-07-06T20:19:22.535 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 . 2023-07-06T20:19:22.535 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:19 .. 2023-07-06T20:19:22.535 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 0000 2023-07-06T20:19:22.535 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.535 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-07-06T20:19:22.535 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2023-07-06T20:19:22.535 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 . 2023-07-06T20:19:22.535 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:19 .. 2023-07-06T20:19:22.536 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 0000 2023-07-06T20:19:22.536 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.536 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-07-06T20:19:22.536 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2023-07-06T20:19:22.536 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 . 2023-07-06T20:19:22.536 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.536 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 0000 2023-07-06T20:19:22.537 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.537 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-07-06T20:19:22.537 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.537 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.537 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.537 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.537 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-07-06T20:19:22.537 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.538 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.538 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.538 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.538 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-07-06T20:19:22.538 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.538 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.538 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.538 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.539 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-07-06T20:19:22.539 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.539 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.539 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.539 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.539 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-07-06T20:19:22.539 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.540 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.540 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.540 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.540 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-07-06T20:19:22.540 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2023-07-06T20:19:22.540 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 . 2023-07-06T20:19:22.540 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.541 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 0000 2023-07-06T20:19:22.541 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.541 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-07-06T20:19:22.541 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.541 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.541 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.541 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.542 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-07-06T20:19:22.542 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.542 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.542 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.542 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.542 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-07-06T20:19:22.542 INFO:teuthology.orchestra.run.smithi069.stdout:total 16384 2023-07-06T20:19:22.542 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:19 . 2023-07-06T20:19:22.543 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.543 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:19 000000010000000000000001 2023-07-06T20:19:22.543 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 archive_status 2023-07-06T20:19:22.543 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.543 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-07-06T20:19:22.543 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:19:22.543 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:19:22.543 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:19 .. 2023-07-06T20:19:22.544 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:22.544 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-07-06T20:19:22.544 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2023-07-06T20:19:22.544 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 . 2023-07-06T20:19:22.544 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jul 6 20:19 .. 2023-07-06T20:19:22.544 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 0000 2023-07-06T20:19:22.545 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-07-06T20:19:22.780 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 500 -i' 2023-07-06T20:19:22.865 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:22 smithi069 ceph-mon[126399]: pgmap v329: 97 pgs: 97 active+clean; 14 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2023-07-06T20:19:22.891 INFO:teuthology.orchestra.run.smithi069.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-07-06T20:19:22.898 INFO:teuthology.orchestra.run.smithi069.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-07-06T20:19:22.902 INFO:teuthology.orchestra.run.smithi069.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-07-06T20:19:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-07-06T20:19:23.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:22 smithi156 ceph-mon[129280]: pgmap v329: 97 pgs: 97 active+clean; 14 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2023-07-06T20:19:23.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:22 smithi098 ceph-mon[128968]: pgmap v329: 97 pgs: 97 active+clean; 14 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2023-07-06T20:19:23.187 INFO:teuthology.orchestra.run.smithi069.stderr:creating tables... 2023-07-06T20:19:23.295 INFO:teuthology.orchestra.run.smithi069.stderr:100000 of 50000000 tuples (0%) done (elapsed 0.11 s, remaining 53.35 s) 2023-07-06T20:19:23.584 INFO:teuthology.orchestra.run.smithi069.stderr:200000 of 50000000 tuples (0%) done (elapsed 0.40 s, remaining 98.60 s) 2023-07-06T20:19:23.859 INFO:teuthology.orchestra.run.smithi069.stderr:300000 of 50000000 tuples (0%) done (elapsed 0.67 s, remaining 111.00 s) 2023-07-06T20:19:24.010 INFO:teuthology.orchestra.run.smithi069.stderr:400000 of 50000000 tuples (0%) done (elapsed 0.82 s, remaining 101.85 s) 2023-07-06T20:19:24.293 INFO:teuthology.orchestra.run.smithi069.stderr:500000 of 50000000 tuples (1%) done (elapsed 1.10 s, remaining 109.33 s) 2023-07-06T20:19:24.588 INFO:teuthology.orchestra.run.smithi069.stderr:600000 of 50000000 tuples (1%) done (elapsed 1.40 s, remaining 115.25 s) 2023-07-06T20:19:24.707 INFO:teuthology.orchestra.run.smithi069.stderr:700000 of 50000000 tuples (1%) done (elapsed 1.52 s, remaining 106.98 s) 2023-07-06T20:19:25.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:24 smithi069 ceph-mon[126399]: pgmap v330: 97 pgs: 97 active+clean; 23 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.0 MiB/s wr, 41 op/s 2023-07-06T20:19:25.033 INFO:teuthology.orchestra.run.smithi069.stderr:800000 of 50000000 tuples (1%) done (elapsed 1.84 s, remaining 113.44 s) 2023-07-06T20:19:25.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:24 smithi156 ceph-mon[129280]: pgmap v330: 97 pgs: 97 active+clean; 23 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.0 MiB/s wr, 41 op/s 2023-07-06T20:19:25.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:24 smithi098 ceph-mon[128968]: pgmap v330: 97 pgs: 97 active+clean; 23 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.0 MiB/s wr, 41 op/s 2023-07-06T20:19:25.308 INFO:teuthology.orchestra.run.smithi069.stderr:900000 of 50000000 tuples (1%) done (elapsed 2.12 s, remaining 115.67 s) 2023-07-06T20:19:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:1000000 of 50000000 tuples (2%) done (elapsed 2.22 s, remaining 108.63 s) 2023-07-06T20:19:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:1100000 of 50000000 tuples (2%) done (elapsed 2.51 s, remaining 111.57 s) 2023-07-06T20:19:25.835 INFO:teuthology.orchestra.run.smithi069.stderr:1200000 of 50000000 tuples (2%) done (elapsed 2.65 s, remaining 107.66 s) 2023-07-06T20:19:26.125 INFO:teuthology.orchestra.run.smithi069.stderr:1300000 of 50000000 tuples (2%) done (elapsed 2.94 s, remaining 110.04 s) 2023-07-06T20:19:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:1400000 of 50000000 tuples (2%) done (elapsed 3.22 s, remaining 111.88 s) 2023-07-06T20:19:26.537 INFO:teuthology.orchestra.run.smithi069.stderr:1500000 of 50000000 tuples (3%) done (elapsed 3.35 s, remaining 108.27 s) 2023-07-06T20:19:26.814 INFO:teuthology.orchestra.run.smithi069.stderr:1600000 of 50000000 tuples (3%) done (elapsed 3.63 s, remaining 109.66 s) 2023-07-06T20:19:26.976 INFO:teuthology.orchestra.run.smithi069.stderr:1700000 of 50000000 tuples (3%) done (elapsed 3.79 s, remaining 107.62 s) 2023-07-06T20:19:27.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:26 smithi069 ceph-mon[126399]: pgmap v331: 97 pgs: 97 active+clean; 62 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 8.3 MiB/s wr, 92 op/s 2023-07-06T20:19:27.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:26 smithi156 ceph-mon[129280]: pgmap v331: 97 pgs: 97 active+clean; 62 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 8.3 MiB/s wr, 92 op/s 2023-07-06T20:19:27.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:26 smithi098 ceph-mon[128968]: pgmap v331: 97 pgs: 97 active+clean; 62 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 8.3 MiB/s wr, 92 op/s 2023-07-06T20:19:27.259 INFO:teuthology.orchestra.run.smithi069.stderr:1800000 of 50000000 tuples (3%) done (elapsed 4.07 s, remaining 108.99 s) 2023-07-06T20:19:27.558 INFO:teuthology.orchestra.run.smithi069.stderr:1900000 of 50000000 tuples (3%) done (elapsed 4.37 s, remaining 110.63 s) 2023-07-06T20:19:27.731 INFO:teuthology.orchestra.run.smithi069.stderr:2000000 of 50000000 tuples (4%) done (elapsed 4.54 s, remaining 109.04 s) 2023-07-06T20:19:28.001 INFO:teuthology.orchestra.run.smithi069.stderr:2100000 of 50000000 tuples (4%) done (elapsed 4.81 s, remaining 109.77 s) 2023-07-06T20:19:28.288 INFO:teuthology.orchestra.run.smithi069.stderr:2200000 of 50000000 tuples (4%) done (elapsed 5.10 s, remaining 110.80 s) 2023-07-06T20:19:28.439 INFO:teuthology.orchestra.run.smithi069.stderr:2300000 of 50000000 tuples (4%) done (elapsed 5.25 s, remaining 108.89 s) 2023-07-06T20:19:28.712 INFO:teuthology.orchestra.run.smithi069.stderr:2400000 of 50000000 tuples (4%) done (elapsed 5.52 s, remaining 109.56 s) 2023-07-06T20:19:29.002 INFO:teuthology.orchestra.run.smithi069.stderr:2500000 of 50000000 tuples (5%) done (elapsed 5.81 s, remaining 110.46 s) 2023-07-06T20:19:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:28 smithi069 ceph-mon[126399]: pgmap v332: 97 pgs: 97 active+clean; 135 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 20 MiB/s wr, 119 op/s 2023-07-06T20:19:29.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:28 smithi069 ceph-mon[126399]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:19:29.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:28 smithi156 ceph-mon[129280]: pgmap v332: 97 pgs: 97 active+clean; 135 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 20 MiB/s wr, 119 op/s 2023-07-06T20:19:29.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:28 smithi156 ceph-mon[129280]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:19:29.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:28 smithi098 ceph-mon[128968]: pgmap v332: 97 pgs: 97 active+clean; 135 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 20 MiB/s wr, 119 op/s 2023-07-06T20:19:29.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:28 smithi098 ceph-mon[128968]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:19:29.128 INFO:teuthology.orchestra.run.smithi069.stderr:2600000 of 50000000 tuples (5%) done (elapsed 5.94 s, remaining 108.29 s) 2023-07-06T20:19:29.450 INFO:teuthology.orchestra.run.smithi069.stderr:2700000 of 50000000 tuples (5%) done (elapsed 6.26 s, remaining 109.70 s) 2023-07-06T20:19:29.740 INFO:teuthology.orchestra.run.smithi069.stderr:2800000 of 50000000 tuples (5%) done (elapsed 6.55 s, remaining 110.45 s) 2023-07-06T20:19:29.873 INFO:teuthology.orchestra.run.smithi069.stderr:2900000 of 50000000 tuples (5%) done (elapsed 6.68 s, remaining 108.57 s) 2023-07-06T20:19:30.144 INFO:teuthology.orchestra.run.smithi069.stderr:3000000 of 50000000 tuples (6%) done (elapsed 6.96 s, remaining 108.97 s) 2023-07-06T20:19:30.295 INFO:teuthology.orchestra.run.smithi069.stderr:3100000 of 50000000 tuples (6%) done (elapsed 7.11 s, remaining 107.52 s) 2023-07-06T20:19:30.549 INFO:teuthology.orchestra.run.smithi069.stderr:3200000 of 50000000 tuples (6%) done (elapsed 7.36 s, remaining 107.65 s) 2023-07-06T20:19:30.822 INFO:teuthology.orchestra.run.smithi069.stderr:3300000 of 50000000 tuples (6%) done (elapsed 7.63 s, remaining 108.03 s) 2023-07-06T20:19:31.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:30 smithi069 ceph-mon[126399]: pgmap v333: 97 pgs: 97 active+clean; 187 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 29 MiB/s wr, 122 op/s 2023-07-06T20:19:31.106 INFO:teuthology.orchestra.run.smithi069.stderr:3400000 of 50000000 tuples (6%) done (elapsed 7.92 s, remaining 108.52 s) 2023-07-06T20:19:31.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:30 smithi156 ceph-mon[129280]: pgmap v333: 97 pgs: 97 active+clean; 187 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 29 MiB/s wr, 122 op/s 2023-07-06T20:19:31.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:30 smithi098 ceph-mon[128968]: pgmap v333: 97 pgs: 97 active+clean; 187 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 29 MiB/s wr, 122 op/s 2023-07-06T20:19:31.218 INFO:teuthology.orchestra.run.smithi069.stderr:3500000 of 50000000 tuples (7%) done (elapsed 8.03 s, remaining 106.68 s) 2023-07-06T20:19:31.494 INFO:teuthology.orchestra.run.smithi069.stderr:3600000 of 50000000 tuples (7%) done (elapsed 8.31 s, remaining 107.04 s) 2023-07-06T20:19:31.817 INFO:teuthology.orchestra.run.smithi069.stderr:3700000 of 50000000 tuples (7%) done (elapsed 8.63 s, remaining 107.97 s) 2023-07-06T20:19:31.921 INFO:teuthology.orchestra.run.smithi069.stderr:3800000 of 50000000 tuples (7%) done (elapsed 8.73 s, remaining 106.17 s) 2023-07-06T20:19:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:3900000 of 50000000 tuples (7%) done (elapsed 9.02 s, remaining 106.65 s) 2023-07-06T20:19:32.358 INFO:teuthology.orchestra.run.smithi069.stderr:4000000 of 50000000 tuples (8%) done (elapsed 9.17 s, remaining 105.45 s) 2023-07-06T20:19:32.627 INFO:teuthology.orchestra.run.smithi069.stderr:4100000 of 50000000 tuples (8%) done (elapsed 9.44 s, remaining 105.66 s) 2023-07-06T20:19:32.792 INFO:teuthology.orchestra.run.smithi069.stderr:4200000 of 50000000 tuples (8%) done (elapsed 9.60 s, remaining 104.72 s) 2023-07-06T20:19:33.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:32 smithi069 ceph-mon[126399]: pgmap v334: 97 pgs: 97 active+clean; 303 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 48 MiB/s wr, 127 op/s 2023-07-06T20:19:33.044 INFO:teuthology.orchestra.run.smithi069.stderr:4300000 of 50000000 tuples (8%) done (elapsed 9.86 s, remaining 104.75 s) 2023-07-06T20:19:33.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:32 smithi156 ceph-mon[129280]: pgmap v334: 97 pgs: 97 active+clean; 303 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 48 MiB/s wr, 127 op/s 2023-07-06T20:19:33.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:32 smithi098 ceph-mon[128968]: pgmap v334: 97 pgs: 97 active+clean; 303 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 48 MiB/s wr, 127 op/s 2023-07-06T20:19:33.307 INFO:teuthology.orchestra.run.smithi069.stderr:4400000 of 50000000 tuples (8%) done (elapsed 10.12 s, remaining 104.87 s) 2023-07-06T20:19:33.475 INFO:teuthology.orchestra.run.smithi069.stderr:4500000 of 50000000 tuples (9%) done (elapsed 10.29 s, remaining 104.01 s) 2023-07-06T20:19:33.742 INFO:teuthology.orchestra.run.smithi069.stderr:4600000 of 50000000 tuples (9%) done (elapsed 10.55 s, remaining 104.16 s) 2023-07-06T20:19:33.999 INFO:teuthology.orchestra.run.smithi069.stderr:4700000 of 50000000 tuples (9%) done (elapsed 10.81 s, remaining 104.20 s) 2023-07-06T20:19:34.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:33 smithi069 ceph-mon[126399]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:19:34.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:33 smithi156 ceph-mon[129280]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:19:34.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:33 smithi098 ceph-mon[128968]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:19:34.166 INFO:teuthology.orchestra.run.smithi069.stderr:4800000 of 50000000 tuples (9%) done (elapsed 10.98 s, remaining 103.37 s) 2023-07-06T20:19:34.421 INFO:teuthology.orchestra.run.smithi069.stderr:4900000 of 50000000 tuples (9%) done (elapsed 11.23 s, remaining 103.39 s) 2023-07-06T20:19:34.687 INFO:teuthology.orchestra.run.smithi069.stderr:5000000 of 50000000 tuples (10%) done (elapsed 11.50 s, remaining 103.48 s) 2023-07-06T20:19:34.862 INFO:teuthology.orchestra.run.smithi069.stderr:5100000 of 50000000 tuples (10%) done (elapsed 11.67 s, remaining 102.78 s) 2023-07-06T20:19:35.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:34 smithi069 ceph-mon[126399]: pgmap v335: 97 pgs: 97 active+clean; 367 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 57 MiB/s wr, 138 op/s 2023-07-06T20:19:35.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:34 smithi156 ceph-mon[129280]: pgmap v335: 97 pgs: 97 active+clean; 367 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 57 MiB/s wr, 138 op/s 2023-07-06T20:19:35.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:34 smithi098 ceph-mon[128968]: pgmap v335: 97 pgs: 97 active+clean; 367 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 57 MiB/s wr, 138 op/s 2023-07-06T20:19:35.133 INFO:teuthology.orchestra.run.smithi069.stderr:5200000 of 50000000 tuples (10%) done (elapsed 11.95 s, remaining 102.91 s) 2023-07-06T20:19:35.450 INFO:teuthology.orchestra.run.smithi069.stderr:5300000 of 50000000 tuples (10%) done (elapsed 12.26 s, remaining 103.41 s) 2023-07-06T20:19:35.640 INFO:teuthology.orchestra.run.smithi069.stderr:5400000 of 50000000 tuples (10%) done (elapsed 12.45 s, remaining 102.84 s) 2023-07-06T20:19:35.920 INFO:teuthology.orchestra.run.smithi069.stderr:5500000 of 50000000 tuples (11%) done (elapsed 12.73 s, remaining 103.02 s) 2023-07-06T20:19:36.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:35 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:19:36.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:35 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:19:36.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:35 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:19:36.237 INFO:teuthology.orchestra.run.smithi069.stderr:5600000 of 50000000 tuples (11%) done (elapsed 13.05 s, remaining 103.46 s) 2023-07-06T20:19:36.345 INFO:teuthology.orchestra.run.smithi069.stderr:5700000 of 50000000 tuples (11%) done (elapsed 13.16 s, remaining 102.25 s) 2023-07-06T20:19:36.697 INFO:teuthology.orchestra.run.smithi069.stderr:5800000 of 50000000 tuples (11%) done (elapsed 13.51 s, remaining 102.95 s) 2023-07-06T20:19:37.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:36 smithi069 ceph-mon[126399]: pgmap v336: 97 pgs: 97 active+clean; 451 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 70 MiB/s wr, 114 op/s 2023-07-06T20:19:37.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:36 smithi156 ceph-mon[129280]: pgmap v336: 97 pgs: 97 active+clean; 451 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 70 MiB/s wr, 114 op/s 2023-07-06T20:19:37.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:36 smithi098 ceph-mon[128968]: pgmap v336: 97 pgs: 97 active+clean; 451 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 70 MiB/s wr, 114 op/s 2023-07-06T20:19:37.371 INFO:teuthology.orchestra.run.smithi069.stderr:5900000 of 50000000 tuples (11%) done (elapsed 14.17 s, remaining 105.91 s) 2023-07-06T20:19:37.820 INFO:teuthology.orchestra.run.smithi069.stderr:6000000 of 50000000 tuples (12%) done (elapsed 14.63 s, remaining 107.30 s) 2023-07-06T20:19:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:6100000 of 50000000 tuples (12%) done (elapsed 15.09 s, remaining 108.59 s) 2023-07-06T20:19:38.383 INFO:teuthology.orchestra.run.smithi069.stderr:6200000 of 50000000 tuples (12%) done (elapsed 15.19 s, remaining 107.34 s) 2023-07-06T20:19:38.715 INFO:teuthology.orchestra.run.smithi069.stderr:6300000 of 50000000 tuples (12%) done (elapsed 15.53 s, remaining 107.70 s) 2023-07-06T20:19:39.040 INFO:teuthology.orchestra.run.smithi069.stderr:6400000 of 50000000 tuples (12%) done (elapsed 15.85 s, remaining 107.99 s) 2023-07-06T20:19:39.161 INFO:teuthology.orchestra.run.smithi069.stderr:6500000 of 50000000 tuples (13%) done (elapsed 15.97 s, remaining 106.89 s) 2023-07-06T20:19:39.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:38 smithi069 ceph-mon[126399]: pgmap v337: 97 pgs: 97 active+clean; 640 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 89 MiB/s wr, 81 op/s 2023-07-06T20:19:39.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:38 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:39.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:38 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:39.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:38 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:39.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:38 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:39.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:38 smithi156 ceph-mon[129280]: pgmap v337: 97 pgs: 97 active+clean; 640 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 89 MiB/s wr, 81 op/s 2023-07-06T20:19:39.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:39.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:39.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:39.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:38 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:39.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:38 smithi098 ceph-mon[128968]: pgmap v337: 97 pgs: 97 active+clean; 640 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 89 MiB/s wr, 81 op/s 2023-07-06T20:19:39.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:38 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:39.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:38 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:39.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:38 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:39.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:38 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:39.450 INFO:teuthology.orchestra.run.smithi069.stderr:6600000 of 50000000 tuples (13%) done (elapsed 16.26 s, remaining 106.93 s) 2023-07-06T20:19:39.635 INFO:teuthology.orchestra.run.smithi069.stderr:6700000 of 50000000 tuples (13%) done (elapsed 16.45 s, remaining 106.29 s) 2023-07-06T20:19:39.911 INFO:teuthology.orchestra.run.smithi069.stderr:6800000 of 50000000 tuples (13%) done (elapsed 16.72 s, remaining 106.24 s) 2023-07-06T20:19:40.203 INFO:teuthology.orchestra.run.smithi069.stderr:6900000 of 50000000 tuples (13%) done (elapsed 17.01 s, remaining 106.28 s) 2023-07-06T20:19:40.369 INFO:teuthology.orchestra.run.smithi069.stderr:7000000 of 50000000 tuples (14%) done (elapsed 17.18 s, remaining 105.54 s) 2023-07-06T20:19:40.643 INFO:teuthology.orchestra.run.smithi069.stderr:7100000 of 50000000 tuples (14%) done (elapsed 17.45 s, remaining 105.47 s) 2023-07-06T20:19:40.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:40.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:40.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:40 smithi069 ceph-mon[126399]: pgmap v338: 97 pgs: 97 active+clean; 961 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 109 MiB/s wr, 63 op/s 2023-07-06T20:19:40.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:19:40.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:19:40.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:40.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:40 smithi156 ceph-mon[129280]: pgmap v338: 97 pgs: 97 active+clean; 961 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 109 MiB/s wr, 63 op/s 2023-07-06T20:19:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:19:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:19:40.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:40.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:40.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:40.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:40 smithi098 ceph-mon[128968]: pgmap v338: 97 pgs: 97 active+clean; 961 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 109 MiB/s wr, 63 op/s 2023-07-06T20:19:40.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:19:40.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:19:40.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:19:40.908 INFO:teuthology.orchestra.run.smithi069.stderr:7200000 of 50000000 tuples (14%) done (elapsed 17.72 s, remaining 105.34 s) 2023-07-06T20:19:41.065 INFO:teuthology.orchestra.run.smithi069.stderr:7300000 of 50000000 tuples (14%) done (elapsed 17.88 s, remaining 104.57 s) 2023-07-06T20:19:41.321 INFO:teuthology.orchestra.run.smithi069.stderr:7400000 of 50000000 tuples (14%) done (elapsed 18.13 s, remaining 104.38 s) 2023-07-06T20:19:41.578 INFO:teuthology.orchestra.run.smithi069.stderr:7500000 of 50000000 tuples (15%) done (elapsed 18.39 s, remaining 104.21 s) 2023-07-06T20:19:41.735 INFO:teuthology.orchestra.run.smithi069.stderr:7600000 of 50000000 tuples (15%) done (elapsed 18.55 s, remaining 103.47 s) 2023-07-06T20:19:42.001 INFO:teuthology.orchestra.run.smithi069.stderr:7700000 of 50000000 tuples (15%) done (elapsed 18.81 s, remaining 103.35 s) 2023-07-06T20:19:42.269 INFO:teuthology.orchestra.run.smithi069.stderr:7800000 of 50000000 tuples (15%) done (elapsed 19.08 s, remaining 103.23 s) 2023-07-06T20:19:42.440 INFO:teuthology.orchestra.run.smithi069.stderr:7900000 of 50000000 tuples (15%) done (elapsed 19.25 s, remaining 102.60 s) 2023-07-06T20:19:42.706 INFO:teuthology.orchestra.run.smithi069.stderr:8000000 of 50000000 tuples (16%) done (elapsed 19.52 s, remaining 102.47 s) 2023-07-06T20:19:42.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:42 smithi098 ceph-mon[128968]: pgmap v339: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 140 MiB/s wr, 71 op/s 2023-07-06T20:19:42.975 INFO:teuthology.orchestra.run.smithi069.stderr:8100000 of 50000000 tuples (16%) done (elapsed 19.79 s, remaining 102.36 s) 2023-07-06T20:19:43.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:42 smithi069 ceph-mon[126399]: pgmap v339: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 140 MiB/s wr, 71 op/s 2023-07-06T20:19:43.087 INFO:teuthology.orchestra.run.smithi069.stderr:8200000 of 50000000 tuples (16%) done (elapsed 19.90 s, remaining 101.43 s) 2023-07-06T20:19:43.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:42 smithi156 ceph-mon[129280]: pgmap v339: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 140 MiB/s wr, 71 op/s 2023-07-06T20:19:43.414 INFO:teuthology.orchestra.run.smithi069.stderr:8300000 of 50000000 tuples (16%) done (elapsed 20.23 s, remaining 101.61 s) 2023-07-06T20:19:43.697 INFO:teuthology.orchestra.run.smithi069.stderr:8400000 of 50000000 tuples (16%) done (elapsed 20.51 s, remaining 101.57 s) 2023-07-06T20:19:43.822 INFO:teuthology.orchestra.run.smithi069.stderr:8500000 of 50000000 tuples (17%) done (elapsed 20.63 s, remaining 100.74 s) 2023-07-06T20:19:43.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:43 smithi098 ceph-mon[128968]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:19:44.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:43 smithi069 ceph-mon[126399]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:19:44.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:43 smithi156 ceph-mon[129280]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:19:44.145 INFO:teuthology.orchestra.run.smithi069.stderr:8600000 of 50000000 tuples (17%) done (elapsed 20.96 s, remaining 100.89 s) 2023-07-06T20:19:44.250 INFO:teuthology.orchestra.run.smithi069.stderr:8700000 of 50000000 tuples (17%) done (elapsed 21.06 s, remaining 99.98 s) 2023-07-06T20:19:44.519 INFO:teuthology.orchestra.run.smithi069.stderr:8800000 of 50000000 tuples (17%) done (elapsed 21.33 s, remaining 99.87 s) 2023-07-06T20:19:44.727 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:19:44.813 INFO:teuthology.orchestra.run.smithi069.stderr:8900000 of 50000000 tuples (17%) done (elapsed 21.62 s, remaining 99.86 s) 2023-07-06T20:19:44.953 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.950+0000 7fb6febff700 1 -- 172.21.15.69:0/4202931999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6f8074600 msgr2=0x7fb6f8072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:44.953 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.950+0000 7fb6febff700 1 --2- 172.21.15.69:0/4202931999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6f8074600 0x7fb6f8072a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009990 tx=0x7fb6f4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.954 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.950+0000 7fb6febff700 1 -- 172.21.15.69:0/4202931999 shutdown_connections 2023-07-06T20:19:44.954 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.950+0000 7fb6febff700 1 --2- 172.21.15.69:0/4202931999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6f8107ad0 0x7fb6f8109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.954 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.950+0000 7fb6febff700 1 --2- 172.21.15.69:0/4202931999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6f8074600 0x7fb6f8072a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.954 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.950+0000 7fb6febff700 1 --2- 172.21.15.69:0/4202931999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f8073020 0x7fb6f8073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.950+0000 7fb6febff700 1 -- 172.21.15.69:0/4202931999 >> 172.21.15.69:0/4202931999 conn(0x7fb6f80fb260 msgr2=0x7fb6f80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:44.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.951+0000 7fb6febff700 1 -- 172.21.15.69:0/4202931999 shutdown_connections 2023-07-06T20:19:44.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.951+0000 7fb6febff700 1 -- 172.21.15.69:0/4202931999 wait complete. 2023-07-06T20:19:44.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6febff700 1 Processor -- start 2023-07-06T20:19:44.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6febff700 1 -- start start 2023-07-06T20:19:44.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6febff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6f8073020 0x7fb6f8197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6febff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f8074600 0x7fb6f81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.959 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6febff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6f8107ad0 0x7fb6f819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.959 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6febff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f819f820 con 0x7fb6f8074600 2023-07-06T20:19:44.959 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6febff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f819f990 con 0x7fb6f8107ad0 2023-07-06T20:19:44.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6febff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f819fb50 con 0x7fb6f8073020 2023-07-06T20:19:44.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6fc99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6f8073020 0x7fb6f8197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6effff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f8074600 0x7fb6f81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6fc99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6f8073020 0x7fb6f8197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43068/0 (socket says 172.21.15.69:43068) 2023-07-06T20:19:44.961 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6fc99b700 1 -- 172.21.15.69:0/2993822 learned_addr learned my addr 172.21.15.69:0/2993822 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:44.961 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.952+0000 7fb6fd19c700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6f8107ad0 0x7fb6f819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.961 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.953+0000 7fb6fc99b700 1 -- 172.21.15.69:0/2993822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6f8107ad0 msgr2=0x7fb6f819f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:44.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.953+0000 7fb6fc99b700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6f8107ad0 0x7fb6f819f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.953+0000 7fb6fc99b700 1 -- 172.21.15.69:0/2993822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f8074600 msgr2=0x7fb6f81981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:44.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.953+0000 7fb6fc99b700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f8074600 0x7fb6f81981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.953+0000 7fb6fc99b700 1 -- 172.21.15.69:0/2993822 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40095f0 con 0x7fb6f8073020 2023-07-06T20:19:44.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.953+0000 7fb6fc99b700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6f8073020 0x7fb6f8197cb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b680 tx=0x7fb6f4005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:44.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.953+0000 7fb6edffb700 1 -- 172.21.15.69:0/2993822 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f401d070 con 0x7fb6f8073020 2023-07-06T20:19:44.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.953+0000 7fb6edffb700 1 -- 172.21.15.69:0/2993822 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f4022470 con 0x7fb6f8073020 2023-07-06T20:19:44.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.953+0000 7fb6febff700 1 -- 172.21.15.69:0/2993822 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f819fdd0 con 0x7fb6f8073020 2023-07-06T20:19:44.964 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.953+0000 7fb6edffb700 1 -- 172.21.15.69:0/2993822 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4017650 con 0x7fb6f8073020 2023-07-06T20:19:44.964 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.953+0000 7fb6febff700 1 -- 172.21.15.69:0/2993822 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f81a01a0 con 0x7fb6f8073020 2023-07-06T20:19:44.964 INFO:teuthology.orchestra.run.smithi069.stderr:9000000 of 50000000 tuples (18%) done (elapsed 21.77 s, remaining 99.17 s) 2023-07-06T20:19:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.957+0000 7fb7124eeb80 1 Processor -- start 2023-07-06T20:19:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb7124eeb80 1 -- start start 2023-07-06T20:19:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb6edffb700 1 -- 172.21.15.69:0/2993822 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb6f40225e0 con 0x7fb6f8073020 2023-07-06T20:19:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb7124eeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fdcf20 0x558b60fdd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb7124eeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60fdde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb7124eeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fde390 0x558b60fde830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb7124eeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b60ee1cb0 con 0x558b60fdcf20 2023-07-06T20:19:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb7124eeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b60f34a40 con 0x558b60fde390 2023-07-06T20:19:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb7124eeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b60f2ded0 con 0x558b60fdd870 2023-07-06T20:19:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb6fd19c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fdcf20 0x558b60fdd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb6fc99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60fdde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb6effff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fde390 0x558b60fde830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb6effff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fde390 0x558b60fde830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:36802/0 (socket says 172.21.15.69:36802) 2023-07-06T20:19:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb6fd19c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fdcf20 0x558b60fdd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47224/0 (socket says 172.21.15.69:47224) 2023-07-06T20:19:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb6effff700 1 -- 172.21.15.69:0/899674317 learned_addr learned my addr 172.21.15.69:0/899674317 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.958+0000 7fb6fc99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60fdde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43078/0 (socket says 172.21.15.69:43078) 2023-07-06T20:19:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.959+0000 7fb6effff700 1 -- 172.21.15.69:0/899674317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 msgr2=0x558b60fdde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.959+0000 7fb6effff700 1 --2- 172.21.15.69:0/899674317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60fdde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.959+0000 7fb6effff700 1 -- 172.21.15.69:0/899674317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fdcf20 msgr2=0x558b60fdd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.959+0000 7fb6effff700 1 --2- 172.21.15.69:0/899674317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fdcf20 0x558b60fdd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.959+0000 7fb6effff700 1 -- 172.21.15.69:0/899674317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b60d5bde0 con 0x558b60fde390 2023-07-06T20:19:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.959+0000 7fb6effff700 1 --2- 172.21.15.69:0/899674317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fde390 0x558b60fde830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009d20 tx=0x7fb6e8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2e7c0c1cea3c2c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:44.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.959+0000 7fb6deffd700 1 -- 172.21.15.69:0/899674317 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8019630 con 0x558b60fde390 2023-07-06T20:19:44.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.960+0000 7fb6deffd700 1 -- 172.21.15.69:0/899674317 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6e8019c50 con 0x558b60fde390 2023-07-06T20:19:44.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.960+0000 7fb6deffd700 1 -- 172.21.15.69:0/899674317 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800ec00 con 0x558b60fde390 2023-07-06T20:19:44.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.960+0000 7fb6edffb700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb6e00a1150 0x7fb6e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.960+0000 7fb6edffb700 1 -- 172.21.15.69:0/2993822 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fb6f40cde50 con 0x7fb6f8073020 2023-07-06T20:19:44.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.960+0000 7fb6fd19c700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb6e00a1150 0x7fb6e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.960+0000 7fb7124eeb80 1 -- 172.21.15.69:0/899674317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fde390 msgr2=0x558b60fde830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:44.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.960+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/899674317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fde390 0x558b60fde830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009d20 tx=0x7fb6e8011720 comp rx=0 tx=0).stop 2023-07-06T20:19:44.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.960+0000 7fb7124eeb80 1 -- 172.21.15.69:0/899674317 shutdown_connections 2023-07-06T20:19:44.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.960+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/899674317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fde390 0x558b60fde830 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.960+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/899674317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fdcf20 0x558b60fdd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.960+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/899674317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60fdde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.960+0000 7fb7124eeb80 1 -- 172.21.15.69:0/899674317 >> 172.21.15.69:0/899674317 conn(0x558b60fd2640 msgr2=0x558b60fd79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:44.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.961+0000 7fb7124eeb80 1 -- 172.21.15.69:0/899674317 shutdown_connections 2023-07-06T20:19:44.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.961+0000 7fb7124eeb80 1 -- 172.21.15.69:0/899674317 wait complete. 2023-07-06T20:19:44.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.961+0000 7fb6fd19c700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb6e00a1150 0x7fb6e00a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00127f0 tx=0x7fb6f000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:44.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.961+0000 7fb7124eeb80 1 -- start start 2023-07-06T20:19:44.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.961+0000 7fb7124eeb80 1 Processor -- start 2023-07-06T20:19:44.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb7124eeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 0x558b60fe6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb7124eeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60fe6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb7124eeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fede00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb7124eeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b60ecdac0 con 0x558b60fde390 2023-07-06T20:19:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb7124eeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b60ecd7c0 con 0x558b60fdcf20 2023-07-06T20:19:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb7124eeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b60db1920 con 0x558b60fdd870 2023-07-06T20:19:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb7124eeb80 10 client.0 _ll_register_callbacks cb 0x558b60ecb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb6fd19c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb6fc99b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 0x558b60fe6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb6fc99b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 0x558b60fe6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:36810/0 (socket says 172.21.15.69:36810) 2023-07-06T20:19:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb6fd19c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fede00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47230/0 (socket says 172.21.15.69:47230) 2023-07-06T20:19:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb6fc99b700 1 -- 172.21.15.69:0/3412151699 learned_addr learned my addr 172.21.15.69:0/3412151699 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb6effff700 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60fe6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb6fc99b700 1 -- 172.21.15.69:0/3412151699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 msgr2=0x558b60fe6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb6fc99b700 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60fe6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb6fc99b700 1 -- 172.21.15.69:0/3412151699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 msgr2=0x558b60fede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.962+0000 7fb6fc99b700 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.963+0000 7fb6fc99b700 1 -- 172.21.15.69:0/3412151699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e8009980 con 0x558b60fdcf20 2023-07-06T20:19:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.963+0000 7fb6effff700 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60fe6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:19:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.963+0000 7fb6fc99b700 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 0x558b60fe6880 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6f409af60 tx=0x7fb6f4022bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5267348621b71bdb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.963+0000 7fb6d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:19:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.963+0000 7fb6d7fff700 1 -- 172.21.15.69:0/3412151699 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f409a950 con 0x558b60fdcf20 2023-07-06T20:19:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.963+0000 7fb7124eeb80 1 -- 172.21.15.69:0/3412151699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b60e874f0 con 0x558b60fdcf20 2023-07-06T20:19:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.963+0000 7fb6d7fff700 1 -- 172.21.15.69:0/3412151699 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fb6f409b260 con 0x558b60fdcf20 2023-07-06T20:19:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.963+0000 7fb6d7fff700 1 -- 172.21.15.69:0/3412151699 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6f409b030 con 0x558b60fdcf20 2023-07-06T20:19:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.963+0000 7fb7124eeb80 10 client.25343 fetch_fsmap learned FSMap version 26 2023-07-06T20:19:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.963+0000 7fb7124eeb80 1 -- 172.21.15.69:0/3412151699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558b60d44c60 con 0x558b60fdcf20 2023-07-06T20:19:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.963+0000 7fb6d7fff700 1 -- 172.21.15.69:0/3412151699 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b60e874f0 con 0x558b60fdcf20 2023-07-06T20:19:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.964+0000 7fb6d7fff700 1 -- 172.21.15.69:0/3412151699 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fb6f40a3190 con 0x558b60fdcf20 2023-07-06T20:19:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.964+0000 7fb7124eeb80 10 client.25343 fetch_fsmap finished waiting for FSMap version 26 2023-07-06T20:19:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.964+0000 7fb7124eeb80 10 client.25343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:19:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.964+0000 7fb7124eeb80 20 client.25343 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.964+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558b60ff2280 0x558b60ff4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.979 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.964+0000 7fb7124eeb80 4 client.25343 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:19:44.979 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.964+0000 7fb7124eeb80 1 -- 172.21.15.69:0/3412151699 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b60fabd60 con 0x558b60ff2280 2023-07-06T20:19:44.979 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.964+0000 7fb6effff700 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558b60ff2280 0x558b60ff4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.979 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.964+0000 7fb6d7fff700 10 client.25343 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:44.979 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.964+0000 7fb6effff700 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558b60ff2280 0x558b60ff4660 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30aff9569f760bbe server_cookie=dbc1de80bae9c632 in_seq=0 out_seq=0 2023-07-06T20:19:44.979 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.965+0000 7fb6d7fff700 1 -- 172.21.15.69:0/3412151699 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558b60fabd60 con 0x558b60ff2280 2023-07-06T20:19:44.979 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.965+0000 7fb6d7fff700 10 client.25343 handle_command_reply: tid=0 2023-07-06T20:19:44.980 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.965+0000 7fb7124eeb80 1 client.25343 shutdown 2023-07-06T20:19:44.980 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.965+0000 7fb7124eeb80 1 -- 172.21.15.69:0/3412151699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 msgr2=0x558b60fe6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:44.980 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.965+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 0x558b60fe6880 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6f409af60 tx=0x7fb6f4022bc0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.980 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.965+0000 7fb7124eeb80 1 -- 172.21.15.69:0/3412151699 shutdown_connections 2023-07-06T20:19:44.980 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.965+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558b60ff2280 0x558b60ff4660 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.980 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.965+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.980 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.965+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 0x558b60fe6880 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.981 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.965+0000 7fb6d7fff700 0 client.25343 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:44.981 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.965+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/3412151699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60fe6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.981 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.965+0000 7fb7124eeb80 1 -- 172.21.15.69:0/3412151699 >> 172.21.15.69:0/3412151699 conn(0x558b60fd2640 msgr2=0x558b60fd70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:44.981 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.966+0000 7fb7124eeb80 1 -- 172.21.15.69:0/3412151699 shutdown_connections 2023-07-06T20:19:44.981 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.966+0000 7fb7124eeb80 1 -- 172.21.15.69:0/3412151699 wait complete. 2023-07-06T20:19:44.981 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.966+0000 7fb7124eeb80 20 client.25343 trim_cache size 0 max 16384 2023-07-06T20:19:44.986 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.985+0000 7fb7124eeb80 1 Processor -- start 2023-07-06T20:19:44.987 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.985+0000 7fb7124eeb80 1 -- start start 2023-07-06T20:19:44.987 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.986+0000 7fb7124eeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 0x558b60fe5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.988 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.986+0000 7fb7124eeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60feddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.988 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.986+0000 7fb6fc99b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 0x558b60fe5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.989 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.986+0000 7fb7124eeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.989 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.986+0000 7fb6fc99b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 0x558b60fe5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:36818/0 (socket says 172.21.15.69:36818) 2023-07-06T20:19:44.989 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.986+0000 7fb6fc99b700 1 -- 172.21.15.69:0/4251582105 learned_addr learned my addr 172.21.15.69:0/4251582105 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:44.989 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.986+0000 7fb7124eeb80 1 -- 172.21.15.69:0/4251582105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b60fd9460 con 0x558b60fde390 2023-07-06T20:19:44.989 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.986+0000 7fb6effff700 1 --2- 172.21.15.69:0/4251582105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60feddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.989 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.986+0000 7fb7124eeb80 1 -- 172.21.15.69:0/4251582105 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b60fddc40 con 0x558b60fdcf20 2023-07-06T20:19:44.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.986+0000 7fb7124eeb80 1 -- 172.21.15.69:0/4251582105 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b60fd8d40 con 0x558b60fdd870 2023-07-06T20:19:44.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.986+0000 7fb6fd19c700 1 --2- 172.21.15.69:0/4251582105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.987+0000 7fb6fc99b700 1 -- 172.21.15.69:0/4251582105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 msgr2=0x558b60feddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:44.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.987+0000 7fb6fc99b700 1 --2- 172.21.15.69:0/4251582105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60feddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.987+0000 7fb6fc99b700 1 -- 172.21.15.69:0/4251582105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 msgr2=0x558b60fee6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:44.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.987+0000 7fb6fc99b700 1 --2- 172.21.15.69:0/4251582105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fee6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.987+0000 7fb6fc99b700 1 -- 172.21.15.69:0/4251582105 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b60d82250 con 0x558b60fdcf20 2023-07-06T20:19:44.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.987+0000 7fb6fc99b700 1 --2- 172.21.15.69:0/4251582105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 0x558b60fe5600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4000c00 tx=0x7fb6f40ce090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d7b14dee147545 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:44.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.987+0000 7fb6df7fe700 1 -- 172.21.15.69:0/4251582105 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f409b030 con 0x558b60fdcf20 2023-07-06T20:19:44.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.987+0000 7fb6df7fe700 1 -- 172.21.15.69:0/4251582105 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f40a3670 con 0x558b60fdcf20 2023-07-06T20:19:44.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.987+0000 7fb6df7fe700 1 -- 172.21.15.69:0/4251582105 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f409b260 con 0x558b60fdcf20 2023-07-06T20:19:44.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.988+0000 7fb7124eeb80 1 -- 172.21.15.69:0/4251582105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 msgr2=0x558b60fe5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:44.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.988+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/4251582105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 0x558b60fe5600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4000c00 tx=0x7fb6f40ce090 comp rx=0 tx=0).stop 2023-07-06T20:19:44.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.988+0000 7fb7124eeb80 1 -- 172.21.15.69:0/4251582105 shutdown_connections 2023-07-06T20:19:44.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.988+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/4251582105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fee6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.988+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/4251582105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdcf20 0x558b60fe5600 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.988+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/4251582105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdd870 0x558b60feddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.988+0000 7fb7124eeb80 1 -- 172.21.15.69:0/4251582105 >> 172.21.15.69:0/4251582105 conn(0x558b60fd2640 msgr2=0x558b60ff8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:44.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.988+0000 7fb7124eeb80 1 -- 172.21.15.69:0/4251582105 shutdown_connections 2023-07-06T20:19:44.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.988+0000 7fb7124eeb80 1 -- 172.21.15.69:0/4251582105 wait complete. 2023-07-06T20:19:44.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.989+0000 7fb7124eeb80 1 -- start start 2023-07-06T20:19:44.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.989+0000 7fb7124eeb80 1 Processor -- start 2023-07-06T20:19:44.996 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb7124eeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdcf20 0x558b60fe9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.996 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb7124eeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdd870 0x558b60fea360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.996 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb7124eeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fed380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:44.996 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb7124eeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b60fed8c0 con 0x558b60fde390 2023-07-06T20:19:44.996 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb7124eeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b60feda30 con 0x558b60fdd870 2023-07-06T20:19:44.996 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb7124eeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b60fedbf0 con 0x558b60fdcf20 2023-07-06T20:19:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb7124eeb80 10 client.0 _ll_register_callbacks cb 0x558b60ecb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb6fc99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdcf20 0x558b60fe9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb6effff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdd870 0x558b60fea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb6fc99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdcf20 0x558b60fe9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43108/0 (socket says 172.21.15.69:43108) 2023-07-06T20:19:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb6fc99b700 1 -- 172.21.15.69:0/1247250390 learned_addr learned my addr 172.21.15.69:0/1247250390 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb6fd19c700 1 --2- 172.21.15.69:0/1247250390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fed380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb6fc99b700 1 -- 172.21.15.69:0/1247250390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdd870 msgr2=0x558b60fea360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:44.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb6fc99b700 1 --2- 172.21.15.69:0/1247250390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdd870 0x558b60fea360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb6fc99b700 1 -- 172.21.15.69:0/1247250390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 msgr2=0x558b60fed380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:44.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.990+0000 7fb6fc99b700 1 --2- 172.21.15.69:0/1247250390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fed380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:44.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.991+0000 7fb6fc99b700 1 -- 172.21.15.69:0/1247250390 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b60d44c60 con 0x558b60fdcf20 2023-07-06T20:19:44.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.991+0000 7fb6fc99b700 1 --2- 172.21.15.69:0/1247250390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdcf20 0x558b60fe9e20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6f409ad30 tx=0x7fb6f40a33c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db76ec10cb306185 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:44.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.991+0000 7fb6d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:19:44.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.991+0000 7fb6d7fff700 1 -- 172.21.15.69:0/1247250390 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f409b030 con 0x558b60fdcf20 2023-07-06T20:19:44.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.991+0000 7fb6d7fff700 1 -- 172.21.15.69:0/1247250390 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fb6f40a10f0 con 0x558b60fdcf20 2023-07-06T20:19:44.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.991+0000 7fb7124eeb80 1 -- 172.21.15.69:0/1247250390 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6e8019c50 con 0x558b60fdcf20 2023-07-06T20:19:44.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.991+0000 7fb6d7fff700 1 -- 172.21.15.69:0/1247250390 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6f4011110 con 0x558b60fdcf20 2023-07-06T20:19:44.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.991+0000 7fb7124eeb80 10 client.25309 fetch_fsmap learned FSMap version 26 2023-07-06T20:19:44.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.991+0000 7fb7124eeb80 1 -- 172.21.15.69:0/1247250390 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558b60d5a2b0 con 0x558b60fdcf20 2023-07-06T20:19:45.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.991+0000 7fb6d7fff700 1 -- 172.21.15.69:0/1247250390 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6e8019c50 con 0x558b60fdcf20 2023-07-06T20:19:45.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.992+0000 7fb6d7fff700 1 -- 172.21.15.69:0/1247250390 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fb6f40b3bb0 con 0x558b60fdcf20 2023-07-06T20:19:45.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.992+0000 7fb7124eeb80 10 client.25309 fetch_fsmap finished waiting for FSMap version 26 2023-07-06T20:19:45.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.992+0000 7fb7124eeb80 10 client.25309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:19:45.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.992+0000 7fb7124eeb80 20 client.25309 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:45.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.992+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/1247250390 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558b6104f3c0 0x558b610517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.992+0000 7fb7124eeb80 4 client.25309 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:19:45.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.992+0000 7fb7124eeb80 1 -- 172.21.15.69:0/1247250390 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558b60f26eb0 con 0x558b6104f3c0 2023-07-06T20:19:45.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.992+0000 7fb6effff700 1 --2- 172.21.15.69:0/1247250390 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558b6104f3c0 0x558b610517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb6d7fff700 10 client.25309 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:45.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb6effff700 1 --2- 172.21.15.69:0/1247250390 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558b6104f3c0 0x558b610517a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f68df8f1ec7c90 server_cookie=40ae6666f0d15612 in_seq=0 out_seq=0 2023-07-06T20:19:45.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb6d7fff700 1 -- 172.21.15.69:0/1247250390 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558b60f26eb0 con 0x558b6104f3c0 2023-07-06T20:19:45.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb6d7fff700 10 client.25309 handle_command_reply: tid=0 2023-07-06T20:19:45.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb7124eeb80 1 client.25309 shutdown 2023-07-06T20:19:45.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb7124eeb80 1 -- 172.21.15.69:0/1247250390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdcf20 msgr2=0x558b60fe9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.003 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/1247250390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdcf20 0x558b60fe9e20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6f409ad30 tx=0x7fb6f40a33c0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.003 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb7124eeb80 1 -- 172.21.15.69:0/1247250390 shutdown_connections 2023-07-06T20:19:45.003 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/1247250390 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558b6104f3c0 0x558b610517a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/1247250390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b60fde390 0x558b60fed380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/1247250390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b60fdcf20 0x558b60fe9e20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb6d7fff700 0 client.25309 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:45.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb7124eeb80 1 --2- 172.21.15.69:0/1247250390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b60fdd870 0x558b60fea360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.005 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb7124eeb80 1 -- 172.21.15.69:0/1247250390 >> 172.21.15.69:0/1247250390 conn(0x558b60fd2640 msgr2=0x558b60fee6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:45.005 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb7124eeb80 1 -- 172.21.15.69:0/1247250390 shutdown_connections 2023-07-06T20:19:45.005 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb7124eeb80 1 -- 172.21.15.69:0/1247250390 wait complete. 2023-07-06T20:19:45.005 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.993+0000 7fb7124eeb80 20 client.25309 trim_cache size 0 max 16384 2023-07-06T20:19:45.005 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:19:45.006 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2023-07-06T20:19:45.006 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2023-07-06T20:19:45.006 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:19:45.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.994+0000 7fb6febff700 1 -- 172.21.15.69:0/2993822 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb6e00a1150 msgr2=0x7fb6e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.994+0000 7fb6febff700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb6e00a1150 0x7fb6e00a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00127f0 tx=0x7fb6f000b040 comp rx=0 tx=0).stop 2023-07-06T20:19:45.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.994+0000 7fb6febff700 1 -- 172.21.15.69:0/2993822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6f8073020 msgr2=0x7fb6f8197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.994+0000 7fb6febff700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6f8073020 0x7fb6f8197cb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b680 tx=0x7fb6f4005740 comp rx=0 tx=0).stop 2023-07-06T20:19:45.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.994+0000 7fb6febff700 1 -- 172.21.15.69:0/2993822 shutdown_connections 2023-07-06T20:19:45.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.994+0000 7fb6febff700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb6e00a1150 0x7fb6e00a3600 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.994+0000 7fb6febff700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6f8107ad0 0x7fb6f819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.994+0000 7fb6febff700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6f8073020 0x7fb6f8197cb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.994+0000 7fb6febff700 1 --2- 172.21.15.69:0/2993822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f8074600 0x7fb6f81981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.994+0000 7fb6febff700 1 -- 172.21.15.69:0/2993822 >> 172.21.15.69:0/2993822 conn(0x7fb6f80fb260 msgr2=0x7fb6f80fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:45.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.994+0000 7fb6febff700 1 -- 172.21.15.69:0/2993822 shutdown_connections 2023-07-06T20:19:45.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:44.995+0000 7fb6febff700 1 -- 172.21.15.69:0/2993822 wait complete. 2023-07-06T20:19:45.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:44 smithi069 ceph-mon[126399]: pgmap v340: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 147 MiB/s wr, 83 op/s 2023-07-06T20:19:45.016 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-06T20:19:45.017 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:19:45.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:44 smithi156 ceph-mon[129280]: pgmap v340: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 147 MiB/s wr, 83 op/s 2023-07-06T20:19:45.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:44 smithi098 ceph-mon[128968]: pgmap v340: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 147 MiB/s wr, 83 op/s 2023-07-06T20:19:45.227 INFO:teuthology.orchestra.run.smithi069.stderr:9100000 of 50000000 tuples (18%) done (elapsed 22.04 s, remaining 99.05 s) 2023-07-06T20:19:45.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.231+0000 7f94cb39d700 1 -- 172.21.15.69:0/2071416056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c40fecd0 msgr2=0x7f94c40ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.231+0000 7f94cb39d700 1 --2- 172.21.15.69:0/2071416056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c40fecd0 0x7f94c40ff140 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009990 tx=0x7f94b4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.231+0000 7f94cb39d700 1 -- 172.21.15.69:0/2071416056 shutdown_connections 2023-07-06T20:19:45.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.231+0000 7f94cb39d700 1 --2- 172.21.15.69:0/2071416056 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94c40ff680 0x7f94c40ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.231+0000 7f94cb39d700 1 --2- 172.21.15.69:0/2071416056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c40fe380 0x7f94c40fe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.231+0000 7f94cb39d700 1 --2- 172.21.15.69:0/2071416056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c40fecd0 0x7f94c40ff140 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.231+0000 7f94cb39d700 1 -- 172.21.15.69:0/2071416056 >> 172.21.15.69:0/2071416056 conn(0x7f94c40fa010 msgr2=0x7f94c40fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:45.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.231+0000 7f94cb39d700 1 -- 172.21.15.69:0/2071416056 shutdown_connections 2023-07-06T20:19:45.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.231+0000 7f94cb39d700 1 -- 172.21.15.69:0/2071416056 wait complete. 2023-07-06T20:19:45.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.232+0000 7f94cb39d700 1 Processor -- start 2023-07-06T20:19:45.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.233+0000 7f94cb39d700 1 -- start start 2023-07-06T20:19:45.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.233+0000 7f94cb39d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c40fe380 0x7f94c4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.233+0000 7f94cb39d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c40fecd0 0x7f94c4197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.233+0000 7f94cb39d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94c40ff680 0x7f94c419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.233+0000 7f94cb39d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c419f3f0 con 0x7f94c40fe380 2023-07-06T20:19:45.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.233+0000 7f94cb39d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c419f530 con 0x7f94c40ff680 2023-07-06T20:19:45.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.233+0000 7f94cb39d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c419f6c0 con 0x7f94c40fecd0 2023-07-06T20:19:45.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.233+0000 7f94c9b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c40fecd0 0x7f94c4197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.233+0000 7f94ca39b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c40fe380 0x7f94c4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.233+0000 7f94c9b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c40fecd0 0x7f94c4197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43124/0 (socket says 172.21.15.69:43124) 2023-07-06T20:19:45.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.233+0000 7f94c9b9a700 1 -- 172.21.15.69:0/3898478422 learned_addr learned my addr 172.21.15.69:0/3898478422 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:45.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.233+0000 7f94cab9c700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94c40ff680 0x7f94c419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.234+0000 7f94c9b9a700 1 -- 172.21.15.69:0/3898478422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94c40ff680 msgr2=0x7f94c419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.234+0000 7f94c9b9a700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94c40ff680 0x7f94c419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.234+0000 7f94c9b9a700 1 -- 172.21.15.69:0/3898478422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c40fe380 msgr2=0x7f94c4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.234+0000 7f94c9b9a700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c40fe380 0x7f94c4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.234+0000 7f94c9b9a700 1 -- 172.21.15.69:0/3898478422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b40095f0 con 0x7f94c40fecd0 2023-07-06T20:19:45.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.234+0000 7f94cab9c700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94c40ff680 0x7f94c419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:19:45.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.234+0000 7f94c9b9a700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c40fecd0 0x7f94c4197db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006010 tx=0x7f94b4005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:45.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.234+0000 7f94bb7fe700 1 -- 172.21.15.69:0/3898478422 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b401d070 con 0x7f94c40fecd0 2023-07-06T20:19:45.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.234+0000 7f94bb7fe700 1 -- 172.21.15.69:0/3898478422 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94b4022470 con 0x7f94c40fecd0 2023-07-06T20:19:45.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.234+0000 7f94cb39d700 1 -- 172.21.15.69:0/3898478422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c419f940 con 0x7f94c40fecd0 2023-07-06T20:19:45.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.234+0000 7f94bb7fe700 1 -- 172.21.15.69:0/3898478422 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4017650 con 0x7f94c40fecd0 2023-07-06T20:19:45.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.234+0000 7f94cb39d700 1 -- 172.21.15.69:0/3898478422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c419fd00 con 0x7f94c40fecd0 2023-07-06T20:19:45.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.235+0000 7f94cb39d700 1 -- 172.21.15.69:0/3898478422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c4109160 con 0x7f94c40fecd0 2023-07-06T20:19:45.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.238+0000 7f94bb7fe700 1 -- 172.21.15.69:0/3898478422 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f94b400fad0 con 0x7f94c40fecd0 2023-07-06T20:19:45.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.240+0000 7f94bb7fe700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f94b00a1240 0x7f94b00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.240+0000 7f94bb7fe700 1 -- 172.21.15.69:0/3898478422 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f94b40cdec0 con 0x7f94c40fecd0 2023-07-06T20:19:45.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.240+0000 7f94ca39b700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f94b00a1240 0x7f94b00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.241+0000 7f94ca39b700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f94b00a1240 0x7f94b00a36f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009c60 tx=0x7f94c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:45.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.241+0000 7f94bb7fe700 1 -- 172.21.15.69:0/3898478422 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f94b4097610 con 0x7f94c40fecd0 2023-07-06T20:19:45.406 INFO:teuthology.orchestra.run.smithi069.stderr:9200000 of 50000000 tuples (18%) done (elapsed 22.22 s, remaining 98.53 s) 2023-07-06T20:19:45.513 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.511+0000 7f94cb39d700 1 -- 172.21.15.69:0/3898478422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94c402c8e0 con 0x7f94c40fecd0 2023-07-06T20:19:45.514 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.512+0000 7f94bb7fe700 1 -- 172.21.15.69:0/3898478422 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10799 (secure 0 0 0) 0x7f94b409ac30 con 0x7f94c40fecd0 2023-07-06T20:19:45.514 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:19:45.514 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-06T20:17:28.144582+0000","modified":"2023-07-06T20:19:42.606967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24539,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/3235525224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3235525224},{"type":"v1","addr":"172.21.15.98:6839","nonce":3235525224}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:19:45.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.515+0000 7f94cb39d700 1 -- 172.21.15.69:0/3898478422 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f94b00a1240 msgr2=0x7f94b00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.515+0000 7f94cb39d700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f94b00a1240 0x7f94b00a36f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009c60 tx=0x7f94c0009260 comp rx=0 tx=0).stop 2023-07-06T20:19:45.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.515+0000 7f94cb39d700 1 -- 172.21.15.69:0/3898478422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c40fecd0 msgr2=0x7f94c4197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.515+0000 7f94cb39d700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c40fecd0 0x7f94c4197db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006010 tx=0x7f94b4005740 comp rx=0 tx=0).stop 2023-07-06T20:19:45.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.515+0000 7f94cb39d700 1 -- 172.21.15.69:0/3898478422 shutdown_connections 2023-07-06T20:19:45.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.515+0000 7f94cb39d700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f94b00a1240 0x7f94b00a36f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.518 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.515+0000 7f94cb39d700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94c40ff680 0x7f94c419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.518 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.515+0000 7f94cb39d700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c40fe380 0x7f94c4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.518 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.515+0000 7f94cb39d700 1 --2- 172.21.15.69:0/3898478422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c40fecd0 0x7f94c4197db0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.518 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.515+0000 7f94cb39d700 1 -- 172.21.15.69:0/3898478422 >> 172.21.15.69:0/3898478422 conn(0x7f94c40fa010 msgr2=0x7f94c40fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:45.518 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.515+0000 7f94cb39d700 1 -- 172.21.15.69:0/3898478422 shutdown_connections 2023-07-06T20:19:45.518 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.515+0000 7f94cb39d700 1 -- 172.21.15.69:0/3898478422 wait complete. 2023-07-06T20:19:45.519 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2023-07-06T20:19:45.535 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-06T20:19:45.675 INFO:teuthology.orchestra.run.smithi069.stderr:9300000 of 50000000 tuples (18%) done (elapsed 22.49 s, remaining 98.41 s) 2023-07-06T20:19:45.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.753+0000 7f39fa3ff700 1 -- 172.21.15.69:0/4126388949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40ffc10 msgr2=0x7f39f4100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.753+0000 7f39fa3ff700 1 --2- 172.21.15.69:0/4126388949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40ffc10 0x7f39f4100020 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009990 tx=0x7f39e4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.753+0000 7f39fa3ff700 1 -- 172.21.15.69:0/4126388949 shutdown_connections 2023-07-06T20:19:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.753+0000 7f39fa3ff700 1 --2- 172.21.15.69:0/4126388949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f4101790 0x7f39f4108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.753+0000 7f39fa3ff700 1 --2- 172.21.15.69:0/4126388949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40ffc10 0x7f39f4100020 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.753+0000 7f39fa3ff700 1 --2- 172.21.15.69:0/4126388949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f4100e00 0x7f39f4101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.753+0000 7f39fa3ff700 1 -- 172.21.15.69:0/4126388949 >> 172.21.15.69:0/4126388949 conn(0x7f39f40fb210 msgr2=0x7f39f40fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:45.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.753+0000 7f39fa3ff700 1 -- 172.21.15.69:0/4126388949 shutdown_connections 2023-07-06T20:19:45.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.753+0000 7f39fa3ff700 1 -- 172.21.15.69:0/4126388949 wait complete. 2023-07-06T20:19:45.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.754+0000 7f39fa3ff700 1 Processor -- start 2023-07-06T20:19:45.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.754+0000 7f39fa3ff700 1 -- start start 2023-07-06T20:19:45.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39fa3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40ffc10 0x7f39f40743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39fa3ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f4100e00 0x7f39f4072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39fa3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f4101790 0x7f39f4072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39fa3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f4073710 con 0x7f39f40ffc10 2023-07-06T20:19:45.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39fa3ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f4073880 con 0x7f39f4100e00 2023-07-06T20:19:45.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39fa3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f419fa20 con 0x7f39f4101790 2023-07-06T20:19:45.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40ffc10 0x7f39f40743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40ffc10 0x7f39f40743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47284/0 (socket says 172.21.15.69:47284) 2023-07-06T20:19:45.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39f3fff700 1 -- 172.21.15.69:0/2719416435 learned_addr learned my addr 172.21.15.69:0/2719416435 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:45.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39f899c700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f4101790 0x7f39f4072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39f37fe700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f4100e00 0x7f39f4072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39f3fff700 1 -- 172.21.15.69:0/2719416435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f4101790 msgr2=0x7f39f4072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39f3fff700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f4101790 0x7f39f4072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39f3fff700 1 -- 172.21.15.69:0/2719416435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f4100e00 msgr2=0x7f39f4072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39f3fff700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f4100e00 0x7f39f4072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.755+0000 7f39f3fff700 1 -- 172.21.15.69:0/2719416435 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e40095f0 con 0x7f39f40ffc10 2023-07-06T20:19:45.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.756+0000 7f39f3fff700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40ffc10 0x7f39f40743f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f39e400b620 tx=0x7f39e4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:45.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.756+0000 7f39f17fa700 1 -- 172.21.15.69:0/2719416435 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e401d070 con 0x7f39f40ffc10 2023-07-06T20:19:45.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.756+0000 7f39f17fa700 1 -- 172.21.15.69:0/2719416435 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39e4021410 con 0x7f39f40ffc10 2023-07-06T20:19:45.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.756+0000 7f39fa3ff700 1 -- 172.21.15.69:0/2719416435 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f419fca0 con 0x7f39f40ffc10 2023-07-06T20:19:45.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.756+0000 7f39f17fa700 1 -- 172.21.15.69:0/2719416435 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4017650 con 0x7f39f40ffc10 2023-07-06T20:19:45.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.756+0000 7f39fa3ff700 1 -- 172.21.15.69:0/2719416435 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f41a0120 con 0x7f39f40ffc10 2023-07-06T20:19:45.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.759+0000 7f39f17fa700 1 -- 172.21.15.69:0/2719416435 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f39e4021580 con 0x7f39f40ffc10 2023-07-06T20:19:45.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.761+0000 7f3a0dc98b80 1 Processor -- start 2023-07-06T20:19:45.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.761+0000 7f39f17fa700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f39dc0a11d0 0x7f39dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.761+0000 7f3a0dc98b80 1 -- start start 2023-07-06T20:19:45.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.761+0000 7f39f17fa700 1 -- 172.21.15.69:0/2719416435 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f39e40cdda0 con 0x7f39f40ffc10 2023-07-06T20:19:45.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.761+0000 7f39f899c700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f39dc0a11d0 0x7f39dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.761+0000 7f3a0dc98b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce55f20 0x5591cce56330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.761+0000 7f3a0dc98b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce56870 0x5591cce56e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.761+0000 7f3a0dc98b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce57390 0x5591cce57830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.761+0000 7f3a0dc98b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ccd5acb0 con 0x5591cce56870 2023-07-06T20:19:45.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.761+0000 7f3a0dc98b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ccdada40 con 0x5591cce55f20 2023-07-06T20:19:45.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.761+0000 7f3a0dc98b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ccda6ed0 con 0x5591cce57390 2023-07-06T20:19:45.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce56870 0x5591cce56e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce56870 0x5591cce56e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47294/0 (socket says 172.21.15.69:47294) 2023-07-06T20:19:45.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce55f20 0x5591cce56330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f37fe700 1 -- 172.21.15.69:0/3242917084 learned_addr learned my addr 172.21.15.69:0/3242917084 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:45.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f37fe700 1 -- 172.21.15.69:0/3242917084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce57390 msgr2=0x5591cce57830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:45.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f899c700 1 --2- 172.21.15.69:0/3242917084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce57390 0x5591cce57830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f37fe700 1 --2- 172.21.15.69:0/3242917084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce57390 0x5591cce57830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f37fe700 1 -- 172.21.15.69:0/3242917084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce55f20 msgr2=0x5591cce56330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:45.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f37fe700 1 --2- 172.21.15.69:0/3242917084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce55f20 0x5591cce56330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f37fe700 1 -- 172.21.15.69:0/3242917084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591ccbd4de0 con 0x5591cce56870 2023-07-06T20:19:45.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f3fff700 1 --2- 172.21.15.69:0/3242917084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce55f20 0x5591cce56330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:45.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f899c700 1 --2- 172.21.15.69:0/3242917084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce57390 0x5591cce57830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:19:45.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f899c700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f39dc0a11d0 0x7f39dc0a3680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f39f4074a50 tx=0x7f39ec003f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:45.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.762+0000 7f39f37fe700 1 --2- 172.21.15.69:0/3242917084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce56870 0x5591cce56e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009d80 tx=0x7f39e8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49feb307b93de92a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:45.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.763+0000 7f39da7fc700 1 -- 172.21.15.69:0/3242917084 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8004230 con 0x5591cce56870 2023-07-06T20:19:45.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.763+0000 7f39da7fc700 1 -- 172.21.15.69:0/3242917084 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39e8004390 con 0x5591cce56870 2023-07-06T20:19:45.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.763+0000 7f39da7fc700 1 -- 172.21.15.69:0/3242917084 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8019880 con 0x5591cce56870 2023-07-06T20:19:45.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.763+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3242917084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce56870 msgr2=0x5591cce56e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.763+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/3242917084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce56870 0x5591cce56e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009d80 tx=0x7f39e8011720 comp rx=0 tx=0).stop 2023-07-06T20:19:45.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.763+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3242917084 shutdown_connections 2023-07-06T20:19:45.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.763+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/3242917084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce57390 0x5591cce57830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.763+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/3242917084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce55f20 0x5591cce56330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.764+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/3242917084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce56870 0x5591cce56e00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.764+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3242917084 >> 172.21.15.69:0/3242917084 conn(0x5591cce4b640 msgr2=0x5591cce509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.764+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3242917084 shutdown_connections 2023-07-06T20:19:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.764+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3242917084 wait complete. 2023-07-06T20:19:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.764+0000 7f3a0dc98b80 1 -- start start 2023-07-06T20:19:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.765+0000 7f3a0dc98b80 1 Processor -- start 2023-07-06T20:19:45.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.765+0000 7f3a0dc98b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce55f20 0x5591cce5f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.765+0000 7f3a0dc98b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce56870 0x5591cce5fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.765+0000 7f3a0dc98b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce57390 0x5591cce66e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.765+0000 7f3a0dc98b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ccd46ac0 con 0x5591cce56870 2023-07-06T20:19:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.765+0000 7f3a0dc98b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ccd467c0 con 0x5591cce57390 2023-07-06T20:19:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.765+0000 7f3a0dc98b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591ccc2a920 con 0x5591cce55f20 2023-07-06T20:19:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.765+0000 7f3a0dc98b80 10 client.0 _ll_register_callbacks cb 0x5591ccd44760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:45.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.765+0000 7f39f37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce56870 0x5591cce5fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f39f899c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce57390 0x5591cce66e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f39f899c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce57390 0x5591cce66e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:36876/0 (socket says 172.21.15.69:36876) 2023-07-06T20:19:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f39f899c700 1 -- 172.21.15.69:0/134765461 learned_addr learned my addr 172.21.15.69:0/134765461 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f39f3fff700 1 --2- 172.21.15.69:0/134765461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce55f20 0x5591cce5f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f39f899c700 1 -- 172.21.15.69:0/134765461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce55f20 msgr2=0x5591cce5f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f39f899c700 1 --2- 172.21.15.69:0/134765461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce55f20 0x5591cce5f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f39f899c700 1 -- 172.21.15.69:0/134765461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce56870 msgr2=0x5591cce5fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f39f899c700 1 --2- 172.21.15.69:0/134765461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce56870 0x5591cce5fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f39f899c700 1 -- 172.21.15.69:0/134765461 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f41a0120 con 0x5591cce57390 2023-07-06T20:19:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f39f899c700 1 --2- 172.21.15.69:0/134765461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce57390 0x5591cce66e10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f39ec007ab0 tx=0x7f39ec004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc57268329e98cf5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f39d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:19:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f39d37fe700 1 -- 172.21.15.69:0/134765461 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec01f070 con 0x5591cce57390 2023-07-06T20:19:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.766+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/134765461 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591ccd004f0 con 0x5591cce57390 2023-07-06T20:19:45.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.767+0000 7f39d37fe700 1 -- 172.21.15.69:0/134765461 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f39ec017880 con 0x5591cce57390 2023-07-06T20:19:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.767+0000 7f39d37fe700 1 -- 172.21.15.69:0/134765461 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39ec004930 con 0x5591cce57390 2023-07-06T20:19:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.767+0000 7f3a0dc98b80 10 client.25367 fetch_fsmap learned FSMap version 26 2023-07-06T20:19:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.767+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/134765461 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f39e80099e0 con 0x5591cce57390 2023-07-06T20:19:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.767+0000 7f39d37fe700 1 -- 172.21.15.69:0/134765461 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39ec032080 con 0x5591cce57390 2023-07-06T20:19:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.767+0000 7f39d37fe700 1 -- 172.21.15.69:0/134765461 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f39ec023bb0 con 0x5591cce57390 2023-07-06T20:19:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.767+0000 7f3a0dc98b80 10 client.25367 fetch_fsmap finished waiting for FSMap version 26 2023-07-06T20:19:45.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.767+0000 7f3a0dc98b80 10 client.25367 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-06T20:19:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.767+0000 7f3a0dc98b80 20 client.25367 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.767+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/134765461 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5591cce6b6e0 0x5591cce6daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.767+0000 7f3a0dc98b80 4 client.25367 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:19:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.767+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/134765461 --> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591cce24d60 con 0x5591cce6b6e0 2023-07-06T20:19:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.768+0000 7f39f3fff700 1 --2- 172.21.15.69:0/134765461 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5591cce6b6e0 0x5591cce6daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.768+0000 7f39f3fff700 1 --2- 172.21.15.69:0/134765461 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5591cce6b6e0 0x5591cce6daf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9b63483cb8d33c8 server_cookie=e8484e50a4fd952b in_seq=0 out_seq=0 2023-07-06T20:19:45.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.768+0000 7f39d37fe700 10 client.25367 ms_handle_connect on v2:172.21.15.98:6834/827333150 2023-07-06T20:19:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f39d37fe700 1 -- 172.21.15.69:0/134765461 <== mds.2 v2:172.21.15.98:6834/827333150 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5591cce24d60 con 0x5591cce6b6e0 2023-07-06T20:19:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f39d37fe700 10 client.25367 handle_command_reply: tid=0 2023-07-06T20:19:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f3a0dc98b80 1 client.25367 shutdown 2023-07-06T20:19:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/134765461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce57390 msgr2=0x5591cce66e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/134765461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce57390 0x5591cce66e10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f39ec007ab0 tx=0x7f39ec004b40 comp rx=0 tx=0).stop 2023-07-06T20:19:45.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/134765461 shutdown_connections 2023-07-06T20:19:45.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/134765461 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5591cce6b6e0 0x5591cce6daf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/134765461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce57390 0x5591cce66e10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f39d37fe700 0 client.25367 ms_handle_reset on v2:172.21.15.98:6834/827333150 2023-07-06T20:19:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/134765461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce55f20 0x5591cce5f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/134765461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce56870 0x5591cce5fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/134765461 >> 172.21.15.69:0/134765461 conn(0x5591cce4b640 msgr2=0x5591cce500b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/134765461 shutdown_connections 2023-07-06T20:19:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.769+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/134765461 wait complete. 2023-07-06T20:19:45.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.770+0000 7f3a0dc98b80 20 client.25367 trim_cache size 0 max 16384 2023-07-06T20:19:45.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.792+0000 7f3a0dc98b80 1 Processor -- start 2023-07-06T20:19:45.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.792+0000 7f3a0dc98b80 1 -- start start 2023-07-06T20:19:45.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f3a0dc98b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce55f20 0x5591cce5d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f3a0dc98b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 0x5591cce66dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f3a0dc98b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce57390 0x5591cce5a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f3a0dc98b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591cce5f400 con 0x5591cce55f20 2023-07-06T20:19:45.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f3a0dc98b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591cce52460 con 0x5591cce56870 2023-07-06T20:19:45.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f3a0dc98b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591cce56c40 con 0x5591cce57390 2023-07-06T20:19:45.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f39f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce55f20 0x5591cce5d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f39f37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 0x5591cce66dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f39f37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 0x5591cce66dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:36888/0 (socket says 172.21.15.69:36888) 2023-07-06T20:19:45.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f39f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce55f20 0x5591cce5d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47306/0 (socket says 172.21.15.69:47306) 2023-07-06T20:19:45.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f39f37fe700 1 -- 172.21.15.69:0/623693537 learned_addr learned my addr 172.21.15.69:0/623693537 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:45.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f39f899c700 1 --2- 172.21.15.69:0/623693537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce57390 0x5591cce5a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f39f37fe700 1 -- 172.21.15.69:0/623693537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce57390 msgr2=0x5591cce5a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:45.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f39f37fe700 1 --2- 172.21.15.69:0/623693537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce57390 0x5591cce5a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f39f37fe700 1 -- 172.21.15.69:0/623693537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce55f20 msgr2=0x5591cce5d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:45.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f39f37fe700 1 --2- 172.21.15.69:0/623693537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce55f20 0x5591cce5d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.793+0000 7f39f37fe700 1 -- 172.21.15.69:0/623693537 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591ccbbdc60 con 0x5591cce56870 2023-07-06T20:19:45.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.794+0000 7f39f37fe700 1 --2- 172.21.15.69:0/623693537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 0x5591cce66dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f39e80097c0 tx=0x7f39e80045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5cfcf19e50ff9cb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:45.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.794+0000 7f39daffd700 1 -- 172.21.15.69:0/623693537 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8005240 con 0x5591cce56870 2023-07-06T20:19:45.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.794+0000 7f39daffd700 1 -- 172.21.15.69:0/623693537 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39e80046f0 con 0x5591cce56870 2023-07-06T20:19:45.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.794+0000 7f39daffd700 1 -- 172.21.15.69:0/623693537 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e80196e0 con 0x5591cce56870 2023-07-06T20:19:45.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.795+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/623693537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 msgr2=0x5591cce66dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.795+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/623693537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 0x5591cce66dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f39e80097c0 tx=0x7f39e80045b0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.795+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/623693537 shutdown_connections 2023-07-06T20:19:45.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.795+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/623693537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce57390 0x5591cce5a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.795+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/623693537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce55f20 0x5591cce5d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.795+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/623693537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 0x5591cce66dd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.795+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/623693537 >> 172.21.15.69:0/623693537 conn(0x5591cce4b640 msgr2=0x5591cce5e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:45.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.795+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/623693537 shutdown_connections 2023-07-06T20:19:45.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.795+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/623693537 wait complete. 2023-07-06T20:19:45.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.796+0000 7f3a0dc98b80 1 -- start start 2023-07-06T20:19:45.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.796+0000 7f3a0dc98b80 1 Processor -- start 2023-07-06T20:19:45.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f3a0dc98b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce55f20 0x5591cce62dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f3a0dc98b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 0x5591cce63310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f3a0dc98b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce57390 0x5591cce66330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f3a0dc98b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591cce66870 con 0x5591cce57390 2023-07-06T20:19:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f3a0dc98b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591cce669e0 con 0x5591cce56870 2023-07-06T20:19:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f3a0dc98b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591cce66ba0 con 0x5591cce55f20 2023-07-06T20:19:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f3a0dc98b80 10 client.0 _ll_register_callbacks cb 0x7f39ec004880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:45.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f39f37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 0x5591cce63310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f39f37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 0x5591cce63310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:36892/0 (socket says 172.21.15.69:36892) 2023-07-06T20:19:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f39f3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce55f20 0x5591cce62dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f39f37fe700 1 -- 172.21.15.69:0/3161182677 learned_addr learned my addr 172.21.15.69:0/3161182677 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f39f899c700 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce57390 0x5591cce66330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f39f3fff700 1 -- 172.21.15.69:0/3161182677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 msgr2=0x5591cce63310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:45.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f39f3fff700 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 0x5591cce63310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f39f3fff700 1 -- 172.21.15.69:0/3161182677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce57390 msgr2=0x5591cce66330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:45.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f39f3fff700 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce57390 0x5591cce66330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f39f3fff700 1 -- 172.21.15.69:0/3161182677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591ccbd4de0 con 0x5591cce55f20 2023-07-06T20:19:45.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.797+0000 7f39f37fe700 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 0x5591cce63310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:45.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.798+0000 7f39f3fff700 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce55f20 0x5591cce62dd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f39e400b2e0 tx=0x7f39e4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=333bf9d2802da633 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:45.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.798+0000 7f39d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:19:45.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.798+0000 7f39d37fe700 1 -- 172.21.15.69:0/3161182677 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40a3570 con 0x5591cce55f20 2023-07-06T20:19:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.798+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3161182677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39e8004390 con 0x5591cce55f20 2023-07-06T20:19:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.798+0000 7f39d37fe700 1 -- 172.21.15.69:0/3161182677 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f39e409b410 con 0x5591cce55f20 2023-07-06T20:19:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.798+0000 7f39d37fe700 1 -- 172.21.15.69:0/3161182677 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39e409b5d0 con 0x5591cce55f20 2023-07-06T20:19:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.798+0000 7f3a0dc98b80 10 client.25330 fetch_fsmap learned FSMap version 26 2023-07-06T20:19:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.798+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3161182677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5591ccbfb250 con 0x5591cce55f20 2023-07-06T20:19:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.799+0000 7f39d37fe700 1 -- 172.21.15.69:0/3161182677 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39e8004390 con 0x5591cce55f20 2023-07-06T20:19:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.799+0000 7f39d37fe700 1 -- 172.21.15.69:0/3161182677 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f39e40a30e0 con 0x5591cce55f20 2023-07-06T20:19:45.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.799+0000 7f3a0dc98b80 10 client.25330 fetch_fsmap finished waiting for FSMap version 26 2023-07-06T20:19:45.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.799+0000 7f3a0dc98b80 10 client.25330 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-06T20:19:45.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.799+0000 7f3a0dc98b80 20 client.25330 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:45.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.799+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5591ccec0360 0x5591ccec2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:45.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.799+0000 7f3a0dc98b80 4 client.25330 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-06T20:19:45.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.799+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3161182677 --> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5591ccd9feb0 con 0x5591ccec0360 2023-07-06T20:19:45.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.799+0000 7f39f37fe700 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5591ccec0360 0x5591ccec2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:45.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f39d37fe700 10 client.25330 ms_handle_connect on v2:172.21.15.98:6834/827333150 2023-07-06T20:19:45.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f39f37fe700 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5591ccec0360 0x5591ccec2740 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=358494e2d083ecd0 server_cookie=28e9bab16e197913 in_seq=0 out_seq=0 2023-07-06T20:19:45.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f39d37fe700 1 -- 172.21.15.69:0/3161182677 <== mds.2 v2:172.21.15.98:6834/827333150 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5591ccd9feb0 con 0x5591ccec0360 2023-07-06T20:19:45.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f39d37fe700 10 client.25330 handle_command_reply: tid=0 2023-07-06T20:19:45.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f3a0dc98b80 1 client.25330 shutdown 2023-07-06T20:19:45.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3161182677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce55f20 msgr2=0x5591cce62dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce55f20 0x5591cce62dd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f39e400b2e0 tx=0x7f39e4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3161182677 shutdown_connections 2023-07-06T20:19:45.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5591cce57390 0x5591cce66330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5591cce55f20 0x5591cce62dd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.98:6834/827333150,v1:172.21.15.98:6835/827333150] conn(0x5591ccec0360 0x5591ccec2740 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f3a0dc98b80 1 --2- 172.21.15.69:0/3161182677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5591cce56870 0x5591cce63310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f39d37fe700 0 client.25330 ms_handle_reset on v2:172.21.15.98:6834/827333150 2023-07-06T20:19:45.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3161182677 >> 172.21.15.69:0/3161182677 conn(0x5591cce4b640 msgr2=0x5591cce5d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:45.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3161182677 shutdown_connections 2023-07-06T20:19:45.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f3a0dc98b80 1 -- 172.21.15.69:0/3161182677 wait complete. 2023-07-06T20:19:45.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.800+0000 7f3a0dc98b80 20 client.25330 trim_cache size 0 max 16384 2023-07-06T20:19:45.811 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2023-07-06T20:19:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.801+0000 7f39fa3ff700 1 -- 172.21.15.69:0/2719416435 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f39dc0a11d0 msgr2=0x7f39dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.801+0000 7f39fa3ff700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f39dc0a11d0 0x7f39dc0a3680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f39f4074a50 tx=0x7f39ec003f90 comp rx=0 tx=0).stop 2023-07-06T20:19:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.801+0000 7f39fa3ff700 1 -- 172.21.15.69:0/2719416435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40ffc10 msgr2=0x7f39f40743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:45.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.801+0000 7f39fa3ff700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40ffc10 0x7f39f40743f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f39e400b620 tx=0x7f39e4005740 comp rx=0 tx=0).stop 2023-07-06T20:19:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.801+0000 7f39fa3ff700 1 -- 172.21.15.69:0/2719416435 shutdown_connections 2023-07-06T20:19:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.801+0000 7f39fa3ff700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f39dc0a11d0 0x7f39dc0a3680 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.801+0000 7f39fa3ff700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f4101790 0x7f39f4072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.801+0000 7f39fa3ff700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39f40ffc10 0x7f39f40743f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.801+0000 7f39fa3ff700 1 --2- 172.21.15.69:0/2719416435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f4100e00 0x7f39f4072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.801+0000 7f39fa3ff700 1 -- 172.21.15.69:0/2719416435 >> 172.21.15.69:0/2719416435 conn(0x7f39f40fb210 msgr2=0x7f39f40fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:45.813 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.801+0000 7f39fa3ff700 1 -- 172.21.15.69:0/2719416435 shutdown_connections 2023-07-06T20:19:45.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:45.801+0000 7f39fa3ff700 1 -- 172.21.15.69:0/2719416435 wait complete. 2023-07-06T20:19:45.826 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:45 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3898478422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:19:45.829 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-06T20:19:45.992 INFO:teuthology.orchestra.run.smithi069.stderr:9400000 of 50000000 tuples (18%) done (elapsed 22.80 s, remaining 98.49 s) 2023-07-06T20:19:46.043 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.041+0000 7f9641dff700 1 -- 172.21.15.69:0/1396058423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f963c0ffc20 msgr2=0x7f963c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.041+0000 7f9641dff700 1 --2- 172.21.15.69:0/1396058423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f963c0ffc20 0x7f963c100030 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f962c009990 tx=0x7f962c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.041+0000 7f9641dff700 1 -- 172.21.15.69:0/1396058423 shutdown_connections 2023-07-06T20:19:46.044 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.041+0000 7f9641dff700 1 --2- 172.21.15.69:0/1396058423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f963c1017a0 0x7f963c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.041+0000 7f9641dff700 1 --2- 172.21.15.69:0/1396058423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f963c0ffc20 0x7f963c100030 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.041+0000 7f9641dff700 1 --2- 172.21.15.69:0/1396058423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c100e10 0x7f963c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.041+0000 7f9641dff700 1 -- 172.21.15.69:0/1396058423 >> 172.21.15.69:0/1396058423 conn(0x7f963c0fb260 msgr2=0x7f963c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.041+0000 7f9641dff700 1 -- 172.21.15.69:0/1396058423 shutdown_connections 2023-07-06T20:19:46.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.041+0000 7f9641dff700 1 -- 172.21.15.69:0/1396058423 wait complete. 2023-07-06T20:19:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.042+0000 7f9641dff700 1 Processor -- start 2023-07-06T20:19:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.042+0000 7f9641dff700 1 -- start start 2023-07-06T20:19:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.042+0000 7f9641dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f963c0ffc20 0x7f963c0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.042+0000 7f9641dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c100e10 0x7f963c072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.042+0000 7f9641dff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f963c1017a0 0x7f963c072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.042+0000 7f9641dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c073710 con 0x7f963c100e10 2023-07-06T20:19:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f9641dff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c073880 con 0x7f963c1017a0 2023-07-06T20:19:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f9641dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c19f9c0 con 0x7f963c0ffc20 2023-07-06T20:19:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f963affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c100e10 0x7f963c072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f963b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f963c0ffc20 0x7f963c0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f963b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f963c0ffc20 0x7f963c0743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43200/0 (socket says 172.21.15.69:43200) 2023-07-06T20:19:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f963b7fe700 1 -- 172.21.15.69:0/2153484452 learned_addr learned my addr 172.21.15.69:0/2153484452 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:46.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f963bfff700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f963c1017a0 0x7f963c072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f963b7fe700 1 -- 172.21.15.69:0/2153484452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f963c1017a0 msgr2=0x7f963c072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f963b7fe700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f963c1017a0 0x7f963c072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f963b7fe700 1 -- 172.21.15.69:0/2153484452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c100e10 msgr2=0x7f963c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f963b7fe700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c100e10 0x7f963c072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f963b7fe700 1 -- 172.21.15.69:0/2153484452 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630009520 con 0x7f963c0ffc20 2023-07-06T20:19:46.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.043+0000 7f963b7fe700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f963c0ffc20 0x7f963c0743f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f962c005740 tx=0x7f962c005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:46.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.044+0000 7f9638ff9700 1 -- 172.21.15.69:0/2153484452 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c01d070 con 0x7f963c0ffc20 2023-07-06T20:19:46.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.044+0000 7f9641dff700 1 -- 172.21.15.69:0/2153484452 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9634009c40 con 0x7f963c0ffc20 2023-07-06T20:19:46.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.044+0000 7f9638ff9700 1 -- 172.21.15.69:0/2153484452 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f962c021410 con 0x7f963c0ffc20 2023-07-06T20:19:46.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.044+0000 7f9638ff9700 1 -- 172.21.15.69:0/2153484452 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c017650 con 0x7f963c0ffc20 2023-07-06T20:19:46.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.044+0000 7f9641dff700 1 -- 172.21.15.69:0/2153484452 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c0095f0 con 0x7f963c0ffc20 2023-07-06T20:19:46.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.047+0000 7f9638ff9700 1 -- 172.21.15.69:0/2153484452 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f962c021580 con 0x7f963c0ffc20 2023-07-06T20:19:46.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.048+0000 7f9655648b80 1 Processor -- start 2023-07-06T20:19:46.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.048+0000 7f9655648b80 1 -- start start 2023-07-06T20:19:46.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.048+0000 7f9638ff9700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f96240a1150 0x7f96240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f9655648b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620ceff7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f9638ff9700 1 -- 172.21.15.69:0/2153484452 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f962c0cdc80 con 0x7f963c0ffc20 2023-07-06T20:19:46.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f9655648b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff7870 0x5620ceff7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f9655648b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff8390 0x5620ceff8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f9655648b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ceefbcb0 con 0x5620ceff7870 2023-07-06T20:19:46.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f9655648b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620cef4ea40 con 0x5620ceff6f20 2023-07-06T20:19:46.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f9655648b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620cef47ed0 con 0x5620ceff8390 2023-07-06T20:19:46.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f963b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620ceff7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f963b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620ceff7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:36910/0 (socket says 172.21.15.69:36910) 2023-07-06T20:19:46.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f963b7fe700 1 -- 172.21.15.69:0/1449744967 learned_addr learned my addr 172.21.15.69:0/1449744967 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:46.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f963bfff700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f96240a1150 0x7f96240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f963bfff700 1 --2- 172.21.15.69:0/1449744967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff8390 0x5620ceff8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f963affd700 1 --2- 172.21.15.69:0/1449744967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff7870 0x5620ceff7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f963b7fe700 1 -- 172.21.15.69:0/1449744967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff8390 msgr2=0x5620ceff8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f963b7fe700 1 --2- 172.21.15.69:0/1449744967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff8390 0x5620ceff8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f963b7fe700 1 -- 172.21.15.69:0/1449744967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff7870 msgr2=0x5620ceff7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.049+0000 7f963b7fe700 1 --2- 172.21.15.69:0/1449744967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff7870 0x5620ceff7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.050+0000 7f963b7fe700 1 -- 172.21.15.69:0/1449744967 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620ced75de0 con 0x5620ceff6f20 2023-07-06T20:19:46.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.050+0000 7f963b7fe700 1 --2- 172.21.15.69:0/1449744967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620ceff7330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f962c09b0b0 tx=0x7f962c09ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a9a46aee2c5c197 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:46.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.050+0000 7f963bfff700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f96240a1150 0x7f96240a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9634009550 tx=0x7f9634009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:46.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.050+0000 7f9621ffb700 1 -- 172.21.15.69:0/1449744967 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c09f150 con 0x5620ceff6f20 2023-07-06T20:19:46.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.050+0000 7f9621ffb700 1 -- 172.21.15.69:0/1449744967 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f962c0a34e0 con 0x5620ceff6f20 2023-07-06T20:19:46.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.050+0000 7f9621ffb700 1 -- 172.21.15.69:0/1449744967 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c09a810 con 0x5620ceff6f20 2023-07-06T20:19:46.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.051+0000 7f9655648b80 1 -- 172.21.15.69:0/1449744967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 msgr2=0x5620ceff7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.051+0000 7f9655648b80 1 --2- 172.21.15.69:0/1449744967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620ceff7330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f962c09b0b0 tx=0x7f962c09ace0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.051+0000 7f9655648b80 1 -- 172.21.15.69:0/1449744967 shutdown_connections 2023-07-06T20:19:46.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.051+0000 7f9655648b80 1 --2- 172.21.15.69:0/1449744967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff8390 0x5620ceff8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.051+0000 7f9655648b80 1 --2- 172.21.15.69:0/1449744967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620ceff7330 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.051+0000 7f9655648b80 1 --2- 172.21.15.69:0/1449744967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff7870 0x5620ceff7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.051+0000 7f9655648b80 1 -- 172.21.15.69:0/1449744967 >> 172.21.15.69:0/1449744967 conn(0x5620cefec640 msgr2=0x5620ceff19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:46.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.051+0000 7f9655648b80 1 -- 172.21.15.69:0/1449744967 shutdown_connections 2023-07-06T20:19:46.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.051+0000 7f9655648b80 1 -- 172.21.15.69:0/1449744967 wait complete. 2023-07-06T20:19:46.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.052+0000 7f9655648b80 1 -- start start 2023-07-06T20:19:46.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.052+0000 7f9655648b80 1 Processor -- start 2023-07-06T20:19:46.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.052+0000 7f9655648b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620cf000480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.052+0000 7f9655648b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 0x5620cf0009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.052+0000 7f9655648b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff8390 0x5620cf007a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.052+0000 7f9655648b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ceee7ac0 con 0x5620ceff8390 2023-07-06T20:19:46.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.052+0000 7f9655648b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ceee77c0 con 0x5620ceff6f20 2023-07-06T20:19:46.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.052+0000 7f9655648b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620cedcb920 con 0x5620ceff7870 2023-07-06T20:19:46.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.052+0000 7f9655648b80 10 client.0 _ll_register_callbacks cb 0x5620ceee5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:46.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.052+0000 7f963b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620cf000480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.052+0000 7f963b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620cf000480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:36920/0 (socket says 172.21.15.69:36920) 2023-07-06T20:19:46.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.052+0000 7f963b7fe700 1 -- 172.21.15.69:0/1488451108 learned_addr learned my addr 172.21.15.69:0/1488451108 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:46.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f963bfff700 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff8390 0x5620cf007a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f963affd700 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 0x5620cf0009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f963b7fe700 1 -- 172.21.15.69:0/1488451108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 msgr2=0x5620cf0009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f963b7fe700 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 0x5620cf0009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f963b7fe700 1 -- 172.21.15.69:0/1488451108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff8390 msgr2=0x5620cf007a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f963b7fe700 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff8390 0x5620cf007a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f963b7fe700 1 -- 172.21.15.69:0/1488451108 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c0095f0 con 0x5620ceff6f20 2023-07-06T20:19:46.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f963affd700 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 0x5620cf0009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:19:46.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f963b7fe700 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620cf000480 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f962c009ce0 tx=0x7f962c09b1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4290eaaf65fcae76 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:46.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f961affd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:19:46.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f961affd700 1 -- 172.21.15.69:0/1488451108 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c09b380 con 0x5620ceff6f20 2023-07-06T20:19:46.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f9655648b80 1 -- 172.21.15.69:0/1488451108 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620ceea14f0 con 0x5620ceff6f20 2023-07-06T20:19:46.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.053+0000 7f961affd700 1 -- 172.21.15.69:0/1488451108 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f962c0b92c0 con 0x5620ceff6f20 2023-07-06T20:19:46.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.054+0000 7f961affd700 1 -- 172.21.15.69:0/1488451108 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f962c0b94a0 con 0x5620ceff6f20 2023-07-06T20:19:46.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.054+0000 7f9655648b80 10 client.25385 fetch_fsmap learned FSMap version 26 2023-07-06T20:19:46.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.054+0000 7f9655648b80 1 -- 172.21.15.69:0/1488451108 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5620ced5ec60 con 0x5620ceff6f20 2023-07-06T20:19:46.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.054+0000 7f961affd700 1 -- 172.21.15.69:0/1488451108 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620ceea14f0 con 0x5620ceff6f20 2023-07-06T20:19:46.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.054+0000 7f961affd700 1 -- 172.21.15.69:0/1488451108 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f962c0cbbb0 con 0x5620ceff6f20 2023-07-06T20:19:46.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.054+0000 7f9655648b80 10 client.25385 fetch_fsmap finished waiting for FSMap version 26 2023-07-06T20:19:46.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.054+0000 7f9655648b80 10 client.25385 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-06T20:19:46.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.054+0000 7f9655648b80 20 client.25385 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:46.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.054+0000 7f9655648b80 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x5620cf00bdd0 0x5620cf00e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.054+0000 7f9655648b80 4 client.25385 mds_command: new command op to 24539 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:19:46.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.054+0000 7f9655648b80 1 -- 172.21.15.69:0/1488451108 --> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620cefc5d60 con 0x5620cf00bdd0 2023-07-06T20:19:46.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.055+0000 7f963affd700 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x5620cf00bdd0 0x5620cf00e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.055+0000 7f963affd700 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x5620cf00bdd0 0x5620cf00e1e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bffe79f337f9ad04 server_cookie=a807eb911fd80476 in_seq=0 out_seq=0 2023-07-06T20:19:46.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.055+0000 7f961affd700 10 client.25385 ms_handle_connect on v2:172.21.15.98:6836/4010267682 2023-07-06T20:19:46.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f961affd700 1 -- 172.21.15.69:0/1488451108 <== mds.1 v2:172.21.15.98:6836/4010267682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5620cefc5d60 con 0x5620cf00bdd0 2023-07-06T20:19:46.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f961affd700 10 client.25385 handle_command_reply: tid=0 2023-07-06T20:19:46.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f9655648b80 1 client.25385 shutdown 2023-07-06T20:19:46.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f9655648b80 1 -- 172.21.15.69:0/1488451108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 msgr2=0x5620cf000480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f9655648b80 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620cf000480 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f962c009ce0 tx=0x7f962c09b1e0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f9655648b80 1 -- 172.21.15.69:0/1488451108 shutdown_connections 2023-07-06T20:19:46.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f9655648b80 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x5620cf00bdd0 0x5620cf00e1e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f9655648b80 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff8390 0x5620cf007a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f9655648b80 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620cf000480 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f961affd700 0 client.25385 ms_handle_reset on v2:172.21.15.98:6836/4010267682 2023-07-06T20:19:46.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f9655648b80 1 --2- 172.21.15.69:0/1488451108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 0x5620cf0009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f9655648b80 1 -- 172.21.15.69:0/1488451108 >> 172.21.15.69:0/1488451108 conn(0x5620cefec640 msgr2=0x5620ceff10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:46.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f9655648b80 1 -- 172.21.15.69:0/1488451108 shutdown_connections 2023-07-06T20:19:46.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f9655648b80 1 -- 172.21.15.69:0/1488451108 wait complete. 2023-07-06T20:19:46.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.056+0000 7f9655648b80 20 client.25385 trim_cache size 0 max 16384 2023-07-06T20:19:46.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.083+0000 7f9655648b80 1 Processor -- start 2023-07-06T20:19:46.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.083+0000 7f9655648b80 1 -- start start 2023-07-06T20:19:46.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.083+0000 7f9655648b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620ceffb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.083+0000 7f9655648b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 0x5620ceffb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.083+0000 7f9655648b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff8390 0x5620cf0079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.083+0000 7f9655648b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620cefc5d60 con 0x5620ceff8390 2023-07-06T20:19:46.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.083+0000 7f9655648b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ceea14f0 con 0x5620ceff6f20 2023-07-06T20:19:46.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.083+0000 7f9655648b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620cf001670 con 0x5620ceff7870 2023-07-06T20:19:46.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.084+0000 7f963affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 0x5620ceffb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.084+0000 7f963bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff8390 0x5620cf0079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.084+0000 7f963affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 0x5620ceffb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43216/0 (socket says 172.21.15.69:43216) 2023-07-06T20:19:46.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.084+0000 7f963affd700 1 -- 172.21.15.69:0/3790476643 learned_addr learned my addr 172.21.15.69:0/3790476643 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:46.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.084+0000 7f963b7fe700 1 --2- 172.21.15.69:0/3790476643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620ceffb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.084+0000 7f963affd700 1 -- 172.21.15.69:0/3790476643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 msgr2=0x5620ceffb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.084+0000 7f963affd700 1 --2- 172.21.15.69:0/3790476643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620ceffb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.084+0000 7f963affd700 1 -- 172.21.15.69:0/3790476643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff8390 msgr2=0x5620cf0079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.084+0000 7f963affd700 1 --2- 172.21.15.69:0/3790476643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff8390 0x5620cf0079c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.084+0000 7f963affd700 1 -- 172.21.15.69:0/3790476643 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620ced9c250 con 0x5620ceff7870 2023-07-06T20:19:46.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.084+0000 7f963bfff700 1 --2- 172.21.15.69:0/3790476643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff8390 0x5620cf0079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:46.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.084+0000 7f963affd700 1 --2- 172.21.15.69:0/3790476643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 0x5620ceffb730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5620cf00eac0 tx=0x7f9630004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a82ff1d126bef01 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:46.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.085+0000 7f96227fc700 1 -- 172.21.15.69:0/3790476643 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963001c3c0 con 0x5620ceff7870 2023-07-06T20:19:46.089 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.085+0000 7f96227fc700 1 -- 172.21.15.69:0/3790476643 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9630017040 con 0x5620ceff7870 2023-07-06T20:19:46.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.085+0000 7f96227fc700 1 -- 172.21.15.69:0/3790476643 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96300054e0 con 0x5620ceff7870 2023-07-06T20:19:46.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.086+0000 7f9655648b80 1 -- 172.21.15.69:0/3790476643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 msgr2=0x5620ceffb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.086+0000 7f9655648b80 1 --2- 172.21.15.69:0/3790476643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 0x5620ceffb730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5620cf00eac0 tx=0x7f9630004750 comp rx=0 tx=0).stop 2023-07-06T20:19:46.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.086+0000 7f9655648b80 1 -- 172.21.15.69:0/3790476643 shutdown_connections 2023-07-06T20:19:46.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.086+0000 7f9655648b80 1 --2- 172.21.15.69:0/3790476643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff8390 0x5620cf0079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.086+0000 7f9655648b80 1 --2- 172.21.15.69:0/3790476643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff6f20 0x5620ceffb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.090 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.086+0000 7f9655648b80 1 --2- 172.21.15.69:0/3790476643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff7870 0x5620ceffb730 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.086+0000 7f9655648b80 1 -- 172.21.15.69:0/3790476643 >> 172.21.15.69:0/3790476643 conn(0x5620cefec640 msgr2=0x5620cf0082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:46.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.086+0000 7f9655648b80 1 -- 172.21.15.69:0/3790476643 shutdown_connections 2023-07-06T20:19:46.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.086+0000 7f9655648b80 1 -- 172.21.15.69:0/3790476643 wait complete. 2023-07-06T20:19:46.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.086+0000 7f9655648b80 1 -- start start 2023-07-06T20:19:46.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.087+0000 7f9655648b80 1 Processor -- start 2023-07-06T20:19:46.093 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.087+0000 7f9655648b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff6f20 0x5620cf007350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.093 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.087+0000 7f9655648b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff7870 0x5620cf007890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.087+0000 7f9655648b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff8390 0x5620cf003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.087+0000 7f9655648b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620cf004500 con 0x5620ceff7870 2023-07-06T20:19:46.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.087+0000 7f9655648b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620cf004670 con 0x5620ceff8390 2023-07-06T20:19:46.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.087+0000 7f9655648b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620cf004830 con 0x5620ceff6f20 2023-07-06T20:19:46.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.087+0000 7f9655648b80 10 client.0 _ll_register_callbacks cb 0x7f96300099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:46.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.087+0000 7f963b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff6f20 0x5620cf007350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.094 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.087+0000 7f963bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff8390 0x5620cf003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.087+0000 7f963b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff6f20 0x5620cf007350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43220/0 (socket says 172.21.15.69:43220) 2023-07-06T20:19:46.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.087+0000 7f963b7fe700 1 -- 172.21.15.69:0/3951256049 learned_addr learned my addr 172.21.15.69:0/3951256049 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:46.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.088+0000 7f963affd700 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff7870 0x5620cf007890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.088+0000 7f963b7fe700 1 -- 172.21.15.69:0/3951256049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff8390 msgr2=0x5620cf003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.088+0000 7f963b7fe700 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff8390 0x5620cf003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.095 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.088+0000 7f963b7fe700 1 -- 172.21.15.69:0/3951256049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff7870 msgr2=0x5620cf007890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.088+0000 7f963b7fe700 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff7870 0x5620cf007890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.088+0000 7f963b7fe700 1 -- 172.21.15.69:0/3951256049 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963401ab00 con 0x5620ceff6f20 2023-07-06T20:19:46.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.088+0000 7f963bfff700 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff8390 0x5620cf003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:46.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.088+0000 7f963b7fe700 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff6f20 0x5620cf007350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f962c00b620 tx=0x7f962c0b3440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f721ab2baa56ce5a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:46.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.088+0000 7f961affd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:19:46.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.088+0000 7f961affd700 1 -- 172.21.15.69:0/3951256049 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c09f150 con 0x5620ceff6f20 2023-07-06T20:19:46.096 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.088+0000 7f961affd700 1 -- 172.21.15.69:0/3951256049 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f962c0b92c0 con 0x5620ceff6f20 2023-07-06T20:19:46.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.088+0000 7f9655648b80 1 -- 172.21.15.69:0/3951256049 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f962c0a34e0 con 0x5620ceff6f20 2023-07-06T20:19:46.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.089+0000 7f961affd700 1 -- 172.21.15.69:0/3951256049 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f962c0b94a0 con 0x5620ceff6f20 2023-07-06T20:19:46.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.089+0000 7f9655648b80 10 client.25348 fetch_fsmap learned FSMap version 26 2023-07-06T20:19:46.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.089+0000 7f9655648b80 1 -- 172.21.15.69:0/3951256049 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f963001c960 con 0x5620ceff6f20 2023-07-06T20:19:46.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.089+0000 7f961affd700 1 -- 172.21.15.69:0/3951256049 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f962c0a34e0 con 0x5620ceff6f20 2023-07-06T20:19:46.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.089+0000 7f961affd700 1 -- 172.21.15.69:0/3951256049 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f962c0b3540 con 0x5620ceff6f20 2023-07-06T20:19:46.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.089+0000 7f9655648b80 10 client.25348 fetch_fsmap finished waiting for FSMap version 26 2023-07-06T20:19:46.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.089+0000 7f9655648b80 10 client.25348 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-06T20:19:46.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.089+0000 7f9655648b80 20 client.25348 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:46.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.089+0000 7f9655648b80 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x5620cf060f10 0x5620cf063320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.089+0000 7f9655648b80 4 client.25348 mds_command: new command op to 24539 tid=0[{"prefix": "damage ls"}] 2023-07-06T20:19:46.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.089+0000 7f9655648b80 1 -- 172.21.15.69:0/3951256049 --> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5620cef40eb0 con 0x5620cf060f10 2023-07-06T20:19:46.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.090+0000 7f963affd700 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x5620cf060f10 0x5620cf063320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.090+0000 7f963affd700 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x5620cf060f10 0x5620cf063320 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5589a322287e5d7 server_cookie=4778dc3581a057e0 in_seq=0 out_seq=0 2023-07-06T20:19:46.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.090+0000 7f961affd700 10 client.25348 ms_handle_connect on v2:172.21.15.98:6836/4010267682 2023-07-06T20:19:46.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f961affd700 1 -- 172.21.15.69:0/3951256049 <== mds.1 v2:172.21.15.98:6836/4010267682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5620cef40eb0 con 0x5620cf060f10 2023-07-06T20:19:46.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f961affd700 10 client.25348 handle_command_reply: tid=0 2023-07-06T20:19:46.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f9655648b80 1 client.25348 shutdown 2023-07-06T20:19:46.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f9655648b80 1 -- 172.21.15.69:0/3951256049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff6f20 msgr2=0x5620cf007350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f9655648b80 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff6f20 0x5620cf007350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f962c00b620 tx=0x7f962c0b3440 comp rx=0 tx=0).stop 2023-07-06T20:19:46.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f9655648b80 1 -- 172.21.15.69:0/3951256049 shutdown_connections 2023-07-06T20:19:46.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f9655648b80 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620ceff8390 0x5620cf003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f9655648b80 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620ceff6f20 0x5620cf007350 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f9655648b80 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.98:6836/4010267682,v1:172.21.15.98:6837/4010267682] conn(0x5620cf060f10 0x5620cf063320 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f9655648b80 1 --2- 172.21.15.69:0/3951256049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620ceff7870 0x5620cf007890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f961affd700 0 client.25348 ms_handle_reset on v2:172.21.15.98:6836/4010267682 2023-07-06T20:19:46.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f9655648b80 1 -- 172.21.15.69:0/3951256049 >> 172.21.15.69:0/3951256049 conn(0x5620cefec640 msgr2=0x5620ceffb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:46.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f9655648b80 1 -- 172.21.15.69:0/3951256049 shutdown_connections 2023-07-06T20:19:46.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f9655648b80 1 -- 172.21.15.69:0/3951256049 wait complete. 2023-07-06T20:19:46.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.091+0000 7f9655648b80 20 client.25348 trim_cache size 0 max 16384 2023-07-06T20:19:46.101 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2023-07-06T20:19:46.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.092+0000 7f9641dff700 1 -- 172.21.15.69:0/2153484452 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f96240a1150 msgr2=0x7f96240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.092+0000 7f9641dff700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f96240a1150 0x7f96240a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9634009550 tx=0x7f9634009920 comp rx=0 tx=0).stop 2023-07-06T20:19:46.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.092+0000 7f9641dff700 1 -- 172.21.15.69:0/2153484452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f963c0ffc20 msgr2=0x7f963c0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.092+0000 7f9641dff700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f963c0ffc20 0x7f963c0743f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f962c005740 tx=0x7f962c005770 comp rx=0 tx=0).stop 2023-07-06T20:19:46.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.092+0000 7f9641dff700 1 -- 172.21.15.69:0/2153484452 shutdown_connections 2023-07-06T20:19:46.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.092+0000 7f9641dff700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f96240a1150 0x7f96240a3600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.092+0000 7f9641dff700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f963c1017a0 0x7f963c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.092+0000 7f9641dff700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f963c0ffc20 0x7f963c0743f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.092+0000 7f9641dff700 1 --2- 172.21.15.69:0/2153484452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f963c100e10 0x7f963c072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.092+0000 7f9641dff700 1 -- 172.21.15.69:0/2153484452 >> 172.21.15.69:0/2153484452 conn(0x7f963c0fb260 msgr2=0x7f963c0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:46.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.092+0000 7f9641dff700 1 -- 172.21.15.69:0/2153484452 shutdown_connections 2023-07-06T20:19:46.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.092+0000 7f9641dff700 1 -- 172.21.15.69:0/2153484452 wait complete. 2023-07-06T20:19:46.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:45 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3898478422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:19:46.112 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-06T20:19:46.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:45 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3898478422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:19:46.187 INFO:teuthology.orchestra.run.smithi069.stderr:9500000 of 50000000 tuples (19%) done (elapsed 23.00 s, remaining 98.05 s) 2023-07-06T20:19:46.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.329+0000 7fea17dff700 1 -- 172.21.15.69:0/3981030693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea10100e00 msgr2=0x7fea10101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.329+0000 7fea17dff700 1 --2- 172.21.15.69:0/3981030693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea10100e00 0x7fea10101250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fea00009990 tx=0x7fea00009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.329+0000 7fea17dff700 1 -- 172.21.15.69:0/3981030693 shutdown_connections 2023-07-06T20:19:46.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.329+0000 7fea17dff700 1 --2- 172.21.15.69:0/3981030693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea10101790 0x7fea10108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.329+0000 7fea17dff700 1 --2- 172.21.15.69:0/3981030693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea100ffc10 0x7fea10100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.329+0000 7fea17dff700 1 --2- 172.21.15.69:0/3981030693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea10100e00 0x7fea10101250 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.329+0000 7fea17dff700 1 -- 172.21.15.69:0/3981030693 >> 172.21.15.69:0/3981030693 conn(0x7fea100fb210 msgr2=0x7fea100fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:46.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.329+0000 7fea17dff700 1 -- 172.21.15.69:0/3981030693 shutdown_connections 2023-07-06T20:19:46.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.329+0000 7fea17dff700 1 -- 172.21.15.69:0/3981030693 wait complete. 2023-07-06T20:19:46.334 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.330+0000 7fea17dff700 1 Processor -- start 2023-07-06T20:19:46.335 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.330+0000 7fea17dff700 1 -- start start 2023-07-06T20:19:46.335 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.330+0000 7fea17dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea100ffc10 0x7fea10197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.335 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.330+0000 7fea17dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea10100e00 0x7fea101980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.335 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.330+0000 7fea17dff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea10101790 0x7fea1019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.335 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.330+0000 7fea17dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1019f6e0 con 0x7fea100ffc10 2023-07-06T20:19:46.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.330+0000 7fea17dff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1019f820 con 0x7fea10101790 2023-07-06T20:19:46.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.330+0000 7fea17dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1019f9b0 con 0x7fea10100e00 2023-07-06T20:19:46.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.331+0000 7fea15b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea100ffc10 0x7fea10197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.331+0000 7fea1539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea10100e00 0x7fea101980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.331+0000 7fea1539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea10100e00 0x7fea101980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43242/0 (socket says 172.21.15.69:43242) 2023-07-06T20:19:46.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.331+0000 7fea1539a700 1 -- 172.21.15.69:0/3071245483 learned_addr learned my addr 172.21.15.69:0/3071245483 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:46.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.331+0000 7fea1639c700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea10101790 0x7fea1019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.331+0000 7fea15b9b700 1 -- 172.21.15.69:0/3071245483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea10100e00 msgr2=0x7fea101980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.331+0000 7fea15b9b700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea10100e00 0x7fea101980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.331+0000 7fea15b9b700 1 -- 172.21.15.69:0/3071245483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea10101790 msgr2=0x7fea1019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.331+0000 7fea15b9b700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea10101790 0x7fea1019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.331+0000 7fea15b9b700 1 -- 172.21.15.69:0/3071245483 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea000095f0 con 0x7fea100ffc10 2023-07-06T20:19:46.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.331+0000 7fea15b9b700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea100ffc10 0x7fea10197b60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00e9c0 tx=0x7fea0c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:46.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.332+0000 7fea06ffd700 1 -- 172.21.15.69:0/3071245483 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c004220 con 0x7fea100ffc10 2023-07-06T20:19:46.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.332+0000 7fea17dff700 1 -- 172.21.15.69:0/3071245483 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1019fc90 con 0x7fea100ffc10 2023-07-06T20:19:46.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.332+0000 7fea06ffd700 1 -- 172.21.15.69:0/3071245483 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea0c004380 con 0x7fea100ffc10 2023-07-06T20:19:46.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.332+0000 7fea06ffd700 1 -- 172.21.15.69:0/3071245483 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c003be0 con 0x7fea100ffc10 2023-07-06T20:19:46.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.332+0000 7fea17dff700 1 -- 172.21.15.69:0/3071245483 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea101a01a0 con 0x7fea100ffc10 2023-07-06T20:19:46.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.335+0000 7fea06ffd700 1 -- 172.21.15.69:0/3071245483 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fea0c003d40 con 0x7fea100ffc10 2023-07-06T20:19:46.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.336+0000 7fea06ffd700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe9fc0a1150 0x7fe9fc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.337+0000 7fea06ffd700 1 -- 172.21.15.69:0/3071245483 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fea0c014070 con 0x7fea100ffc10 2023-07-06T20:19:46.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.337+0000 7fea1539a700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe9fc0a1150 0x7fe9fc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.337+0000 7fea1539a700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe9fc0a1150 0x7fe9fc0a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fea00005cd0 tx=0x7fea00005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:46.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.337+0000 7fea2b7cfb80 1 Processor -- start 2023-07-06T20:19:46.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.338+0000 7fea2b7cfb80 1 -- start start 2023-07-06T20:19:46.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.338+0000 7fea2b7cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fa9f20 0x55a073faa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.338+0000 7fea2b7cfb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073faae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.338+0000 7fea2b7cfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fab390 0x55a073fab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.338+0000 7fea2b7cfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a073eaecb0 con 0x55a073fab390 2023-07-06T20:19:46.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.338+0000 7fea2b7cfb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a073f01a40 con 0x55a073faa870 2023-07-06T20:19:46.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.338+0000 7fea2b7cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a073efaed0 con 0x55a073fa9f20 2023-07-06T20:19:46.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.338+0000 7fea15b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fa9f20 0x55a073faa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.338+0000 7fea15b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fa9f20 0x55a073faa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43252/0 (socket says 172.21.15.69:43252) 2023-07-06T20:19:46.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.338+0000 7fea15b9b700 1 -- 172.21.15.69:0/1500246547 learned_addr learned my addr 172.21.15.69:0/1500246547 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:46.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.339+0000 7fea1539a700 1 --2- 172.21.15.69:0/1500246547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073faae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.339+0000 7fea1639c700 1 --2- 172.21.15.69:0/1500246547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fab390 0x55a073fab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.339+0000 7fea15b9b700 1 -- 172.21.15.69:0/1500246547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 msgr2=0x55a073faae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.339+0000 7fea15b9b700 1 --2- 172.21.15.69:0/1500246547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073faae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.339+0000 7fea15b9b700 1 -- 172.21.15.69:0/1500246547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fab390 msgr2=0x55a073fab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.339+0000 7fea15b9b700 1 --2- 172.21.15.69:0/1500246547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fab390 0x55a073fab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.339+0000 7fea15b9b700 1 -- 172.21.15.69:0/1500246547 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a073d28de0 con 0x55a073fa9f20 2023-07-06T20:19:46.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.339+0000 7fea1539a700 1 --2- 172.21.15.69:0/1500246547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073faae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:19:46.344 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.339+0000 7fea15b9b700 1 --2- 172.21.15.69:0/1500246547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fa9f20 0x55a073faa330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea0c099f40 tx=0x7fea0c099990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=643e6492e781b8ea server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:46.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.339+0000 7fe9fbfff700 1 -- 172.21.15.69:0/1500246547 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c099fb0 con 0x55a073fa9f20 2023-07-06T20:19:46.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.339+0000 7fe9fbfff700 1 -- 172.21.15.69:0/1500246547 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea0c09dfe0 con 0x55a073fa9f20 2023-07-06T20:19:46.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.339+0000 7fe9fbfff700 1 -- 172.21.15.69:0/1500246547 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c099700 con 0x55a073fa9f20 2023-07-06T20:19:46.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.340+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/1500246547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fa9f20 msgr2=0x55a073faa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.340+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/1500246547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fa9f20 0x55a073faa330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea0c099f40 tx=0x7fea0c099990 comp rx=0 tx=0).stop 2023-07-06T20:19:46.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.340+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/1500246547 shutdown_connections 2023-07-06T20:19:46.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.340+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/1500246547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fab390 0x55a073fab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.340+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/1500246547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fa9f20 0x55a073faa330 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.340+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/1500246547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073faae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.340+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/1500246547 >> 172.21.15.69:0/1500246547 conn(0x55a073f9f640 msgr2=0x55a073fa49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:46.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.340+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/1500246547 shutdown_connections 2023-07-06T20:19:46.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.340+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/1500246547 wait complete. 2023-07-06T20:19:46.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.341+0000 7fea2b7cfb80 1 -- start start 2023-07-06T20:19:46.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.341+0000 7fea2b7cfb80 1 Processor -- start 2023-07-06T20:19:46.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.342+0000 7fea2b7cfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 0x55a073fb3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.342+0000 7fea2b7cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073faa870 0x55a073fb3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.342+0000 7fea2b7cfb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073fab390 0x55a073fbaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.342+0000 7fea2b7cfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a073e9aac0 con 0x55a073fa9f20 2023-07-06T20:19:46.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.342+0000 7fea2b7cfb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a073e9a7c0 con 0x55a073fab390 2023-07-06T20:19:46.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.342+0000 7fea2b7cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a073d7e920 con 0x55a073faa870 2023-07-06T20:19:46.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.342+0000 7fea2b7cfb80 10 client.0 _ll_register_callbacks cb 0x55a073e98760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:46.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.342+0000 7fea15b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 0x55a073fb3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.342+0000 7fea15b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 0x55a073fb3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47396/0 (socket says 172.21.15.69:47396) 2023-07-06T20:19:46.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.342+0000 7fea15b9b700 1 -- 172.21.15.69:0/2017446107 learned_addr learned my addr 172.21.15.69:0/2017446107 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:46.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.342+0000 7fea1639c700 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073fab390 0x55a073fbaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.342+0000 7fea1539a700 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073faa870 0x55a073fb3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.343+0000 7fea1539a700 1 -- 172.21.15.69:0/2017446107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073fab390 msgr2=0x55a073fbaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.343+0000 7fea1539a700 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073fab390 0x55a073fbaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.343+0000 7fea1539a700 1 -- 172.21.15.69:0/2017446107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 msgr2=0x55a073fb3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.343+0000 7fea1539a700 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 0x55a073fb3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.343+0000 7fea1539a700 1 -- 172.21.15.69:0/2017446107 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea101a01a0 con 0x55a073faa870 2023-07-06T20:19:46.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.343+0000 7fea1639c700 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073fab390 0x55a073fbaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:46.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.343+0000 7fea1539a700 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073faa870 0x55a073fb3e60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea00006010 tx=0x7fea00009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c46c06f80ee660a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:46.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.343+0000 7fe9f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:19:46.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.344+0000 7fe9f97fa700 1 -- 172.21.15.69:0/2017446107 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0000f660 con 0x55a073faa870 2023-07-06T20:19:46.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.344+0000 7fe9f97fa700 1 -- 172.21.15.69:0/2017446107 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fea0000f7c0 con 0x55a073faa870 2023-07-06T20:19:46.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.344+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2017446107 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a073e544f0 con 0x55a073faa870 2023-07-06T20:19:46.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.344+0000 7fe9f97fa700 1 -- 172.21.15.69:0/2017446107 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea00004e50 con 0x55a073faa870 2023-07-06T20:19:46.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.344+0000 7fea2b7cfb80 10 client.25360 fetch_fsmap learned FSMap version 26 2023-07-06T20:19:46.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.344+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2017446107 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a073d11c60 con 0x55a073faa870 2023-07-06T20:19:46.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.344+0000 7fe9f97fa700 1 -- 172.21.15.69:0/2017446107 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a073e544f0 con 0x55a073faa870 2023-07-06T20:19:46.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.344+0000 7fe9f97fa700 1 -- 172.21.15.69:0/2017446107 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fea00031a10 con 0x55a073faa870 2023-07-06T20:19:46.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.345+0000 7fea2b7cfb80 10 client.25360 fetch_fsmap finished waiting for FSMap version 26 2023-07-06T20:19:46.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.345+0000 7fea2b7cfb80 10 client.25360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:19:46.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.345+0000 7fea2b7cfb80 20 client.25360 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:46.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.345+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a073fbf740 0x55a073fc1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.345+0000 7fea2b7cfb80 4 client.25360 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:19:46.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.345+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2017446107 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a073f78d60 con 0x55a073fbf740 2023-07-06T20:19:46.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.345+0000 7fea15b9b700 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a073fbf740 0x55a073fc1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.345+0000 7fea15b9b700 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a073fbf740 0x55a073fc1b50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c753d6ad88caab server_cookie=5636220f1f14e45d in_seq=0 out_seq=0 2023-07-06T20:19:46.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.345+0000 7fe9f97fa700 10 client.25360 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:46.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fe9f97fa700 1 -- 172.21.15.69:0/2017446107 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a073f78d60 con 0x55a073fbf740 2023-07-06T20:19:46.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fe9f97fa700 10 client.25360 handle_command_reply: tid=0 2023-07-06T20:19:46.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fea2b7cfb80 1 client.25360 shutdown 2023-07-06T20:19:46.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2017446107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073faa870 msgr2=0x55a073fb3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073faa870 0x55a073fb3e60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea00006010 tx=0x7fea00009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2017446107 shutdown_connections 2023-07-06T20:19:46.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a073fbf740 0x55a073fc1b50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.355 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073fab390 0x55a073fbaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 0x55a073fb3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fe9f97fa700 0 client.25360 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:46.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/2017446107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073faa870 0x55a073fb3e60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2017446107 >> 172.21.15.69:0/2017446107 conn(0x55a073f9f640 msgr2=0x55a073fa40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:46.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2017446107 shutdown_connections 2023-07-06T20:19:46.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.346+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2017446107 wait complete. 2023-07-06T20:19:46.356 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.347+0000 7fea2b7cfb80 20 client.25360 trim_cache size 0 max 16384 2023-07-06T20:19:46.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.360+0000 7fea2b7cfb80 1 Processor -- start 2023-07-06T20:19:46.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.360+0000 7fea2b7cfb80 1 -- start start 2023-07-06T20:19:46.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.360+0000 7fea2b7cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fa9f20 0x55a073fb25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.360+0000 7fea2b7cfb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073fbae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.360+0000 7fea2b7cfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fab390 0x55a073fb3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.360+0000 7fea2b7cfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a073faac40 con 0x55a073fab390 2023-07-06T20:19:46.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.360+0000 7fea2b7cfb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a073fa6420 con 0x55a073faa870 2023-07-06T20:19:46.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.360+0000 7fea2b7cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a073fbcb10 con 0x55a073fa9f20 2023-07-06T20:19:46.364 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.360+0000 7fea1639c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fab390 0x55a073fb3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.361+0000 7fea1539a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073fbae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.360+0000 7fea1639c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fab390 0x55a073fb3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47410/0 (socket says 172.21.15.69:47410) 2023-07-06T20:19:46.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.361+0000 7fea1639c700 1 -- 172.21.15.69:0/1841944250 learned_addr learned my addr 172.21.15.69:0/1841944250 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:46.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.361+0000 7fea1639c700 1 -- 172.21.15.69:0/1841944250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fa9f20 msgr2=0x55a073fb25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.361+0000 7fea1639c700 1 --2- 172.21.15.69:0/1841944250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fa9f20 0x55a073fb25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.361+0000 7fea1639c700 1 -- 172.21.15.69:0/1841944250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 msgr2=0x55a073fbae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.365 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.361+0000 7fea1639c700 1 --2- 172.21.15.69:0/1841944250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073fbae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.361+0000 7fea1639c700 1 -- 172.21.15.69:0/1841944250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a073d4f250 con 0x55a073fab390 2023-07-06T20:19:46.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.361+0000 7fea1539a700 1 --2- 172.21.15.69:0/1841944250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073fbae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:46.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.361+0000 7fea1639c700 1 --2- 172.21.15.69:0/1841944250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fab390 0x55a073fb3920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea080079e0 tx=0x7fea0800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c1cecdc0db97bb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:46.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.361+0000 7fea04bff700 1 -- 172.21.15.69:0/1841944250 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0800e040 con 0x55a073fab390 2023-07-06T20:19:46.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.361+0000 7fea04bff700 1 -- 172.21.15.69:0/1841944250 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea0800f040 con 0x55a073fab390 2023-07-06T20:19:46.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.361+0000 7fea04bff700 1 -- 172.21.15.69:0/1841944250 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08005020 con 0x55a073fab390 2023-07-06T20:19:46.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.362+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/1841944250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fab390 msgr2=0x55a073fb3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.362+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/1841944250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fab390 0x55a073fb3920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea080079e0 tx=0x7fea0800b040 comp rx=0 tx=0).stop 2023-07-06T20:19:46.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.362+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/1841944250 shutdown_connections 2023-07-06T20:19:46.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.362+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/1841944250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fab390 0x55a073fb3920 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.362+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/1841944250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fa9f20 0x55a073fb25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.362+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/1841944250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073fbae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.362+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/1841944250 >> 172.21.15.69:0/1841944250 conn(0x55a073f9f640 msgr2=0x55a073fbb790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:46.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.362+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/1841944250 shutdown_connections 2023-07-06T20:19:46.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.362+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/1841944250 wait complete. 2023-07-06T20:19:46.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea2b7cfb80 1 -- start start 2023-07-06T20:19:46.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea2b7cfb80 1 Processor -- start 2023-07-06T20:19:46.370 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea2b7cfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 0x55a073fb6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.370 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea2b7cfb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073fb7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.370 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea2b7cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fab390 0x55a073fba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.371 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea2b7cfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a073fba680 con 0x55a073fa9f20 2023-07-06T20:19:46.371 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea2b7cfb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a073fba7f0 con 0x55a073faa870 2023-07-06T20:19:46.371 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea2b7cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a073fba9b0 con 0x55a073fab390 2023-07-06T20:19:46.371 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea2b7cfb80 10 client.0 _ll_register_callbacks cb 0x55a073e98760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:46.371 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea15b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 0x55a073fb6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.371 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea1539a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073fb7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea15b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 0x55a073fb6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47412/0 (socket says 172.21.15.69:47412) 2023-07-06T20:19:46.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.363+0000 7fea15b9b700 1 -- 172.21.15.69:0/2474886856 learned_addr learned my addr 172.21.15.69:0/2474886856 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:46.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fea1639c700 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fab390 0x55a073fba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fea1539a700 1 -- 172.21.15.69:0/2474886856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fab390 msgr2=0x55a073fba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fea1539a700 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fab390 0x55a073fba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fea1539a700 1 -- 172.21.15.69:0/2474886856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 msgr2=0x55a073fb6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:46.372 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fea1539a700 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 0x55a073fb6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.373 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fea1539a700 1 -- 172.21.15.69:0/2474886856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a073d11c60 con 0x55a073faa870 2023-07-06T20:19:46.373 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fea1639c700 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fab390 0x55a073fba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:19:46.373 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fea15b9b700 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 0x55a073fb6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:46.373 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fea1539a700 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073fb7120 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea00031df0 tx=0x7fea0003bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d59c8b685a9a314f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:46.373 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fe9f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:19:46.373 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fe9f97fa700 1 -- 172.21.15.69:0/2474886856 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00029070 con 0x55a073faa870 2023-07-06T20:19:46.373 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2474886856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea0c09dfe0 con 0x55a073faa870 2023-07-06T20:19:46.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.364+0000 7fe9f97fa700 1 -- 172.21.15.69:0/2474886856 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fea00031640 con 0x55a073faa870 2023-07-06T20:19:46.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.365+0000 7fe9f97fa700 1 -- 172.21.15.69:0/2474886856 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea0000beb0 con 0x55a073faa870 2023-07-06T20:19:46.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.365+0000 7fea2b7cfb80 10 client.25409 fetch_fsmap learned FSMap version 26 2023-07-06T20:19:46.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.365+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2474886856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fea08007690 con 0x55a073faa870 2023-07-06T20:19:46.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.365+0000 7fe9f97fa700 1 -- 172.21.15.69:0/2474886856 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea0c09dfe0 con 0x55a073faa870 2023-07-06T20:19:46.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.365+0000 7fe9f97fa700 1 -- 172.21.15.69:0/2474886856 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fea0000fa80 con 0x55a073faa870 2023-07-06T20:19:46.374 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.365+0000 7fea2b7cfb80 10 client.25409 fetch_fsmap finished waiting for FSMap version 26 2023-07-06T20:19:46.375 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.365+0000 7fea2b7cfb80 10 client.25409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:19:46.375 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.365+0000 7fea2b7cfb80 20 client.25409 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:46.375 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.365+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a074014160 0x55a074016570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:46.375 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.365+0000 7fea2b7cfb80 4 client.25409 mds_command: new command op to 24584 tid=0[{"prefix": "damage ls"}] 2023-07-06T20:19:46.375 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.365+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2474886856 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a073ef3eb0 con 0x55a074014160 2023-07-06T20:19:46.375 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.365+0000 7fea15b9b700 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a074014160 0x55a074016570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:46.375 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fe9f97fa700 10 client.25409 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:46.375 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fea15b9b700 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a074014160 0x55a074016570 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2573768669077e72 server_cookie=2e119bc453cd46db in_seq=0 out_seq=0 2023-07-06T20:19:46.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fe9f97fa700 1 -- 172.21.15.69:0/2474886856 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a073ef3eb0 con 0x55a074014160 2023-07-06T20:19:46.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fe9f97fa700 10 client.25409 handle_command_reply: tid=0 2023-07-06T20:19:46.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fea2b7cfb80 1 client.25409 shutdown 2023-07-06T20:19:46.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2474886856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 msgr2=0x55a073fb7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073fb7120 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea00031df0 tx=0x7fea0003bf30 comp rx=0 tx=0).stop 2023-07-06T20:19:46.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2474886856 shutdown_connections 2023-07-06T20:19:46.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a074014160 0x55a074016570 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a073fab390 0x55a073fba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a073fa9f20 0x55a073fb6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fe9f97fa700 0 client.25409 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:46.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fea2b7cfb80 1 --2- 172.21.15.69:0/2474886856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a073faa870 0x55a073fb7120 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.366+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2474886856 >> 172.21.15.69:0/2474886856 conn(0x55a073f9f640 msgr2=0x55a073fb25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:46.377 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.367+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2474886856 shutdown_connections 2023-07-06T20:19:46.378 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.367+0000 7fea2b7cfb80 1 -- 172.21.15.69:0/2474886856 wait complete. 2023-07-06T20:19:46.378 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.367+0000 7fea2b7cfb80 20 client.25409 trim_cache size 0 max 16384 2023-07-06T20:19:46.378 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2023-07-06T20:19:46.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.367+0000 7fea17dff700 1 -- 172.21.15.69:0/3071245483 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe9fc0a1150 msgr2=0x7fe9fc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.367+0000 7fea17dff700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe9fc0a1150 0x7fe9fc0a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fea00005cd0 tx=0x7fea00005c60 comp rx=0 tx=0).stop 2023-07-06T20:19:46.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.367+0000 7fea17dff700 1 -- 172.21.15.69:0/3071245483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea100ffc10 msgr2=0x7fea10197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:46.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.367+0000 7fea17dff700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea100ffc10 0x7fea10197b60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00e9c0 tx=0x7fea0c00ecd0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.379 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.367+0000 7fea17dff700 1 -- 172.21.15.69:0/3071245483 shutdown_connections 2023-07-06T20:19:46.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.367+0000 7fea17dff700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe9fc0a1150 0x7fe9fc0a3600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.367+0000 7fea17dff700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea10101790 0x7fea1019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.367+0000 7fea17dff700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea100ffc10 0x7fea10197b60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.367+0000 7fea17dff700 1 --2- 172.21.15.69:0/3071245483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea10100e00 0x7fea101980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:46.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.368+0000 7fea17dff700 1 -- 172.21.15.69:0/3071245483 >> 172.21.15.69:0/3071245483 conn(0x7fea100fb210 msgr2=0x7fea101066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:46.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.368+0000 7fea17dff700 1 -- 172.21.15.69:0/3071245483 shutdown_connections 2023-07-06T20:19:46.381 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:46.368+0000 7fea17dff700 1 -- 172.21.15.69:0/3071245483 wait complete. 2023-07-06T20:19:46.459 INFO:teuthology.orchestra.run.smithi069.stderr:9600000 of 50000000 tuples (19%) done (elapsed 23.27 s, remaining 97.93 s) 2023-07-06T20:19:46.771 INFO:teuthology.orchestra.run.smithi069.stderr:9700000 of 50000000 tuples (19%) done (elapsed 23.58 s, remaining 97.98 s) 2023-07-06T20:19:47.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:46 smithi069 ceph-mon[126399]: pgmap v341: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 148 MiB/s wr, 71 op/s 2023-07-06T20:19:47.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:46 smithi156 ceph-mon[129280]: pgmap v341: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 148 MiB/s wr, 71 op/s 2023-07-06T20:19:47.119 INFO:teuthology.orchestra.run.smithi069.stderr:9800000 of 50000000 tuples (19%) done (elapsed 23.93 s, remaining 98.16 s) 2023-07-06T20:19:47.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:46 smithi098 ceph-mon[128968]: pgmap v341: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 148 MiB/s wr, 71 op/s 2023-07-06T20:19:47.388 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-06T20:19:47.388 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-06T20:19:47.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.824+0000 7f5b3e99d700 1 -- 172.21.15.69:0/1335850221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b380ffb20 msgr2=0x7f5b380fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:47.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.824+0000 7f5b3e99d700 1 --2- 172.21.15.69:0/1335850221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b380ffb20 0x7f5b380fff30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009990 tx=0x7f5b34009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.824+0000 7f5b3e99d700 1 -- 172.21.15.69:0/1335850221 shutdown_connections 2023-07-06T20:19:47.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.824+0000 7f5b3e99d700 1 --2- 172.21.15.69:0/1335850221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b38101760 0x7f5b38108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.826 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.824+0000 7f5b3e99d700 1 --2- 172.21.15.69:0/1335850221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b380ffb20 0x7f5b380fff30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.824+0000 7f5b3e99d700 1 --2- 172.21.15.69:0/1335850221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b38100db0 0x7f5b38101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.824+0000 7f5b3e99d700 1 -- 172.21.15.69:0/1335850221 >> 172.21.15.69:0/1335850221 conn(0x7f5b380fb140 msgr2=0x7f5b380fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:47.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.824+0000 7f5b3e99d700 1 -- 172.21.15.69:0/1335850221 shutdown_connections 2023-07-06T20:19:47.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.824+0000 7f5b3e99d700 1 -- 172.21.15.69:0/1335850221 wait complete. 2023-07-06T20:19:47.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.825+0000 7f5b3e99d700 1 Processor -- start 2023-07-06T20:19:47.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.825+0000 7f5b3e99d700 1 -- start start 2023-07-06T20:19:47.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.825+0000 7f5b3e99d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b380ffb20 0x7f5b38193510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.825+0000 7f5b3e99d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b38100db0 0x7f5b38193a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.825+0000 7f5b3e99d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b38101760 0x7f5b3819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.825+0000 7f5b3e99d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3819b090 con 0x7f5b38101760 2023-07-06T20:19:47.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.825+0000 7f5b3e99d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3819b1d0 con 0x7f5b38100db0 2023-07-06T20:19:47.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.825+0000 7f5b3e99d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3819b360 con 0x7f5b380ffb20 2023-07-06T20:19:47.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.826+0000 7f5b3d19a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b38100db0 0x7f5b38193a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.826+0000 7f5b3d99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b380ffb20 0x7f5b38193510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.826+0000 7f5b3d19a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b38100db0 0x7f5b38193a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:37018/0 (socket says 172.21.15.69:37018) 2023-07-06T20:19:47.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.826+0000 7f5b3d19a700 1 -- 172.21.15.69:0/234890504 learned_addr learned my addr 172.21.15.69:0/234890504 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:47.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.826+0000 7f5b3e19c700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b38101760 0x7f5b3819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.826+0000 7f5b3d19a700 1 -- 172.21.15.69:0/234890504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b380ffb20 msgr2=0x7f5b38193510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:47.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.826+0000 7f5b3d19a700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b380ffb20 0x7f5b38193510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.826+0000 7f5b3d19a700 1 -- 172.21.15.69:0/234890504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b38101760 msgr2=0x7f5b3819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:47.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.826+0000 7f5b3d19a700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b38101760 0x7f5b3819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.826+0000 7f5b3d19a700 1 -- 172.21.15.69:0/234890504 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b340095f0 con 0x7f5b38100db0 2023-07-06T20:19:47.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.826+0000 7f5b3d19a700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b38100db0 0x7f5b38193a50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5b28012fc0 tx=0x7f5b280100c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:47.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.827+0000 7f5b2effd700 1 -- 172.21.15.69:0/234890504 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800dba0 con 0x7f5b38100db0 2023-07-06T20:19:47.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.827+0000 7f5b3e99d700 1 -- 172.21.15.69:0/234890504 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3819b5e0 con 0x7f5b38100db0 2023-07-06T20:19:47.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.827+0000 7f5b2effd700 1 -- 172.21.15.69:0/234890504 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b28013040 con 0x7f5b38100db0 2023-07-06T20:19:47.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.827+0000 7f5b2effd700 1 -- 172.21.15.69:0/234890504 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800bd40 con 0x7f5b38100db0 2023-07-06T20:19:47.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.827+0000 7f5b3e99d700 1 -- 172.21.15.69:0/234890504 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3819bb40 con 0x7f5b38100db0 2023-07-06T20:19:47.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.830+0000 7f5b2effd700 1 -- 172.21.15.69:0/234890504 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5b28010740 con 0x7f5b38100db0 2023-07-06T20:19:47.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.831+0000 7f5b2effd700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5b240a11f0 0x7f5b240a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.831+0000 7f5b5357db80 1 Processor -- start 2023-07-06T20:19:47.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.831+0000 7f5b2effd700 1 -- 172.21.15.69:0/234890504 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f5b280d09a0 con 0x7f5b38100db0 2023-07-06T20:19:47.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.832+0000 7f5b5357db80 1 -- start start 2023-07-06T20:19:47.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.832+0000 7f5b3d99b700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5b240a11f0 0x7f5b240a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.832+0000 7f5b5357db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19477970 0x558e19477d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.832+0000 7f5b5357db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e194782c0 0x558e194788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.832+0000 7f5b5357db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e19479320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.832+0000 7f5b5357db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e19373d30 con 0x558e194782c0 2023-07-06T20:19:47.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.832+0000 7f5b5357db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e193c6ac0 con 0x558e19478eb0 2023-07-06T20:19:47.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.832+0000 7f5b5357db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e193bff50 con 0x558e19477970 2023-07-06T20:19:47.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.833+0000 7f5b3e19c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e19479320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.833+0000 7f5b3e19c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e19479320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:37024/0 (socket says 172.21.15.69:37024) 2023-07-06T20:19:47.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.833+0000 7f5b3e19c700 1 -- 172.21.15.69:0/2242271169 learned_addr learned my addr 172.21.15.69:0/2242271169 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:47.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.833+0000 7f5b3d19a700 1 --2- 172.21.15.69:0/2242271169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e194782c0 0x558e194788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.833+0000 7f5b3d99b700 1 --2- 172.21.15.69:0/2242271169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19477970 0x558e19477d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.833+0000 7f5b3e19c700 1 -- 172.21.15.69:0/2242271169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19477970 msgr2=0x558e19477d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:47.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.833+0000 7f5b3e19c700 1 --2- 172.21.15.69:0/2242271169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19477970 0x558e19477d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.833+0000 7f5b3e19c700 1 -- 172.21.15.69:0/2242271169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e194782c0 msgr2=0x558e194788b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:47.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.833+0000 7f5b3d99b700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5b240a11f0 0x7f5b240a36a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400b2e0 tx=0x7f5b34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:47.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.833+0000 7f5b3e19c700 1 --2- 172.21.15.69:0/2242271169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e194782c0 0x558e194788b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.833+0000 7f5b3e19c700 1 -- 172.21.15.69:0/2242271169 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e191ede60 con 0x558e19478eb0 2023-07-06T20:19:47.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.833+0000 7f5b3e19c700 1 --2- 172.21.15.69:0/2242271169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e19479320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5b30011680 tx=0x7f5b30009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a781f1edea69b3c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:47.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.834+0000 7f5b23fff700 1 -- 172.21.15.69:0/2242271169 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3000e040 con 0x558e19478eb0 2023-07-06T20:19:47.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.834+0000 7f5b23fff700 1 -- 172.21.15.69:0/2242271169 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b3000f040 con 0x558e19478eb0 2023-07-06T20:19:47.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.834+0000 7f5b23fff700 1 -- 172.21.15.69:0/2242271169 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b300048c0 con 0x558e19478eb0 2023-07-06T20:19:47.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.834+0000 7f5b5357db80 1 -- 172.21.15.69:0/2242271169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 msgr2=0x558e19479320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:47.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.834+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2242271169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e19479320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5b30011680 tx=0x7f5b30009920 comp rx=0 tx=0).stop 2023-07-06T20:19:47.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.834+0000 7f5b5357db80 1 -- 172.21.15.69:0/2242271169 shutdown_connections 2023-07-06T20:19:47.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.834+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2242271169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e19479320 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.834+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2242271169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19477970 0x558e19477d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.834+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2242271169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e194782c0 0x558e194788b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.834+0000 7f5b5357db80 1 -- 172.21.15.69:0/2242271169 >> 172.21.15.69:0/2242271169 conn(0x558e19464820 msgr2=0x558e19469b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:47.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.835+0000 7f5b5357db80 1 -- 172.21.15.69:0/2242271169 shutdown_connections 2023-07-06T20:19:47.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.835+0000 7f5b5357db80 1 -- 172.21.15.69:0/2242271169 wait complete. 2023-07-06T20:19:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.835+0000 7f5b5357db80 1 -- start start 2023-07-06T20:19:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.836+0000 7f5b5357db80 1 Processor -- start 2023-07-06T20:19:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.836+0000 7f5b5357db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19477970 0x558e194812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.836+0000 7f5b5357db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e19481800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.836+0000 7f5b5357db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19486830 0x558e19488c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.836+0000 7f5b5357db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1935fb40 con 0x558e19486830 2023-07-06T20:19:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.836+0000 7f5b5357db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1935f840 con 0x558e19478eb0 2023-07-06T20:19:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.836+0000 7f5b5357db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e192439a0 con 0x558e19477970 2023-07-06T20:19:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.836+0000 7f5b5357db80 10 client.0 _ll_register_callbacks cb 0x558e1935d7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.837+0000 7f5b3d19a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e19481800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.837+0000 7f5b3d19a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e19481800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:37038/0 (socket says 172.21.15.69:37038) 2023-07-06T20:19:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.837+0000 7f5b3d19a700 1 -- 172.21.15.69:0/2817808780 learned_addr learned my addr 172.21.15.69:0/2817808780 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.837+0000 7f5b3d99b700 1 --2- 172.21.15.69:0/2817808780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19477970 0x558e194812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.837+0000 7f5b3d19a700 1 -- 172.21.15.69:0/2817808780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19477970 msgr2=0x558e194812c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:47.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.837+0000 7f5b3d19a700 1 --2- 172.21.15.69:0/2817808780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19477970 0x558e194812c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.837+0000 7f5b3d19a700 1 -- 172.21.15.69:0/2817808780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19486830 msgr2=0x558e19488c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:19:47.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.837+0000 7f5b3d19a700 1 --2- 172.21.15.69:0/2817808780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19486830 0x558e19488c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.837+0000 7f5b3d19a700 1 -- 172.21.15.69:0/2817808780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b30011330 con 0x558e19478eb0 2023-07-06T20:19:47.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.837+0000 7f5b3d19a700 1 --2- 172.21.15.69:0/2817808780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e19481800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b2809da20 tx=0x7f5b28010cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d5c45709435f9f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:47.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.838+0000 7f5b217fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:19:47.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.838+0000 7f5b217fa700 1 -- 172.21.15.69:0/2817808780 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b280a6170 con 0x558e19478eb0 2023-07-06T20:19:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.838+0000 7f5b217fa700 1 -- 172.21.15.69:0/2817808780 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f5b2800b370 con 0x558e19478eb0 2023-07-06T20:19:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.838+0000 7f5b5357db80 1 -- 172.21.15.69:0/2817808780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e19319570 con 0x558e19478eb0 2023-07-06T20:19:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.838+0000 7f5b217fa700 1 -- 172.21.15.69:0/2817808780 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b2800b530 con 0x558e19478eb0 2023-07-06T20:19:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.838+0000 7f5b5357db80 10 client.25421 fetch_fsmap learned FSMap version 27 2023-07-06T20:19:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.838+0000 7f5b5357db80 1 -- 172.21.15.69:0/2817808780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558e191d6ce0 con 0x558e19478eb0 2023-07-06T20:19:47.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.838+0000 7f5b217fa700 1 -- 172.21.15.69:0/2817808780 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b280ba080 con 0x558e19478eb0 2023-07-06T20:19:47.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.838+0000 7f5b217fa700 1 -- 172.21.15.69:0/2817808780 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5b280bbf90 con 0x558e19478eb0 2023-07-06T20:19:47.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.840+0000 7f5b5357db80 10 client.25421 fetch_fsmap finished waiting for FSMap version 27 2023-07-06T20:19:47.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.840+0000 7f5b5357db80 10 client.25421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:19:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.840+0000 7f5b5357db80 20 client.25421 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.840+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2817808780 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558e1948d4d0 0x558e1948f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.840+0000 7f5b5357db80 4 client.25421 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:19:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.840+0000 7f5b5357db80 1 -- 172.21.15.69:0/2817808780 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e1943dde0 con 0x558e1948d4d0 2023-07-06T20:19:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.841+0000 7f5b3d99b700 1 --2- 172.21.15.69:0/2817808780 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558e1948d4d0 0x558e1948f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.842+0000 7f5b3d99b700 1 --2- 172.21.15.69:0/2817808780 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558e1948d4d0 0x558e1948f8e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2561cee35b24a042 server_cookie=ee938c98f79cb615 in_seq=0 out_seq=0 2023-07-06T20:19:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.842+0000 7f5b217fa700 10 client.25421 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:47.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b217fa700 1 -- 172.21.15.69:0/2817808780 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558e1943dde0 con 0x558e1948d4d0 2023-07-06T20:19:47.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b217fa700 10 client.25421 handle_command_reply: tid=0 2023-07-06T20:19:47.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b5357db80 1 client.25421 shutdown 2023-07-06T20:19:47.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b5357db80 1 -- 172.21.15.69:0/2817808780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 msgr2=0x558e19481800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:47.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2817808780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e19481800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b2809da20 tx=0x7f5b28010cd0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b5357db80 1 -- 172.21.15.69:0/2817808780 shutdown_connections 2023-07-06T20:19:47.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2817808780 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558e1948d4d0 0x558e1948f8e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2817808780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19486830 0x558e19488c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2817808780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19477970 0x558e194812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b217fa700 0 client.25421 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2817808780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e19481800 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b5357db80 1 -- 172.21.15.69:0/2817808780 >> 172.21.15.69:0/2817808780 conn(0x558e19464820 msgr2=0x558e19469290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:47.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.843+0000 7f5b5357db80 1 -- 172.21.15.69:0/2817808780 shutdown_connections 2023-07-06T20:19:47.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.844+0000 7f5b5357db80 1 -- 172.21.15.69:0/2817808780 wait complete. 2023-07-06T20:19:47.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.844+0000 7f5b5357db80 20 client.25421 trim_cache size 0 max 16384 2023-07-06T20:19:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.862+0000 7f5b5357db80 1 Processor -- start 2023-07-06T20:19:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.862+0000 7f5b5357db80 1 -- start start 2023-07-06T20:19:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b5357db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19477970 0x558e1949d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b5357db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e1947bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b5357db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19486830 0x558e1947c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b5357db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e19478d10 con 0x558e19477970 2023-07-06T20:19:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b5357db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1946b680 con 0x558e19478eb0 2023-07-06T20:19:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b5357db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1946ac30 con 0x558e19486830 2023-07-06T20:19:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b3d19a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e1947bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b3d99b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19477970 0x558e1949d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b3d19a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e1947bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:37052/0 (socket says 172.21.15.69:37052) 2023-07-06T20:19:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b3d19a700 1 -- 172.21.15.69:0/3927914102 learned_addr learned my addr 172.21.15.69:0/3927914102 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b3d99b700 1 -- 172.21.15.69:0/3927914102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19486830 msgr2=0x558e1947c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b3d99b700 1 --2- 172.21.15.69:0/3927914102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19486830 0x558e1947c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b3d99b700 1 -- 172.21.15.69:0/3927914102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 msgr2=0x558e1947bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.863+0000 7f5b3d99b700 1 --2- 172.21.15.69:0/3927914102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e1947bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.864+0000 7f5b3d99b700 1 -- 172.21.15.69:0/3927914102 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e19480330 con 0x558e19477970 2023-07-06T20:19:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.864+0000 7f5b3d19a700 1 --2- 172.21.15.69:0/3927914102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e1947bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.864+0000 7f5b3d99b700 1 --2- 172.21.15.69:0/3927914102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19477970 0x558e1949d130 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5b34000c00 tx=0x7f5b34020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901f2fd81e81bbd0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.864+0000 7f5b2cbff700 1 -- 172.21.15.69:0/3927914102 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34032d10 con 0x558e19477970 2023-07-06T20:19:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.864+0000 7f5b2cbff700 1 -- 172.21.15.69:0/3927914102 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b3402a030 con 0x558e19477970 2023-07-06T20:19:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.864+0000 7f5b2cbff700 1 -- 172.21.15.69:0/3927914102 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400fa30 con 0x558e19477970 2023-07-06T20:19:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.865+0000 7f5b5357db80 1 -- 172.21.15.69:0/3927914102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19477970 msgr2=0x558e1949d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.865+0000 7f5b5357db80 1 --2- 172.21.15.69:0/3927914102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19477970 0x558e1949d130 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5b34000c00 tx=0x7f5b34020000 comp rx=0 tx=0).stop 2023-07-06T20:19:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.865+0000 7f5b5357db80 1 -- 172.21.15.69:0/3927914102 shutdown_connections 2023-07-06T20:19:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.865+0000 7f5b5357db80 1 --2- 172.21.15.69:0/3927914102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19486830 0x558e1947c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.865+0000 7f5b5357db80 1 --2- 172.21.15.69:0/3927914102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19477970 0x558e1949d130 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.865+0000 7f5b5357db80 1 --2- 172.21.15.69:0/3927914102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19478eb0 0x558e1947bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.865+0000 7f5b5357db80 1 -- 172.21.15.69:0/3927914102 >> 172.21.15.69:0/3927914102 conn(0x558e19464820 msgr2=0x558e194894d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.865+0000 7f5b5357db80 1 -- 172.21.15.69:0/3927914102 shutdown_connections 2023-07-06T20:19:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.865+0000 7f5b5357db80 1 -- 172.21.15.69:0/3927914102 wait complete. 2023-07-06T20:19:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.865+0000 7f5b5357db80 1 -- start start 2023-07-06T20:19:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b5357db80 1 Processor -- start 2023-07-06T20:19:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b5357db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19477970 0x558e1949ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b5357db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19478eb0 0x558e1949bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b5357db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19486830 0x558e1949c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b5357db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1949d680 con 0x558e19478eb0 2023-07-06T20:19:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b5357db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e194e1ae0 con 0x558e19477970 2023-07-06T20:19:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b5357db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e194e1c20 con 0x558e19486830 2023-07-06T20:19:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b5357db80 10 client.0 _ll_register_callbacks cb 0x7f5b2809dbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b3d99b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19477970 0x558e1949ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b3d99b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19477970 0x558e1949ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:37066/0 (socket says 172.21.15.69:37066) 2023-07-06T20:19:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b3d99b700 1 -- 172.21.15.69:0/2233536295 learned_addr learned my addr 172.21.15.69:0/2233536295 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b3d19a700 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19478eb0 0x558e1949bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.866+0000 7f5b3e19c700 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19486830 0x558e1949c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b3d99b700 1 -- 172.21.15.69:0/2233536295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19486830 msgr2=0x558e1949c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b3d99b700 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19486830 0x558e1949c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b3d99b700 1 -- 172.21.15.69:0/2233536295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19478eb0 msgr2=0x558e1949bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b3d99b700 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19478eb0 0x558e1949bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b3d99b700 1 -- 172.21.15.69:0/2233536295 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e191d6ce0 con 0x558e19477970 2023-07-06T20:19:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b3e19c700 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19486830 0x558e1949c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:19:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b3d99b700 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19477970 0x558e1949ba00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5b34005920 tx=0x7f5b3400f4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d08a7ce526051a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b217fa700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:19:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b217fa700 1 -- 172.21.15.69:0/2233536295 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b340325a0 con 0x558e19477970 2023-07-06T20:19:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b5357db80 1 -- 172.21.15.69:0/2233536295 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b3000f040 con 0x558e19477970 2023-07-06T20:19:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b217fa700 1 -- 172.21.15.69:0/2233536295 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f5b3400f770 con 0x558e19477970 2023-07-06T20:19:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b217fa700 1 -- 172.21.15.69:0/2233536295 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b3400f930 con 0x558e19477970 2023-07-06T20:19:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b5357db80 10 client.25427 fetch_fsmap learned FSMap version 27 2023-07-06T20:19:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.867+0000 7f5b5357db80 1 -- 172.21.15.69:0/2233536295 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f5b38108870 con 0x558e19477970 2023-07-06T20:19:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.868+0000 7f5b217fa700 1 -- 172.21.15.69:0/2233536295 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b3000f040 con 0x558e19477970 2023-07-06T20:19:47.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.868+0000 7f5b217fa700 1 -- 172.21.15.69:0/2233536295 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5b3403b030 con 0x558e19477970 2023-07-06T20:19:47.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.868+0000 7f5b5357db80 10 client.25427 fetch_fsmap finished waiting for FSMap version 27 2023-07-06T20:19:47.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.868+0000 7f5b5357db80 10 client.25427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:19:47.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.868+0000 7f5b5357db80 20 client.25427 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:47.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.868+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558e194e31d0 0x558e194e55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:47.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.868+0000 7f5b5357db80 4 client.25427 mds_command: new command op to 24584 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-06T20:19:47.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.868+0000 7f5b5357db80 1 -- 172.21.15.69:0/2233536295 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558e193b8f30 con 0x558e194e31d0 2023-07-06T20:19:47.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.868+0000 7f5b3d19a700 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558e194e31d0 0x558e194e55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:47.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.869+0000 7f5b217fa700 10 client.25427 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:47.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.869+0000 7f5b3d19a700 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558e194e31d0 0x558e194e55e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af39315beacd1985 server_cookie=871f32ec0615d25d in_seq=0 out_seq=0 2023-07-06T20:19:47.878 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b217fa700 1 -- 172.21.15.69:0/2233536295 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558e193b8f30 con 0x558e194e31d0 2023-07-06T20:19:47.878 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b217fa700 10 client.25427 handle_command_reply: tid=0 2023-07-06T20:19:47.878 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b5357db80 1 client.25427 shutdown 2023-07-06T20:19:47.878 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b5357db80 1 -- 172.21.15.69:0/2233536295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19477970 msgr2=0x558e1949ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:47.878 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19477970 0x558e1949ba00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5b34005920 tx=0x7f5b3400f4e0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.878 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b5357db80 1 -- 172.21.15.69:0/2233536295 shutdown_connections 2023-07-06T20:19:47.878 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558e194e31d0 0x558e194e55e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e19486830 0x558e1949c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e19477970 0x558e1949ba00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b5357db80 1 --2- 172.21.15.69:0/2233536295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e19478eb0 0x558e1949bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b217fa700 0 client.25427 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:47.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b5357db80 1 -- 172.21.15.69:0/2233536295 >> 172.21.15.69:0/2233536295 conn(0x558e19464820 msgr2=0x558e1947bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:47.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b5357db80 1 -- 172.21.15.69:0/2233536295 shutdown_connections 2023-07-06T20:19:47.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.870+0000 7f5b5357db80 1 -- 172.21.15.69:0/2233536295 wait complete. 2023-07-06T20:19:47.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.871+0000 7f5b5357db80 20 client.25427 trim_cache size 0 max 16384 2023-07-06T20:19:47.880 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:19:47.880 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2023-07-06T20:19:47.880 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:19:47.880 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2023-07-06T20:19:47.880 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:19:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.871+0000 7f5b3e99d700 1 -- 172.21.15.69:0/234890504 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5b240a11f0 msgr2=0x7f5b240a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.871+0000 7f5b3e99d700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5b240a11f0 0x7f5b240a36a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400b2e0 tx=0x7f5b34005fb0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.871+0000 7f5b3e99d700 1 -- 172.21.15.69:0/234890504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b38100db0 msgr2=0x7f5b38193a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.871+0000 7f5b3e99d700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b38100db0 0x7f5b38193a50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5b28012fc0 tx=0x7f5b280100c0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.871+0000 7f5b3e99d700 1 -- 172.21.15.69:0/234890504 shutdown_connections 2023-07-06T20:19:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.871+0000 7f5b3e99d700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5b240a11f0 0x7f5b240a36a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.871+0000 7f5b3e99d700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b38101760 0x7f5b3819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.871+0000 7f5b3e99d700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b380ffb20 0x7f5b38193510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.871+0000 7f5b3e99d700 1 --2- 172.21.15.69:0/234890504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b38100db0 0x7f5b38193a50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.871+0000 7f5b3e99d700 1 -- 172.21.15.69:0/234890504 >> 172.21.15.69:0/234890504 conn(0x7f5b380fb140 msgr2=0x7f5b381041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.872+0000 7f5b3e99d700 1 -- 172.21.15.69:0/234890504 shutdown_connections 2023-07-06T20:19:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:47.872+0000 7f5b3e99d700 1 -- 172.21.15.69:0/234890504 wait complete. 2023-07-06T20:19:47.893 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:19:47.900 INFO:teuthology.orchestra.run.smithi069.stderr:9900000 of 50000000 tuples (19%) done (elapsed 24.71 s, remaining 100.09 s) 2023-07-06T20:19:48.140 INFO:teuthology.orchestra.run.smithi069.stderr:10000000 of 50000000 tuples (20%) done (elapsed 24.95 s, remaining 99.81 s) 2023-07-06T20:19:48.152 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.150+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3762091538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf00ffc30 msgr2=0x7f5bf0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:48.152 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.150+0000 7f5bf59ff700 1 --2- 172.21.15.69:0/3762091538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf00ffc30 0x7f5bf0100040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009990 tx=0x7f5be0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.152 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.150+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3762091538 shutdown_connections 2023-07-06T20:19:48.152 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.150+0000 7f5bf59ff700 1 --2- 172.21.15.69:0/3762091538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bf01017b0 0x7f5bf0108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.152 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.150+0000 7f5bf59ff700 1 --2- 172.21.15.69:0/3762091538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf00ffc30 0x7f5bf0100040 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.150+0000 7f5bf59ff700 1 --2- 172.21.15.69:0/3762091538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bf0100e20 0x7f5bf0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.150+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3762091538 >> 172.21.15.69:0/3762091538 conn(0x7f5bf00fb250 msgr2=0x7f5bf00fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:48.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.150+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3762091538 shutdown_connections 2023-07-06T20:19:48.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.150+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3762091538 wait complete. 2023-07-06T20:19:48.154 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.151+0000 7f5bf59ff700 1 Processor -- start 2023-07-06T20:19:48.154 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.151+0000 7f5bf59ff700 1 -- start start 2023-07-06T20:19:48.155 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.151+0000 7f5bf59ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bf00ffc30 0x7f5bf0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.155 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.151+0000 7f5bf59ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf0100e20 0x7f5bf01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.155 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.151+0000 7f5bf59ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bf01017b0 0x7f5bf019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.155 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5bf59ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf019f700 con 0x7f5bf00ffc30 2023-07-06T20:19:48.155 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5bf59ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf019f840 con 0x7f5bf01017b0 2023-07-06T20:19:48.155 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5bf59ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf019f9d0 con 0x7f5bf0100e20 2023-07-06T20:19:48.155 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5beeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bf00ffc30 0x7f5bf0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.155 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5bef7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bf01017b0 0x7f5bf019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.156 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5beeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bf00ffc30 0x7f5bf0197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47486/0 (socket says 172.21.15.69:47486) 2023-07-06T20:19:48.156 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5beeffd700 1 -- 172.21.15.69:0/3597629541 learned_addr learned my addr 172.21.15.69:0/3597629541 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:48.156 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf0100e20 0x7f5bf01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.156 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5beeffd700 1 -- 172.21.15.69:0/3597629541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf0100e20 msgr2=0x7f5bf01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:48.156 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5beeffd700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf0100e20 0x7f5bf01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.156 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5beeffd700 1 -- 172.21.15.69:0/3597629541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bf01017b0 msgr2=0x7f5bf019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:48.156 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5beeffd700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bf01017b0 0x7f5bf019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5beeffd700 1 -- 172.21.15.69:0/3597629541 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be00095f0 con 0x7f5bf00ffc30 2023-07-06T20:19:48.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf0100e20 0x7f5bf01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:48.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bf01017b0 0x7f5bf019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:48.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.152+0000 7f5beeffd700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bf00ffc30 0x7f5bf0197b80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5be0004710 tx=0x7f5be0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:48.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.153+0000 7f5bf49fd700 1 -- 172.21.15.69:0/3597629541 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be001d070 con 0x7f5bf00ffc30 2023-07-06T20:19:48.158 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.153+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3597629541 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf019fc50 con 0x7f5bf00ffc30 2023-07-06T20:19:48.158 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.153+0000 7f5bf49fd700 1 -- 172.21.15.69:0/3597629541 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5be0021410 con 0x7f5bf00ffc30 2023-07-06T20:19:48.158 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.153+0000 7f5bf49fd700 1 -- 172.21.15.69:0/3597629541 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0017650 con 0x7f5bf00ffc30 2023-07-06T20:19:48.158 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.153+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3597629541 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf01a0120 con 0x7f5bf00ffc30 2023-07-06T20:19:48.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.158+0000 7f5c093b0b80 1 Processor -- start 2023-07-06T20:19:48.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.158+0000 7f5bf49fd700 1 -- 172.21.15.69:0/3597629541 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5be0021580 con 0x7f5bf00ffc30 2023-07-06T20:19:48.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.158+0000 7f5c093b0b80 1 -- start start 2023-07-06T20:19:48.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.158+0000 7f5c093b0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7eef20 0x55a2aa7ef330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.158+0000 7f5c093b0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7ef870 0x55a2aa7efe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.158+0000 7f5c093b0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7f0390 0x55a2aa7f0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.158+0000 7f5c093b0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aa6f3cb0 con 0x55a2aa7ef870 2023-07-06T20:19:48.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.158+0000 7f5c093b0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aa746a40 con 0x55a2aa7eef20 2023-07-06T20:19:48.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.158+0000 7f5c093b0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aa73fed0 con 0x55a2aa7f0390 2023-07-06T20:19:48.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.159+0000 7f5bef7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7eef20 0x55a2aa7ef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.159+0000 7f5bef7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7eef20 0x55a2aa7ef330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:37090/0 (socket says 172.21.15.69:37090) 2023-07-06T20:19:48.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.159+0000 7f5bef7fe700 1 -- 172.21.15.69:0/2408582159 learned_addr learned my addr 172.21.15.69:0/2408582159 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:48.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.159+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/2408582159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7f0390 0x55a2aa7f0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.159+0000 7f5beeffd700 1 --2- 172.21.15.69:0/2408582159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7ef870 0x55a2aa7efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.159+0000 7f5bef7fe700 1 -- 172.21.15.69:0/2408582159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7f0390 msgr2=0x55a2aa7f0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:48.163 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.159+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/2408582159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7f0390 0x55a2aa7f0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.163 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.159+0000 7f5bef7fe700 1 -- 172.21.15.69:0/2408582159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7ef870 msgr2=0x55a2aa7efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:48.163 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.159+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/2408582159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7ef870 0x55a2aa7efe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.163 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.159+0000 7f5bef7fe700 1 -- 172.21.15.69:0/2408582159 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2aa56dde0 con 0x55a2aa7eef20 2023-07-06T20:19:48.163 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.159+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/2408582159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7eef20 0x55a2aa7ef330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5be8003eb0 tx=0x7f5be8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d6fd1fd5ac92447 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:48.165 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.160+0000 7f5bd5ffb700 1 -- 172.21.15.69:0/2408582159 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be80092d0 con 0x55a2aa7eef20 2023-07-06T20:19:48.165 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.160+0000 7f5bd5ffb700 1 -- 172.21.15.69:0/2408582159 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5be800f040 con 0x55a2aa7eef20 2023-07-06T20:19:48.165 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.160+0000 7f5bd5ffb700 1 -- 172.21.15.69:0/2408582159 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be800e7e0 con 0x55a2aa7eef20 2023-07-06T20:19:48.165 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.160+0000 7f5bf49fd700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5bd80a1150 0x7f5bd80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.160+0000 7f5bf49fd700 1 -- 172.21.15.69:0/3597629541 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f5be00cdcd0 con 0x7f5bf00ffc30 2023-07-06T20:19:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.160+0000 7f5beeffd700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5bd80a1150 0x7f5bd80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.160+0000 7f5c093b0b80 1 -- 172.21.15.69:0/2408582159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7eef20 msgr2=0x55a2aa7ef330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.160+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/2408582159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7eef20 0x55a2aa7ef330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5be8003eb0 tx=0x7f5be8009920 comp rx=0 tx=0).stop 2023-07-06T20:19:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.161+0000 7f5beeffd700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5bd80a1150 0x7f5bd80a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5be009ae40 tx=0x7f5be0009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.161+0000 7f5c093b0b80 1 -- 172.21.15.69:0/2408582159 shutdown_connections 2023-07-06T20:19:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.161+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/2408582159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7f0390 0x55a2aa7f0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.161+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/2408582159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7eef20 0x55a2aa7ef330 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.161+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/2408582159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7ef870 0x55a2aa7efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.161+0000 7f5c093b0b80 1 -- 172.21.15.69:0/2408582159 >> 172.21.15.69:0/2408582159 conn(0x55a2aa7e4640 msgr2=0x55a2aa7e99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:48.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.161+0000 7f5c093b0b80 1 -- 172.21.15.69:0/2408582159 shutdown_connections 2023-07-06T20:19:48.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.161+0000 7f5c093b0b80 1 -- 172.21.15.69:0/2408582159 wait complete. 2023-07-06T20:19:48.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.161+0000 7f5c093b0b80 1 -- start start 2023-07-06T20:19:48.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.162+0000 7f5c093b0b80 1 Processor -- start 2023-07-06T20:19:48.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.162+0000 7f5c093b0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7eef20 0x55a2aa7f8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.162+0000 7f5c093b0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7ef870 0x55a2aa7f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.162+0000 7f5c093b0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7f0390 0x55a2aa7ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.162+0000 7f5c093b0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aa6dfac0 con 0x55a2aa7eef20 2023-07-06T20:19:48.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.162+0000 7f5c093b0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aa6df7c0 con 0x55a2aa7f0390 2023-07-06T20:19:48.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.162+0000 7f5c093b0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aa5c3920 con 0x55a2aa7ef870 2023-07-06T20:19:48.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.162+0000 7f5c093b0b80 10 client.0 _ll_register_callbacks cb 0x55a2aa6dd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:48.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.162+0000 7f5bef7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7eef20 0x55a2aa7f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.162+0000 7f5bef7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7eef20 0x55a2aa7f8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47512/0 (socket says 172.21.15.69:47512) 2023-07-06T20:19:48.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.162+0000 7f5bef7fe700 1 -- 172.21.15.69:0/1660419564 learned_addr learned my addr 172.21.15.69:0/1660419564 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:48.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.163+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7ef870 0x55a2aa7f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.163+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7f0390 0x55a2aa7ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.163+0000 7f5bee7fc700 1 -- 172.21.15.69:0/1660419564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7f0390 msgr2=0x55a2aa7ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:48.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.163+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7f0390 0x55a2aa7ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.163+0000 7f5bee7fc700 1 -- 172.21.15.69:0/1660419564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7eef20 msgr2=0x55a2aa7f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:48.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.163+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7eef20 0x55a2aa7f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.163+0000 7f5bee7fc700 1 -- 172.21.15.69:0/1660419564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be8003b90 con 0x55a2aa7ef870 2023-07-06T20:19:48.170 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.163+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7eef20 0x55a2aa7f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:19:48.171 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.164+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7ef870 0x55a2aa7f8da0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5be4011d10 tx=0x7f5be400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44aa6d3a8f29cf71 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:48.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.164+0000 7f5bceffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:19:48.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.164+0000 7f5bceffd700 1 -- 172.21.15.69:0/1660419564 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4016070 con 0x55a2aa7ef870 2023-07-06T20:19:48.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.164+0000 7f5c093b0b80 1 -- 172.21.15.69:0/1660419564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2aa6994f0 con 0x55a2aa7ef870 2023-07-06T20:19:48.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.164+0000 7f5bceffd700 1 -- 172.21.15.69:0/1660419564 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f5be400fd90 con 0x55a2aa7ef870 2023-07-06T20:19:48.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.164+0000 7f5bceffd700 1 -- 172.21.15.69:0/1660419564 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5be4004050 con 0x55a2aa7ef870 2023-07-06T20:19:48.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.164+0000 7f5c093b0b80 10 client.25390 fetch_fsmap learned FSMap version 27 2023-07-06T20:19:48.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.164+0000 7f5c093b0b80 1 -- 172.21.15.69:0/1660419564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a2aa556c60 con 0x55a2aa7ef870 2023-07-06T20:19:48.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.165+0000 7f5bceffd700 1 -- 172.21.15.69:0/1660419564 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a2aa6994f0 con 0x55a2aa7ef870 2023-07-06T20:19:48.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.165+0000 7f5bceffd700 1 -- 172.21.15.69:0/1660419564 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5be401a410 con 0x55a2aa7ef870 2023-07-06T20:19:48.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.165+0000 7f5c093b0b80 10 client.25390 fetch_fsmap finished waiting for FSMap version 27 2023-07-06T20:19:48.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.165+0000 7f5c093b0b80 10 client.25390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:19:48.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.165+0000 7f5c093b0b80 20 client.25390 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:48.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.165+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a2aa8046e0 0x55a2aa806af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.165+0000 7f5c093b0b80 4 client.25390 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:19:48.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.165+0000 7f5c093b0b80 1 -- 172.21.15.69:0/1660419564 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2aa7bdd60 con 0x55a2aa8046e0 2023-07-06T20:19:48.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.165+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a2aa8046e0 0x55a2aa806af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.166+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a2aa8046e0 0x55a2aa806af0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f19b807c6c0681 server_cookie=9ef8e244cfbd258e in_seq=0 out_seq=0 2023-07-06T20:19:48.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.166+0000 7f5bceffd700 10 client.25390 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:48.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5bceffd700 1 -- 172.21.15.69:0/1660419564 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a2aa7bdd60 con 0x55a2aa8046e0 2023-07-06T20:19:48.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5bceffd700 10 client.25390 handle_command_reply: tid=0 2023-07-06T20:19:48.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5c093b0b80 1 client.25390 shutdown 2023-07-06T20:19:48.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5c093b0b80 1 -- 172.21.15.69:0/1660419564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7ef870 msgr2=0x55a2aa7f8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:48.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7ef870 0x55a2aa7f8da0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5be4011d10 tx=0x7f5be400f6a0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5c093b0b80 1 -- 172.21.15.69:0/1660419564 shutdown_connections 2023-07-06T20:19:48.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a2aa8046e0 0x55a2aa806af0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7f0390 0x55a2aa7ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5bceffd700 0 client.25390 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:48.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7eef20 0x55a2aa7f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/1660419564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7ef870 0x55a2aa7f8da0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5c093b0b80 1 -- 172.21.15.69:0/1660419564 >> 172.21.15.69:0/1660419564 conn(0x55a2aa7e4640 msgr2=0x55a2aa7e90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:48.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5c093b0b80 1 -- 172.21.15.69:0/1660419564 shutdown_connections 2023-07-06T20:19:48.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.167+0000 7f5c093b0b80 1 -- 172.21.15.69:0/1660419564 wait complete. 2023-07-06T20:19:48.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.168+0000 7f5c093b0b80 20 client.25390 trim_cache size 0 max 16384 2023-07-06T20:19:48.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.203+0000 7f5c093b0b80 1 Processor -- start 2023-07-06T20:19:48.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.203+0000 7f5c093b0b80 1 -- start start 2023-07-06T20:19:48.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.203+0000 7f5c093b0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7eef20 0x55a2aa7f6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.203+0000 7f5c093b0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 0x55a2aa7f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.203+0000 7f5c093b0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7f0390 0x55a2aa7ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.203+0000 7f5c093b0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aa801ab0 con 0x55a2aa7eef20 2023-07-06T20:19:48.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.203+0000 7f5c093b0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aa7ea850 con 0x55a2aa7ef870 2023-07-06T20:19:48.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.203+0000 7f5c093b0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aa7f7d60 con 0x55a2aa7f0390 2023-07-06T20:19:48.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.203+0000 7f5bee7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 0x55a2aa7f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.203+0000 7f5bee7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 0x55a2aa7f6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:37096/0 (socket says 172.21.15.69:37096) 2023-07-06T20:19:48.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.203+0000 7f5bee7fc700 1 -- 172.21.15.69:0/2794041074 learned_addr learned my addr 172.21.15.69:0/2794041074 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:48.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.203+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/2794041074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7eef20 0x55a2aa7f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.204+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/2794041074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7f0390 0x55a2aa7ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.204+0000 7f5bee7fc700 1 -- 172.21.15.69:0/2794041074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7f0390 msgr2=0x55a2aa7ffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:48.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.204+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/2794041074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7f0390 0x55a2aa7ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.204+0000 7f5bee7fc700 1 -- 172.21.15.69:0/2794041074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7eef20 msgr2=0x55a2aa7f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:48.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.204+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/2794041074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7eef20 0x55a2aa7f6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.204+0000 7f5bee7fc700 1 -- 172.21.15.69:0/2794041074 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2aa594250 con 0x55a2aa7ef870 2023-07-06T20:19:48.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.204+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/2794041074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 0x55a2aa7f6c90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5be401a740 tx=0x7f5be400f9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=429ed181871f1ede server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:48.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.204+0000 7f5bd67fc700 1 -- 172.21.15.69:0/2794041074 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4016040 con 0x55a2aa7ef870 2023-07-06T20:19:48.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.204+0000 7f5bd67fc700 1 -- 172.21.15.69:0/2794041074 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5be40050e0 con 0x55a2aa7ef870 2023-07-06T20:19:48.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.204+0000 7f5bd67fc700 1 -- 172.21.15.69:0/2794041074 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be401aea0 con 0x55a2aa7ef870 2023-07-06T20:19:48.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.205+0000 7f5c093b0b80 1 -- 172.21.15.69:0/2794041074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 msgr2=0x55a2aa7f6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:48.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.205+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/2794041074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 0x55a2aa7f6c90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5be401a740 tx=0x7f5be400f9f0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.205+0000 7f5c093b0b80 1 -- 172.21.15.69:0/2794041074 shutdown_connections 2023-07-06T20:19:48.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.205+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/2794041074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7f0390 0x55a2aa7ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.205+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/2794041074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7eef20 0x55a2aa7f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.205+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/2794041074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 0x55a2aa7f6c90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.205+0000 7f5c093b0b80 1 -- 172.21.15.69:0/2794041074 >> 172.21.15.69:0/2794041074 conn(0x55a2aa7e4640 msgr2=0x55a2aa7f75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:48.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.205+0000 7f5c093b0b80 1 -- 172.21.15.69:0/2794041074 shutdown_connections 2023-07-06T20:19:48.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.205+0000 7f5c093b0b80 1 -- 172.21.15.69:0/2794041074 wait complete. 2023-07-06T20:19:48.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.206+0000 7f5c093b0b80 1 -- start start 2023-07-06T20:19:48.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.206+0000 7f5c093b0b80 1 Processor -- start 2023-07-06T20:19:48.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.206+0000 7f5c093b0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7eef20 0x55a2aa7fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.206+0000 7f5c093b0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 0x55a2aa7fc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.206+0000 7f5c093b0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7f0390 0x55a2aa7ff190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.206+0000 7f5c093b0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aa7ff6d0 con 0x55a2aa7f0390 2023-07-06T20:19:48.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.206+0000 7f5c093b0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aa7ff840 con 0x55a2aa7ef870 2023-07-06T20:19:48.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.206+0000 7f5c093b0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aa7ffa00 con 0x55a2aa7eef20 2023-07-06T20:19:48.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.206+0000 7f5c093b0b80 10 client.0 _ll_register_callbacks cb 0x7f5be40117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:19:48.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.207+0000 7f5bee7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 0x55a2aa7fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.207+0000 7f5bee7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 0x55a2aa7fc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:37106/0 (socket says 172.21.15.69:37106) 2023-07-06T20:19:48.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.207+0000 7f5bee7fc700 1 -- 172.21.15.69:0/885396219 learned_addr learned my addr 172.21.15.69:0/885396219 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:19:48.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.207+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7eef20 0x55a2aa7fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.207+0000 7f5bee7fc700 1 -- 172.21.15.69:0/885396219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7eef20 msgr2=0x55a2aa7fbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:48.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.207+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7eef20 0x55a2aa7fbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.207+0000 7f5bee7fc700 1 -- 172.21.15.69:0/885396219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7f0390 msgr2=0x55a2aa7ff190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:19:48.215 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.207+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7f0390 0x55a2aa7ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.215 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.207+0000 7f5bee7fc700 1 -- 172.21.15.69:0/885396219 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2aa556c60 con 0x55a2aa7ef870 2023-07-06T20:19:48.215 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.207+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7eef20 0x55a2aa7fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:19:48.215 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.207+0000 7f5bee7fc700 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 0x55a2aa7fc170 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5be402f040 tx=0x7f5be401bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2428f8e06b333923 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:19:48.215 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.207+0000 7f5bceffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:19:48.215 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5bceffd700 1 -- 172.21.15.69:0/885396219 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4016040 con 0x55a2aa7ef870 2023-07-06T20:19:48.215 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5c093b0b80 1 -- 172.21.15.69:0/885396219 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5be800f040 con 0x55a2aa7ef870 2023-07-06T20:19:48.215 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5bceffd700 1 -- 172.21.15.69:0/885396219 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f5be4000e40 con 0x55a2aa7ef870 2023-07-06T20:19:48.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5bceffd700 1 -- 172.21.15.69:0/885396219 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5be401a770 con 0x55a2aa7ef870 2023-07-06T20:19:48.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5c093b0b80 10 client.25445 fetch_fsmap learned FSMap version 27 2023-07-06T20:19:48.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5c093b0b80 1 -- 172.21.15.69:0/885396219 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a2aa56c2b0 con 0x55a2aa7ef870 2023-07-06T20:19:48.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5bceffd700 1 -- 172.21.15.69:0/885396219 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5be4045080 con 0x55a2aa7ef870 2023-07-06T20:19:48.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5bceffd700 1 -- 172.21.15.69:0/885396219 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5be402b030 con 0x55a2aa7ef870 2023-07-06T20:19:48.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5c093b0b80 10 client.25445 fetch_fsmap finished waiting for FSMap version 27 2023-07-06T20:19:48.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5c093b0b80 10 client.25445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:19:48.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5c093b0b80 20 client.25445 populate_metadata read hostname 'smithi069' 2023-07-06T20:19:48.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a2aa859350 0x55a2aa85b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:19:48.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5c093b0b80 4 client.25445 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:19:48.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.208+0000 7f5c093b0b80 1 -- 172.21.15.69:0/885396219 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a2aa738eb0 con 0x55a2aa859350 2023-07-06T20:19:48.218 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a2aa859350 0x55a2aa85b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:19:48.218 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5bceffd700 10 client.25445 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:48.218 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5bef7fe700 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a2aa859350 0x55a2aa85b760 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf3c444dbe7d35c server_cookie=62f2cda8596e184d in_seq=0 out_seq=0 2023-07-06T20:19:48.219 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5bceffd700 1 -- 172.21.15.69:0/885396219 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a2aa738eb0 con 0x55a2aa859350 2023-07-06T20:19:48.219 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5bceffd700 10 client.25445 handle_command_reply: tid=0 2023-07-06T20:19:48.219 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5c093b0b80 1 client.25445 shutdown 2023-07-06T20:19:48.219 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5c093b0b80 1 -- 172.21.15.69:0/885396219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 msgr2=0x55a2aa7fc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:48.219 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 0x55a2aa7fc170 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5be402f040 tx=0x7f5be401bd50 comp rx=0 tx=0).stop 2023-07-06T20:19:48.219 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5c093b0b80 1 -- 172.21.15.69:0/885396219 shutdown_connections 2023-07-06T20:19:48.219 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a2aa859350 0x55a2aa85b760 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a2aa7f0390 0x55a2aa7ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2aa7eef20 0x55a2aa7fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5c093b0b80 1 --2- 172.21.15.69:0/885396219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a2aa7ef870 0x55a2aa7fc170 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5bceffd700 0 client.25445 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:19:48.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.209+0000 7f5c093b0b80 1 -- 172.21.15.69:0/885396219 >> 172.21.15.69:0/885396219 conn(0x55a2aa7e4640 msgr2=0x55a2aa7f6750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:48.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.210+0000 7f5c093b0b80 1 -- 172.21.15.69:0/885396219 shutdown_connections 2023-07-06T20:19:48.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.210+0000 7f5c093b0b80 1 -- 172.21.15.69:0/885396219 wait complete. 2023-07-06T20:19:48.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.210+0000 7f5c093b0b80 20 client.25445 trim_cache size 0 max 16384 2023-07-06T20:19:48.221 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:19:48.221 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:19:48.221 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:19:48.221 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:19:48.221 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:19:48.221 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:19:48.222 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:19:48.222 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:19:48.222 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:19:48.222 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:19:48.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.210+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3597629541 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5bd80a1150 msgr2=0x7f5bd80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:48.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.210+0000 7f5bf59ff700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5bd80a1150 0x7f5bd80a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5be009ae40 tx=0x7f5be0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.210+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3597629541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bf00ffc30 msgr2=0x7f5bf0197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:19:48.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.210+0000 7f5bf59ff700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bf00ffc30 0x7f5bf0197b80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5be0004710 tx=0x7f5be0009580 comp rx=0 tx=0).stop 2023-07-06T20:19:48.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.211+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3597629541 shutdown_connections 2023-07-06T20:19:48.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.211+0000 7f5bf59ff700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f5bd80a1150 0x7f5bd80a3600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.211+0000 7f5bf59ff700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bf01017b0 0x7f5bf019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.211+0000 7f5bf59ff700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5bf00ffc30 0x7f5bf0197b80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.211+0000 7f5bf59ff700 1 --2- 172.21.15.69:0/3597629541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bf0100e20 0x7f5bf01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:19:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.211+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3597629541 >> 172.21.15.69:0/3597629541 conn(0x7f5bf00fb250 msgr2=0x7f5bf00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:19:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.211+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3597629541 shutdown_connections 2023-07-06T20:19:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:19:48.211+0000 7f5bf59ff700 1 -- 172.21.15.69:0/3597629541 wait complete. 2023-07-06T20:19:48.231 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:19:48.338 INFO:teuthology.orchestra.run.smithi069.stderr:10100000 of 50000000 tuples (20%) done (elapsed 25.15 s, remaining 99.35 s) 2023-07-06T20:19:48.457 INFO:teuthology.orchestra.run.smithi069.stderr:10200000 of 50000000 tuples (20%) done (elapsed 25.27 s, remaining 98.60 s) 2023-07-06T20:19:48.638 INFO:teuthology.orchestra.run.smithi069.stderr:10300000 of 50000000 tuples (20%) done (elapsed 25.45 s, remaining 98.09 s) 2023-07-06T20:19:48.818 INFO:teuthology.orchestra.run.smithi069.stderr:10400000 of 50000000 tuples (20%) done (elapsed 25.63 s, remaining 97.59 s) 2023-07-06T20:19:48.947 INFO:teuthology.orchestra.run.smithi069.stderr:10500000 of 50000000 tuples (21%) done (elapsed 25.76 s, remaining 96.90 s) 2023-07-06T20:19:49.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:48 smithi069 ceph-mon[126399]: pgmap v342: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 157 MiB/s wr, 82 op/s 2023-07-06T20:19:49.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:48 smithi069 ceph-mon[126399]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:19:49.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:48 smithi069 ceph-mon[126399]: scrub queued for path: / 2023-07-06T20:19:49.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:48 smithi069 ceph-mon[126399]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:49.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:48 smithi069 ceph-mon[126399]: scrub summary: active paths [/] 2023-07-06T20:19:49.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:48 smithi069 ceph-mon[126399]: scrub summary: active paths [/] 2023-07-06T20:19:49.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:48 smithi069 ceph-mon[126399]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:49.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:48 smithi069 ceph-mon[126399]: scrub summary: active paths [/] 2023-07-06T20:19:49.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:48 smithi069 ceph-mon[126399]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:49.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:48 smithi156 ceph-mon[129280]: pgmap v342: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 157 MiB/s wr, 82 op/s 2023-07-06T20:19:49.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:48 smithi156 ceph-mon[129280]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:19:49.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:48 smithi156 ceph-mon[129280]: scrub queued for path: / 2023-07-06T20:19:49.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:48 smithi156 ceph-mon[129280]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:49.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:48 smithi156 ceph-mon[129280]: scrub summary: active paths [/] 2023-07-06T20:19:49.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:48 smithi156 ceph-mon[129280]: scrub summary: active paths [/] 2023-07-06T20:19:49.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:48 smithi156 ceph-mon[129280]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:49.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:48 smithi156 ceph-mon[129280]: scrub summary: active paths [/] 2023-07-06T20:19:49.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:48 smithi156 ceph-mon[129280]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:49.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:48 smithi098 ceph-mon[128968]: pgmap v342: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 157 MiB/s wr, 82 op/s 2023-07-06T20:19:49.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:48 smithi098 ceph-mon[128968]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:19:49.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:48 smithi098 ceph-mon[128968]: scrub queued for path: / 2023-07-06T20:19:49.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:48 smithi098 ceph-mon[128968]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:49.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:48 smithi098 ceph-mon[128968]: scrub summary: active paths [/] 2023-07-06T20:19:49.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:48 smithi098 ceph-mon[128968]: scrub summary: active paths [/] 2023-07-06T20:19:49.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:48 smithi098 ceph-mon[128968]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:49.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:48 smithi098 ceph-mon[128968]: scrub summary: active paths [/] 2023-07-06T20:19:49.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:48 smithi098 ceph-mon[128968]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:49.132 INFO:teuthology.orchestra.run.smithi069.stderr:10600000 of 50000000 tuples (21%) done (elapsed 25.94 s, remaining 96.43 s) 2023-07-06T20:19:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:10700000 of 50000000 tuples (21%) done (elapsed 26.14 s, remaining 96.02 s) 2023-07-06T20:19:49.525 INFO:teuthology.orchestra.run.smithi069.stderr:10800000 of 50000000 tuples (21%) done (elapsed 26.34 s, remaining 95.59 s) 2023-07-06T20:19:49.712 INFO:teuthology.orchestra.run.smithi069.stderr:10900000 of 50000000 tuples (21%) done (elapsed 26.52 s, remaining 95.14 s) 2023-07-06T20:19:49.826 INFO:teuthology.orchestra.run.smithi069.stderr:11000000 of 50000000 tuples (22%) done (elapsed 26.64 s, remaining 94.44 s) 2023-07-06T20:19:50.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:49 smithi069 ceph-mon[126399]: scrub summary: active paths [/] 2023-07-06T20:19:50.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:49 smithi069 ceph-mon[126399]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:50.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:49 smithi069 ceph-mon[126399]: scrub summary: active paths [/] 2023-07-06T20:19:50.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:49 smithi069 ceph-mon[126399]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:50.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:49 smithi069 ceph-mon[126399]: scrub summary: active paths [/] 2023-07-06T20:19:50.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:49 smithi069 ceph-mon[126399]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:50.014 INFO:teuthology.orchestra.run.smithi069.stderr:11100000 of 50000000 tuples (22%) done (elapsed 26.83 s, remaining 94.01 s) 2023-07-06T20:19:50.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:49 smithi156 ceph-mon[129280]: scrub summary: active paths [/] 2023-07-06T20:19:50.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:49 smithi156 ceph-mon[129280]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:50.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:49 smithi156 ceph-mon[129280]: scrub summary: active paths [/] 2023-07-06T20:19:50.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:49 smithi156 ceph-mon[129280]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:50.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:49 smithi156 ceph-mon[129280]: scrub summary: active paths [/] 2023-07-06T20:19:50.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:49 smithi156 ceph-mon[129280]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:50.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:49 smithi098 ceph-mon[128968]: scrub summary: active paths [/] 2023-07-06T20:19:50.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:49 smithi098 ceph-mon[128968]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:50.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:49 smithi098 ceph-mon[128968]: scrub summary: active paths [/] 2023-07-06T20:19:50.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:49 smithi098 ceph-mon[128968]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:50.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:49 smithi098 ceph-mon[128968]: scrub summary: active paths [/] 2023-07-06T20:19:50.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:49 smithi098 ceph-mon[128968]: scrub summary: idle+waiting paths [/] 2023-07-06T20:19:50.188 INFO:teuthology.orchestra.run.smithi069.stderr:11200000 of 50000000 tuples (22%) done (elapsed 27.00 s, remaining 93.54 s) 2023-07-06T20:19:50.331 INFO:teuthology.orchestra.run.smithi069.stderr:11300000 of 50000000 tuples (22%) done (elapsed 27.14 s, remaining 92.96 s) 2023-07-06T20:19:50.512 INFO:teuthology.orchestra.run.smithi069.stderr:11400000 of 50000000 tuples (22%) done (elapsed 27.32 s, remaining 92.52 s) 2023-07-06T20:19:50.668 INFO:teuthology.orchestra.run.smithi069.stderr:11500000 of 50000000 tuples (23%) done (elapsed 27.48 s, remaining 92.00 s) 2023-07-06T20:19:50.808 INFO:teuthology.orchestra.run.smithi069.stderr:11600000 of 50000000 tuples (23%) done (elapsed 27.62 s, remaining 91.43 s) 2023-07-06T20:19:50.956 INFO:teuthology.orchestra.run.smithi069.stderr:11700000 of 50000000 tuples (23%) done (elapsed 27.77 s, remaining 90.90 s) 2023-07-06T20:19:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:50 smithi069 ceph-mon[126399]: pgmap v343: 97 pgs: 97 active+clean; 2.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 158 MiB/s wr, 73 op/s 2023-07-06T20:19:51.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:50 smithi156 ceph-mon[129280]: pgmap v343: 97 pgs: 97 active+clean; 2.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 158 MiB/s wr, 73 op/s 2023-07-06T20:19:51.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:50 smithi098 ceph-mon[128968]: pgmap v343: 97 pgs: 97 active+clean; 2.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 158 MiB/s wr, 73 op/s 2023-07-06T20:19:51.148 INFO:teuthology.orchestra.run.smithi069.stderr:11800000 of 50000000 tuples (23%) done (elapsed 27.96 s, remaining 90.52 s) 2023-07-06T20:19:51.280 INFO:teuthology.orchestra.run.smithi069.stderr:11900000 of 50000000 tuples (23%) done (elapsed 28.09 s, remaining 89.94 s) 2023-07-06T20:19:51.443 INFO:teuthology.orchestra.run.smithi069.stderr:12000000 of 50000000 tuples (24%) done (elapsed 28.25 s, remaining 89.47 s) 2023-07-06T20:19:51.645 INFO:teuthology.orchestra.run.smithi069.stderr:12100000 of 50000000 tuples (24%) done (elapsed 28.46 s, remaining 89.13 s) 2023-07-06T20:19:51.772 INFO:teuthology.orchestra.run.smithi069.stderr:12200000 of 50000000 tuples (24%) done (elapsed 28.58 s, remaining 88.56 s) 2023-07-06T20:19:51.945 INFO:teuthology.orchestra.run.smithi069.stderr:12300000 of 50000000 tuples (24%) done (elapsed 28.76 s, remaining 88.14 s) 2023-07-06T20:19:52.085 INFO:teuthology.orchestra.run.smithi069.stderr:12400000 of 50000000 tuples (24%) done (elapsed 28.90 s, remaining 87.62 s) 2023-07-06T20:19:52.275 INFO:teuthology.orchestra.run.smithi069.stderr:12500000 of 50000000 tuples (25%) done (elapsed 29.09 s, remaining 87.26 s) 2023-07-06T20:19:52.428 INFO:teuthology.orchestra.run.smithi069.stderr:12600000 of 50000000 tuples (25%) done (elapsed 29.24 s, remaining 86.79 s) 2023-07-06T20:19:52.564 INFO:teuthology.orchestra.run.smithi069.stderr:12700000 of 50000000 tuples (25%) done (elapsed 29.38 s, remaining 86.28 s) 2023-07-06T20:19:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:12800000 of 50000000 tuples (25%) done (elapsed 29.58 s, remaining 85.96 s) 2023-07-06T20:19:52.926 INFO:teuthology.orchestra.run.smithi069.stderr:12900000 of 50000000 tuples (25%) done (elapsed 29.74 s, remaining 85.52 s) 2023-07-06T20:19:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:52 smithi069 ceph-mon[126399]: pgmap v344: 97 pgs: 97 active+clean; 2.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 151 MiB/s wr, 70 op/s 2023-07-06T20:19:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:13000000 of 50000000 tuples (26%) done (elapsed 29.88 s, remaining 85.04 s) 2023-07-06T20:19:53.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:52 smithi156 ceph-mon[129280]: pgmap v344: 97 pgs: 97 active+clean; 2.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 151 MiB/s wr, 70 op/s 2023-07-06T20:19:53.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:52 smithi098 ceph-mon[128968]: pgmap v344: 97 pgs: 97 active+clean; 2.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 151 MiB/s wr, 70 op/s 2023-07-06T20:19:53.226 INFO:teuthology.orchestra.run.smithi069.stderr:13100000 of 50000000 tuples (26%) done (elapsed 30.04 s, remaining 84.61 s) 2023-07-06T20:19:53.400 INFO:teuthology.orchestra.run.smithi069.stderr:13200000 of 50000000 tuples (26%) done (elapsed 30.21 s, remaining 84.23 s) 2023-07-06T20:19:53.601 INFO:teuthology.orchestra.run.smithi069.stderr:13300000 of 50000000 tuples (26%) done (elapsed 30.41 s, remaining 83.92 s) 2023-07-06T20:19:53.762 INFO:teuthology.orchestra.run.smithi069.stderr:13400000 of 50000000 tuples (26%) done (elapsed 30.57 s, remaining 83.51 s) 2023-07-06T20:19:53.900 INFO:teuthology.orchestra.run.smithi069.stderr:13500000 of 50000000 tuples (27%) done (elapsed 30.71 s, remaining 83.04 s) 2023-07-06T20:19:54.082 INFO:teuthology.orchestra.run.smithi069.stderr:13600000 of 50000000 tuples (27%) done (elapsed 30.89 s, remaining 82.68 s) 2023-07-06T20:19:54.305 INFO:teuthology.orchestra.run.smithi069.stderr:13700000 of 50000000 tuples (27%) done (elapsed 31.12 s, remaining 82.45 s) 2023-07-06T20:19:55.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:54 smithi069 ceph-mon[126399]: pgmap v345: 97 pgs: 97 active+clean; 2.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 124 MiB/s wr, 73 op/s 2023-07-06T20:19:55.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:54 smithi156 ceph-mon[129280]: pgmap v345: 97 pgs: 97 active+clean; 2.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 124 MiB/s wr, 73 op/s 2023-07-06T20:19:55.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:54 smithi098 ceph-mon[128968]: pgmap v345: 97 pgs: 97 active+clean; 2.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 124 MiB/s wr, 73 op/s 2023-07-06T20:19:55.146 INFO:teuthology.orchestra.run.smithi069.stderr:13800000 of 50000000 tuples (27%) done (elapsed 31.96 s, remaining 83.83 s) 2023-07-06T20:19:55.328 INFO:teuthology.orchestra.run.smithi069.stderr:13900000 of 50000000 tuples (27%) done (elapsed 32.14 s, remaining 83.47 s) 2023-07-06T20:19:55.503 INFO:teuthology.orchestra.run.smithi069.stderr:14000000 of 50000000 tuples (28%) done (elapsed 32.32 s, remaining 83.10 s) 2023-07-06T20:19:55.627 INFO:teuthology.orchestra.run.smithi069.stderr:14100000 of 50000000 tuples (28%) done (elapsed 32.44 s, remaining 82.59 s) 2023-07-06T20:19:55.796 INFO:teuthology.orchestra.run.smithi069.stderr:14200000 of 50000000 tuples (28%) done (elapsed 32.61 s, remaining 82.21 s) 2023-07-06T20:19:56.002 INFO:teuthology.orchestra.run.smithi069.stderr:14300000 of 50000000 tuples (28%) done (elapsed 32.81 s, remaining 81.92 s) 2023-07-06T20:19:56.139 INFO:teuthology.orchestra.run.smithi069.stderr:14400000 of 50000000 tuples (28%) done (elapsed 32.95 s, remaining 81.46 s) 2023-07-06T20:19:56.292 INFO:teuthology.orchestra.run.smithi069.stderr:14500000 of 50000000 tuples (29%) done (elapsed 33.10 s, remaining 81.05 s) 2023-07-06T20:19:56.488 INFO:teuthology.orchestra.run.smithi069.stderr:14600000 of 50000000 tuples (29%) done (elapsed 33.30 s, remaining 80.74 s) 2023-07-06T20:19:56.621 INFO:teuthology.orchestra.run.smithi069.stderr:14700000 of 50000000 tuples (29%) done (elapsed 33.43 s, remaining 80.28 s) 2023-07-06T20:19:56.807 INFO:teuthology.orchestra.run.smithi069.stderr:14800000 of 50000000 tuples (29%) done (elapsed 33.62 s, remaining 79.96 s) 2023-07-06T20:19:56.952 INFO:teuthology.orchestra.run.smithi069.stderr:14900000 of 50000000 tuples (29%) done (elapsed 33.76 s, remaining 79.54 s) 2023-07-06T20:19:57.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:56 smithi069 ceph-mon[126399]: pgmap v346: 97 pgs: 97 active+clean; 2.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 112 MiB/s wr, 62 op/s 2023-07-06T20:19:57.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:56 smithi156 ceph-mon[129280]: pgmap v346: 97 pgs: 97 active+clean; 2.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 112 MiB/s wr, 62 op/s 2023-07-06T20:19:57.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:56 smithi098 ceph-mon[128968]: pgmap v346: 97 pgs: 97 active+clean; 2.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 112 MiB/s wr, 62 op/s 2023-07-06T20:19:57.156 INFO:teuthology.orchestra.run.smithi069.stderr:15000000 of 50000000 tuples (30%) done (elapsed 33.97 s, remaining 79.26 s) 2023-07-06T20:19:57.327 INFO:teuthology.orchestra.run.smithi069.stderr:15100000 of 50000000 tuples (30%) done (elapsed 34.14 s, remaining 78.90 s) 2023-07-06T20:19:57.511 INFO:teuthology.orchestra.run.smithi069.stderr:15200000 of 50000000 tuples (30%) done (elapsed 34.32 s, remaining 78.58 s) 2023-07-06T20:19:57.710 INFO:teuthology.orchestra.run.smithi069.stderr:15300000 of 50000000 tuples (30%) done (elapsed 34.52 s, remaining 78.29 s) 2023-07-06T20:19:57.818 INFO:teuthology.orchestra.run.smithi069.stderr:15400000 of 50000000 tuples (30%) done (elapsed 34.63 s, remaining 77.80 s) 2023-07-06T20:19:58.004 INFO:teuthology.orchestra.run.smithi069.stderr:15500000 of 50000000 tuples (31%) done (elapsed 34.82 s, remaining 77.49 s) 2023-07-06T20:19:58.166 INFO:teuthology.orchestra.run.smithi069.stderr:15600000 of 50000000 tuples (31%) done (elapsed 34.98 s, remaining 77.13 s) 2023-07-06T20:19:58.367 INFO:teuthology.orchestra.run.smithi069.stderr:15700000 of 50000000 tuples (31%) done (elapsed 35.18 s, remaining 76.85 s) 2023-07-06T20:19:58.495 INFO:teuthology.orchestra.run.smithi069.stderr:15800000 of 50000000 tuples (31%) done (elapsed 35.31 s, remaining 76.42 s) 2023-07-06T20:19:58.666 INFO:teuthology.orchestra.run.smithi069.stderr:15900000 of 50000000 tuples (31%) done (elapsed 35.48 s, remaining 76.09 s) 2023-07-06T20:19:58.880 INFO:teuthology.orchestra.run.smithi069.stderr:16000000 of 50000000 tuples (32%) done (elapsed 35.69 s, remaining 75.84 s) 2023-07-06T20:19:59.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:19:58 smithi069 ceph-mon[126399]: pgmap v347: 97 pgs: 97 active+clean; 2.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 135 MiB/s wr, 77 op/s 2023-07-06T20:19:59.035 INFO:teuthology.orchestra.run.smithi069.stderr:16100000 of 50000000 tuples (32%) done (elapsed 35.85 s, remaining 75.48 s) 2023-07-06T20:19:59.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:19:58 smithi156 ceph-mon[129280]: pgmap v347: 97 pgs: 97 active+clean; 2.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 135 MiB/s wr, 77 op/s 2023-07-06T20:19:59.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:19:58 smithi098 ceph-mon[128968]: pgmap v347: 97 pgs: 97 active+clean; 2.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 135 MiB/s wr, 77 op/s 2023-07-06T20:19:59.250 INFO:teuthology.orchestra.run.smithi069.stderr:16200000 of 50000000 tuples (32%) done (elapsed 36.06 s, remaining 75.24 s) 2023-07-06T20:19:59.372 INFO:teuthology.orchestra.run.smithi069.stderr:16300000 of 50000000 tuples (32%) done (elapsed 36.18 s, remaining 74.81 s) 2023-07-06T20:19:59.543 INFO:teuthology.orchestra.run.smithi069.stderr:16400000 of 50000000 tuples (32%) done (elapsed 36.35 s, remaining 74.48 s) 2023-07-06T20:19:59.717 INFO:teuthology.orchestra.run.smithi069.stderr:16500000 of 50000000 tuples (33%) done (elapsed 36.53 s, remaining 74.16 s) 2023-07-06T20:19:59.837 INFO:teuthology.orchestra.run.smithi069.stderr:16600000 of 50000000 tuples (33%) done (elapsed 36.65 s, remaining 73.74 s) 2023-07-06T20:19:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:16700000 of 50000000 tuples (33%) done (elapsed 36.79 s, remaining 73.36 s) 2023-07-06T20:20:00.162 INFO:teuthology.orchestra.run.smithi069.stderr:16800000 of 50000000 tuples (33%) done (elapsed 36.97 s, remaining 73.07 s) 2023-07-06T20:20:00.312 INFO:teuthology.orchestra.run.smithi069.stderr:16900000 of 50000000 tuples (33%) done (elapsed 37.12 s, remaining 72.71 s) 2023-07-06T20:20:00.468 INFO:teuthology.orchestra.run.smithi069.stderr:17000000 of 50000000 tuples (34%) done (elapsed 37.28 s, remaining 72.37 s) 2023-07-06T20:20:00.667 INFO:teuthology.orchestra.run.smithi069.stderr:17100000 of 50000000 tuples (34%) done (elapsed 37.48 s, remaining 72.11 s) 2023-07-06T20:20:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:17200000 of 50000000 tuples (34%) done (elapsed 37.68 s, remaining 71.86 s) 2023-07-06T20:20:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:00 smithi069 ceph-mon[126399]: pgmap v348: 97 pgs: 97 active+clean; 2.7 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 133 MiB/s wr, 68 op/s 2023-07-06T20:20:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:00 smithi069 ceph-mon[126399]: overall HEALTH_OK 2023-07-06T20:20:01.030 INFO:teuthology.orchestra.run.smithi069.stderr:17300000 of 50000000 tuples (34%) done (elapsed 37.84 s, remaining 71.53 s) 2023-07-06T20:20:01.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:00 smithi156 ceph-mon[129280]: pgmap v348: 97 pgs: 97 active+clean; 2.7 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 133 MiB/s wr, 68 op/s 2023-07-06T20:20:01.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:00 smithi156 ceph-mon[129280]: overall HEALTH_OK 2023-07-06T20:20:01.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:00 smithi098 ceph-mon[128968]: pgmap v348: 97 pgs: 97 active+clean; 2.7 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 133 MiB/s wr, 68 op/s 2023-07-06T20:20:01.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:00 smithi098 ceph-mon[128968]: overall HEALTH_OK 2023-07-06T20:20:01.165 INFO:teuthology.orchestra.run.smithi069.stderr:17400000 of 50000000 tuples (34%) done (elapsed 37.98 s, remaining 71.15 s) 2023-07-06T20:20:01.326 INFO:teuthology.orchestra.run.smithi069.stderr:17500000 of 50000000 tuples (35%) done (elapsed 38.14 s, remaining 70.83 s) 2023-07-06T20:20:01.793 INFO:teuthology.orchestra.run.smithi069.stderr:17600000 of 50000000 tuples (35%) done (elapsed 38.60 s, remaining 71.07 s) 2023-07-06T20:20:02.392 INFO:teuthology.orchestra.run.smithi069.stderr:17700000 of 50000000 tuples (35%) done (elapsed 39.20 s, remaining 71.54 s) 2023-07-06T20:20:02.544 INFO:teuthology.orchestra.run.smithi069.stderr:17800000 of 50000000 tuples (35%) done (elapsed 39.36 s, remaining 71.19 s) 2023-07-06T20:20:02.707 INFO:teuthology.orchestra.run.smithi069.stderr:17900000 of 50000000 tuples (35%) done (elapsed 39.52 s, remaining 70.87 s) 2023-07-06T20:20:02.912 INFO:teuthology.orchestra.run.smithi069.stderr:18000000 of 50000000 tuples (36%) done (elapsed 39.72 s, remaining 70.62 s) 2023-07-06T20:20:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:02 smithi069 ceph-mon[126399]: pgmap v349: 97 pgs: 97 active+clean; 2.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 119 MiB/s wr, 64 op/s 2023-07-06T20:20:03.085 INFO:teuthology.orchestra.run.smithi069.stderr:18100000 of 50000000 tuples (36%) done (elapsed 39.90 s, remaining 70.31 s) 2023-07-06T20:20:03.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:02 smithi156 ceph-mon[129280]: pgmap v349: 97 pgs: 97 active+clean; 2.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 119 MiB/s wr, 64 op/s 2023-07-06T20:20:03.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:02 smithi098 ceph-mon[128968]: pgmap v349: 97 pgs: 97 active+clean; 2.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 119 MiB/s wr, 64 op/s 2023-07-06T20:20:03.221 INFO:teuthology.orchestra.run.smithi069.stderr:18200000 of 50000000 tuples (36%) done (elapsed 40.03 s, remaining 69.95 s) 2023-07-06T20:20:03.410 INFO:teuthology.orchestra.run.smithi069.stderr:18300000 of 50000000 tuples (36%) done (elapsed 40.22 s, remaining 69.67 s) 2023-07-06T20:20:03.578 INFO:teuthology.orchestra.run.smithi069.stderr:18400000 of 50000000 tuples (36%) done (elapsed 40.39 s, remaining 69.37 s) 2023-07-06T20:20:03.695 INFO:teuthology.orchestra.run.smithi069.stderr:18500000 of 50000000 tuples (37%) done (elapsed 40.51 s, remaining 68.97 s) 2023-07-06T20:20:03.894 INFO:teuthology.orchestra.run.smithi069.stderr:18600000 of 50000000 tuples (37%) done (elapsed 40.71 s, remaining 68.72 s) 2023-07-06T20:20:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:18700000 of 50000000 tuples (37%) done (elapsed 40.88 s, remaining 68.42 s) 2023-07-06T20:20:04.196 INFO:teuthology.orchestra.run.smithi069.stderr:18800000 of 50000000 tuples (37%) done (elapsed 41.01 s, remaining 68.06 s) 2023-07-06T20:20:04.416 INFO:teuthology.orchestra.run.smithi069.stderr:18900000 of 50000000 tuples (37%) done (elapsed 41.23 s, remaining 67.84 s) 2023-07-06T20:20:04.577 INFO:teuthology.orchestra.run.smithi069.stderr:19000000 of 50000000 tuples (38%) done (elapsed 41.39 s, remaining 67.53 s) 2023-07-06T20:20:04.727 INFO:teuthology.orchestra.run.smithi069.stderr:19100000 of 50000000 tuples (38%) done (elapsed 41.54 s, remaining 67.20 s) 2023-07-06T20:20:04.926 INFO:teuthology.orchestra.run.smithi069.stderr:19200000 of 50000000 tuples (38%) done (elapsed 41.74 s, remaining 66.95 s) 2023-07-06T20:20:05.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:04 smithi069 ceph-mon[126399]: pgmap v350: 97 pgs: 97 active+clean; 2.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 117 MiB/s wr, 66 op/s 2023-07-06T20:20:05.086 INFO:teuthology.orchestra.run.smithi069.stderr:19300000 of 50000000 tuples (38%) done (elapsed 41.90 s, remaining 66.65 s) 2023-07-06T20:20:05.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:04 smithi156 ceph-mon[129280]: pgmap v350: 97 pgs: 97 active+clean; 2.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 117 MiB/s wr, 66 op/s 2023-07-06T20:20:05.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:04 smithi098 ceph-mon[128968]: pgmap v350: 97 pgs: 97 active+clean; 2.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 117 MiB/s wr, 66 op/s 2023-07-06T20:20:05.277 INFO:teuthology.orchestra.run.smithi069.stderr:19400000 of 50000000 tuples (38%) done (elapsed 42.09 s, remaining 66.39 s) 2023-07-06T20:20:05.384 INFO:teuthology.orchestra.run.smithi069.stderr:19500000 of 50000000 tuples (39%) done (elapsed 42.20 s, remaining 66.00 s) 2023-07-06T20:20:05.578 INFO:teuthology.orchestra.run.smithi069.stderr:19600000 of 50000000 tuples (39%) done (elapsed 42.39 s, remaining 65.75 s) 2023-07-06T20:20:05.748 INFO:teuthology.orchestra.run.smithi069.stderr:19700000 of 50000000 tuples (39%) done (elapsed 42.56 s, remaining 65.46 s) 2023-07-06T20:20:05.923 INFO:teuthology.orchestra.run.smithi069.stderr:19800000 of 50000000 tuples (39%) done (elapsed 42.73 s, remaining 65.18 s) 2023-07-06T20:20:06.037 INFO:teuthology.orchestra.run.smithi069.stderr:19900000 of 50000000 tuples (39%) done (elapsed 42.85 s, remaining 64.81 s) 2023-07-06T20:20:06.213 INFO:teuthology.orchestra.run.smithi069.stderr:20000000 of 50000000 tuples (40%) done (elapsed 43.03 s, remaining 64.54 s) 2023-07-06T20:20:06.385 INFO:teuthology.orchestra.run.smithi069.stderr:20100000 of 50000000 tuples (40%) done (elapsed 43.20 s, remaining 64.26 s) 2023-07-06T20:20:06.510 INFO:teuthology.orchestra.run.smithi069.stderr:20200000 of 50000000 tuples (40%) done (elapsed 43.32 s, remaining 63.91 s) 2023-07-06T20:20:06.703 INFO:teuthology.orchestra.run.smithi069.stderr:20300000 of 50000000 tuples (40%) done (elapsed 43.51 s, remaining 63.66 s) 2023-07-06T20:20:06.885 INFO:teuthology.orchestra.run.smithi069.stderr:20400000 of 50000000 tuples (40%) done (elapsed 43.70 s, remaining 63.40 s) 2023-07-06T20:20:07.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:06 smithi069 ceph-mon[126399]: pgmap v351: 97 pgs: 97 active+clean; 3.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 130 MiB/s wr, 60 op/s 2023-07-06T20:20:07.034 INFO:teuthology.orchestra.run.smithi069.stderr:20500000 of 50000000 tuples (41%) done (elapsed 43.85 s, remaining 63.09 s) 2023-07-06T20:20:07.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:06 smithi156 ceph-mon[129280]: pgmap v351: 97 pgs: 97 active+clean; 3.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 130 MiB/s wr, 60 op/s 2023-07-06T20:20:07.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:06 smithi098 ceph-mon[128968]: pgmap v351: 97 pgs: 97 active+clean; 3.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 130 MiB/s wr, 60 op/s 2023-07-06T20:20:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:20600000 of 50000000 tuples (41%) done (elapsed 44.00 s, remaining 62.80 s) 2023-07-06T20:20:07.383 INFO:teuthology.orchestra.run.smithi069.stderr:20700000 of 50000000 tuples (41%) done (elapsed 44.19 s, remaining 62.56 s) 2023-07-06T20:20:07.528 INFO:teuthology.orchestra.run.smithi069.stderr:20800000 of 50000000 tuples (41%) done (elapsed 44.34 s, remaining 62.25 s) 2023-07-06T20:20:07.676 INFO:teuthology.orchestra.run.smithi069.stderr:20900000 of 50000000 tuples (41%) done (elapsed 44.49 s, remaining 61.94 s) 2023-07-06T20:20:07.878 INFO:teuthology.orchestra.run.smithi069.stderr:21000000 of 50000000 tuples (42%) done (elapsed 44.69 s, remaining 61.71 s) 2023-07-06T20:20:08.034 INFO:teuthology.orchestra.run.smithi069.stderr:21100000 of 50000000 tuples (42%) done (elapsed 44.85 s, remaining 61.42 s) 2023-07-06T20:20:08.277 INFO:teuthology.orchestra.run.smithi069.stderr:21200000 of 50000000 tuples (42%) done (elapsed 45.09 s, remaining 61.25 s) 2023-07-06T20:20:08.380 INFO:teuthology.orchestra.run.smithi069.stderr:21300000 of 50000000 tuples (42%) done (elapsed 45.19 s, remaining 60.89 s) 2023-07-06T20:20:08.550 INFO:teuthology.orchestra.run.smithi069.stderr:21400000 of 50000000 tuples (42%) done (elapsed 45.36 s, remaining 60.62 s) 2023-07-06T20:20:08.807 INFO:teuthology.orchestra.run.smithi069.stderr:21500000 of 50000000 tuples (43%) done (elapsed 45.62 s, remaining 60.47 s) 2023-07-06T20:20:09.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:08 smithi156 ceph-mon[129280]: pgmap v352: 97 pgs: 97 active+clean; 3.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 136 MiB/s wr, 68 op/s 2023-07-06T20:20:09.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:08 smithi098 ceph-mon[128968]: pgmap v352: 97 pgs: 97 active+clean; 3.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 136 MiB/s wr, 68 op/s 2023-07-06T20:20:09.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:08 smithi069 ceph-mon[126399]: pgmap v352: 97 pgs: 97 active+clean; 3.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 136 MiB/s wr, 68 op/s 2023-07-06T20:20:09.574 INFO:teuthology.orchestra.run.smithi069.stderr:21600000 of 50000000 tuples (43%) done (elapsed 46.39 s, remaining 60.99 s) 2023-07-06T20:20:09.730 INFO:teuthology.orchestra.run.smithi069.stderr:21700000 of 50000000 tuples (43%) done (elapsed 46.54 s, remaining 60.70 s) 2023-07-06T20:20:09.891 INFO:teuthology.orchestra.run.smithi069.stderr:21800000 of 50000000 tuples (43%) done (elapsed 46.70 s, remaining 60.41 s) 2023-07-06T20:20:10.078 INFO:teuthology.orchestra.run.smithi069.stderr:21900000 of 50000000 tuples (43%) done (elapsed 46.89 s, remaining 60.16 s) 2023-07-06T20:20:10.237 INFO:teuthology.orchestra.run.smithi069.stderr:22000000 of 50000000 tuples (44%) done (elapsed 47.05 s, remaining 59.88 s) 2023-07-06T20:20:10.419 INFO:teuthology.orchestra.run.smithi069.stderr:22100000 of 50000000 tuples (44%) done (elapsed 47.23 s, remaining 59.63 s) 2023-07-06T20:20:10.551 INFO:teuthology.orchestra.run.smithi069.stderr:22200000 of 50000000 tuples (44%) done (elapsed 47.36 s, remaining 59.31 s) 2023-07-06T20:20:10.721 INFO:teuthology.orchestra.run.smithi069.stderr:22300000 of 50000000 tuples (44%) done (elapsed 47.53 s, remaining 59.04 s) 2023-07-06T20:20:10.855 INFO:teuthology.orchestra.run.smithi069.stderr:22400000 of 50000000 tuples (44%) done (elapsed 47.67 s, remaining 58.73 s) 2023-07-06T20:20:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:22500000 of 50000000 tuples (45%) done (elapsed 47.86 s, remaining 58.50 s) 2023-07-06T20:20:11.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:10 smithi156 ceph-mon[129280]: pgmap v353: 97 pgs: 97 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 114 MiB/s wr, 52 op/s 2023-07-06T20:20:11.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:10 smithi098 ceph-mon[128968]: pgmap v353: 97 pgs: 97 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 114 MiB/s wr, 52 op/s 2023-07-06T20:20:11.229 INFO:teuthology.orchestra.run.smithi069.stderr:22600000 of 50000000 tuples (45%) done (elapsed 48.04 s, remaining 58.24 s) 2023-07-06T20:20:11.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:10 smithi069 ceph-mon[126399]: pgmap v353: 97 pgs: 97 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 114 MiB/s wr, 52 op/s 2023-07-06T20:20:11.386 INFO:teuthology.orchestra.run.smithi069.stderr:22700000 of 50000000 tuples (45%) done (elapsed 48.20 s, remaining 57.96 s) 2023-07-06T20:20:11.573 INFO:teuthology.orchestra.run.smithi069.stderr:22800000 of 50000000 tuples (45%) done (elapsed 48.38 s, remaining 57.72 s) 2023-07-06T20:20:11.721 INFO:teuthology.orchestra.run.smithi069.stderr:22900000 of 50000000 tuples (45%) done (elapsed 48.53 s, remaining 57.43 s) 2023-07-06T20:20:11.928 INFO:teuthology.orchestra.run.smithi069.stderr:23000000 of 50000000 tuples (46%) done (elapsed 48.74 s, remaining 57.22 s) 2023-07-06T20:20:12.034 INFO:teuthology.orchestra.run.smithi069.stderr:23100000 of 50000000 tuples (46%) done (elapsed 48.85 s, remaining 56.88 s) 2023-07-06T20:20:12.245 INFO:teuthology.orchestra.run.smithi069.stderr:23200000 of 50000000 tuples (46%) done (elapsed 49.06 s, remaining 56.67 s) 2023-07-06T20:20:12.413 INFO:teuthology.orchestra.run.smithi069.stderr:23300000 of 50000000 tuples (46%) done (elapsed 49.22 s, remaining 56.41 s) 2023-07-06T20:20:12.577 INFO:teuthology.orchestra.run.smithi069.stderr:23400000 of 50000000 tuples (46%) done (elapsed 49.39 s, remaining 56.14 s) 2023-07-06T20:20:12.703 INFO:teuthology.orchestra.run.smithi069.stderr:23500000 of 50000000 tuples (47%) done (elapsed 49.51 s, remaining 55.84 s) 2023-07-06T20:20:12.873 INFO:teuthology.orchestra.run.smithi069.stderr:23600000 of 50000000 tuples (47%) done (elapsed 49.69 s, remaining 55.58 s) 2023-07-06T20:20:13.050 INFO:teuthology.orchestra.run.smithi069.stderr:23700000 of 50000000 tuples (47%) done (elapsed 49.86 s, remaining 55.33 s) 2023-07-06T20:20:13.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:12 smithi156 ceph-mon[129280]: pgmap v354: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 120 MiB/s wr, 52 op/s 2023-07-06T20:20:13.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:12 smithi098 ceph-mon[128968]: pgmap v354: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 120 MiB/s wr, 52 op/s 2023-07-06T20:20:13.173 INFO:teuthology.orchestra.run.smithi069.stderr:23800000 of 50000000 tuples (47%) done (elapsed 49.98 s, remaining 55.02 s) 2023-07-06T20:20:13.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:12 smithi069 ceph-mon[126399]: pgmap v354: 97 pgs: 97 active+clean; 3.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 120 MiB/s wr, 52 op/s 2023-07-06T20:20:13.383 INFO:teuthology.orchestra.run.smithi069.stderr:23900000 of 50000000 tuples (47%) done (elapsed 50.19 s, remaining 54.81 s) 2023-07-06T20:20:13.535 INFO:teuthology.orchestra.run.smithi069.stderr:24000000 of 50000000 tuples (48%) done (elapsed 50.35 s, remaining 54.54 s) 2023-07-06T20:20:13.673 INFO:teuthology.orchestra.run.smithi069.stderr:24100000 of 50000000 tuples (48%) done (elapsed 50.48 s, remaining 54.26 s) 2023-07-06T20:20:13.868 INFO:teuthology.orchestra.run.smithi069.stderr:24200000 of 50000000 tuples (48%) done (elapsed 50.68 s, remaining 54.03 s) 2023-07-06T20:20:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:24300000 of 50000000 tuples (48%) done (elapsed 50.84 s, remaining 53.77 s) 2023-07-06T20:20:14.168 INFO:teuthology.orchestra.run.smithi069.stderr:24400000 of 50000000 tuples (48%) done (elapsed 50.98 s, remaining 53.49 s) 2023-07-06T20:20:14.333 INFO:teuthology.orchestra.run.smithi069.stderr:24500000 of 50000000 tuples (49%) done (elapsed 51.14 s, remaining 53.23 s) 2023-07-06T20:20:14.541 INFO:teuthology.orchestra.run.smithi069.stderr:24600000 of 50000000 tuples (49%) done (elapsed 51.35 s, remaining 53.02 s) 2023-07-06T20:20:14.683 INFO:teuthology.orchestra.run.smithi069.stderr:24700000 of 50000000 tuples (49%) done (elapsed 51.49 s, remaining 52.75 s) 2023-07-06T20:20:14.849 INFO:teuthology.orchestra.run.smithi069.stderr:24800000 of 50000000 tuples (49%) done (elapsed 51.66 s, remaining 52.49 s) 2023-07-06T20:20:15.052 INFO:teuthology.orchestra.run.smithi069.stderr:24900000 of 50000000 tuples (49%) done (elapsed 51.86 s, remaining 52.28 s) 2023-07-06T20:20:15.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:14 smithi156 ceph-mon[129280]: pgmap v355: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 135 MiB/s wr, 64 op/s 2023-07-06T20:20:15.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:14 smithi098 ceph-mon[128968]: pgmap v355: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 135 MiB/s wr, 64 op/s 2023-07-06T20:20:15.208 INFO:teuthology.orchestra.run.smithi069.stderr:25000000 of 50000000 tuples (50%) done (elapsed 52.02 s, remaining 52.02 s) 2023-07-06T20:20:15.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:14 smithi069 ceph-mon[126399]: pgmap v355: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 135 MiB/s wr, 64 op/s 2023-07-06T20:20:15.362 INFO:teuthology.orchestra.run.smithi069.stderr:25100000 of 50000000 tuples (50%) done (elapsed 52.17 s, remaining 51.76 s) 2023-07-06T20:20:15.557 INFO:teuthology.orchestra.run.smithi069.stderr:25200000 of 50000000 tuples (50%) done (elapsed 52.37 s, remaining 51.54 s) 2023-07-06T20:20:15.720 INFO:teuthology.orchestra.run.smithi069.stderr:25300000 of 50000000 tuples (50%) done (elapsed 52.53 s, remaining 51.29 s) 2023-07-06T20:20:15.864 INFO:teuthology.orchestra.run.smithi069.stderr:25400000 of 50000000 tuples (50%) done (elapsed 52.68 s, remaining 51.02 s) 2023-07-06T20:20:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:25500000 of 50000000 tuples (51%) done (elapsed 53.23 s, remaining 51.14 s) 2023-07-06T20:20:16.975 INFO:teuthology.orchestra.run.smithi069.stderr:25600000 of 50000000 tuples (51%) done (elapsed 53.79 s, remaining 51.27 s) 2023-07-06T20:20:17.083 INFO:teuthology.orchestra.run.smithi069.stderr:25700000 of 50000000 tuples (51%) done (elapsed 53.90 s, remaining 50.96 s) 2023-07-06T20:20:17.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:16 smithi156 ceph-mon[129280]: pgmap v356: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 124 MiB/s wr, 58 op/s 2023-07-06T20:20:17.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:16 smithi098 ceph-mon[128968]: pgmap v356: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 124 MiB/s wr, 58 op/s 2023-07-06T20:20:17.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:16 smithi069 ceph-mon[126399]: pgmap v356: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 124 MiB/s wr, 58 op/s 2023-07-06T20:20:17.274 INFO:teuthology.orchestra.run.smithi069.stderr:25800000 of 50000000 tuples (51%) done (elapsed 54.09 s, remaining 50.73 s) 2023-07-06T20:20:17.430 INFO:teuthology.orchestra.run.smithi069.stderr:25900000 of 50000000 tuples (51%) done (elapsed 54.24 s, remaining 50.47 s) 2023-07-06T20:20:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:26000000 of 50000000 tuples (52%) done (elapsed 54.45 s, remaining 50.26 s) 2023-07-06T20:20:17.773 INFO:teuthology.orchestra.run.smithi069.stderr:26100000 of 50000000 tuples (52%) done (elapsed 54.59 s, remaining 49.98 s) 2023-07-06T20:20:17.926 INFO:teuthology.orchestra.run.smithi069.stderr:26200000 of 50000000 tuples (52%) done (elapsed 54.74 s, remaining 49.72 s) 2023-07-06T20:20:18.074 INFO:teuthology.orchestra.run.smithi069.stderr:26300000 of 50000000 tuples (52%) done (elapsed 54.89 s, remaining 49.46 s) 2023-07-06T20:20:18.232 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:20:18.308 INFO:teuthology.orchestra.run.smithi069.stderr:26400000 of 50000000 tuples (52%) done (elapsed 55.12 s, remaining 49.27 s) 2023-07-06T20:20:18.462 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.460+0000 7f17a59ff700 1 -- 172.21.15.69:0/3826073976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a0100e00 msgr2=0x7f17a0101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.460+0000 7f17a59ff700 1 --2- 172.21.15.69:0/3826073976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a0100e00 0x7f17a0101250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f17940098f0 tx=0x7f1794009c00 comp rx=0 tx=0).stop 2023-07-06T20:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.460+0000 7f17a59ff700 1 -- 172.21.15.69:0/3826073976 shutdown_connections 2023-07-06T20:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.460+0000 7f17a59ff700 1 --2- 172.21.15.69:0/3826073976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17a0101790 0x7f17a0108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.460+0000 7f17a59ff700 1 --2- 172.21.15.69:0/3826073976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17a00ffc10 0x7f17a0100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.460+0000 7f17a59ff700 1 --2- 172.21.15.69:0/3826073976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a0100e00 0x7f17a0101250 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.460+0000 7f17a59ff700 1 -- 172.21.15.69:0/3826073976 >> 172.21.15.69:0/3826073976 conn(0x7f17a00fb210 msgr2=0x7f17a00fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:20:18.463 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.460+0000 7f17a59ff700 1 -- 172.21.15.69:0/3826073976 shutdown_connections 2023-07-06T20:20:18.464 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.460+0000 7f17a59ff700 1 -- 172.21.15.69:0/3826073976 wait complete. 2023-07-06T20:20:18.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.461+0000 7f17a59ff700 1 Processor -- start 2023-07-06T20:20:18.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.461+0000 7f17a59ff700 1 -- start start 2023-07-06T20:20:18.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f17a59ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17a00ffc10 0x7f17a0197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f17a59ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a0100e00 0x7f17a0198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f17a59ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17a0101790 0x7f17a019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f17a59ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a019f760 con 0x7f17a0100e00 2023-07-06T20:20:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f17a59ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a019f8a0 con 0x7f17a0101790 2023-07-06T20:20:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f17a59ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a019fa30 con 0x7f17a00ffc10 2023-07-06T20:20:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f179e7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a0100e00 0x7f17a0198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f179f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17a0101790 0x7f17a019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f179f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17a0101790 0x7f17a019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:53842/0 (socket says 172.21.15.69:53842) 2023-07-06T20:20:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f179f7fe700 1 -- 172.21.15.69:0/141418858 learned_addr learned my addr 172.21.15.69:0/141418858 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:20:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f179effd700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17a00ffc10 0x7f17a0197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f179e7fc700 1 -- 172.21.15.69:0/141418858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17a00ffc10 msgr2=0x7f17a0197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f179e7fc700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17a00ffc10 0x7f17a0197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f179e7fc700 1 -- 172.21.15.69:0/141418858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17a0101790 msgr2=0x7f17a019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f179e7fc700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17a0101790 0x7f17a019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.462+0000 7f179e7fc700 1 -- 172.21.15.69:0/141418858 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1794009520 con 0x7f17a0100e00 2023-07-06T20:20:18.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.463+0000 7f179e7fc700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a0100e00 0x7f17a0198120 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1794009230 tx=0x7f179400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:20:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.463+0000 7f17a49fd700 1 -- 172.21.15.69:0/141418858 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179401c070 con 0x7f17a0100e00 2023-07-06T20:20:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.463+0000 7f17a49fd700 1 -- 172.21.15.69:0/141418858 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f179400f7f0 con 0x7f17a0100e00 2023-07-06T20:20:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.463+0000 7f17a49fd700 1 -- 172.21.15.69:0/141418858 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794003990 con 0x7f17a0100e00 2023-07-06T20:20:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.463+0000 7f17a59ff700 1 -- 172.21.15.69:0/141418858 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a019fcb0 con 0x7f17a0100e00 2023-07-06T20:20:18.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.463+0000 7f17a59ff700 1 -- 172.21.15.69:0/141418858 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a01a0080 con 0x7f17a0100e00 2023-07-06T20:20:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.467+0000 7f17a49fd700 1 -- 172.21.15.69:0/141418858 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1794020030 con 0x7f17a0100e00 2023-07-06T20:20:18.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.468+0000 7f17b925ab80 1 Processor -- start 2023-07-06T20:20:18.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.468+0000 7f17a49fd700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f17880a1150 0x7f17880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.468+0000 7f17b925ab80 1 -- start start 2023-07-06T20:20:18.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.468+0000 7f17a49fd700 1 -- 172.21.15.69:0/141418858 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f17940cd800 con 0x7f17a0100e00 2023-07-06T20:20:18.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.468+0000 7f179f7fe700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f17880a1150 0x7f17880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.468+0000 7f17b925ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb3a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.468+0000 7f17b925ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 0x55dc1cb3ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.468+0000 7f17b925ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3b390 0x55dc1cb3b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.468+0000 7f17b925ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1ca3ecb0 con 0x55dc1cb3b390 2023-07-06T20:20:18.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.468+0000 7f17b925ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1ca91a40 con 0x55dc1cb3a870 2023-07-06T20:20:18.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.468+0000 7f17b925ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1ca8aed0 con 0x55dc1cb39f20 2023-07-06T20:20:18.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 0x55dc1cb3ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb3a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3b390 0x55dc1cb3b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 0x55dc1cb3ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:53858/0 (socket says 172.21.15.69:53858) 2023-07-06T20:20:18.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb3a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:56342/0 (socket says 172.21.15.69:56342) 2023-07-06T20:20:18.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179e7fc700 1 -- 172.21.15.69:0/296560407 learned_addr learned my addr 172.21.15.69:0/296560407 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:20:18.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179e7fc700 1 -- 172.21.15.69:0/296560407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 msgr2=0x55dc1cb3a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:18.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179e7fc700 1 --2- 172.21.15.69:0/296560407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb3a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179e7fc700 1 -- 172.21.15.69:0/296560407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3b390 msgr2=0x55dc1cb3b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:18.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179e7fc700 1 --2- 172.21.15.69:0/296560407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3b390 0x55dc1cb3b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179e7fc700 1 -- 172.21.15.69:0/296560407 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc1c8b8de0 con 0x55dc1cb3a870 2023-07-06T20:20:18.474 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179effd700 1 --2- 172.21.15.69:0/296560407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb3a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:20:18.474 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179f7fe700 1 --2- 172.21.15.69:0/296560407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3b390 0x55dc1cb3b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:20:18.474 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179f7fe700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f17880a1150 0x7f17880a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1798009c10 tx=0x7f179800fd50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:20:18.474 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.469+0000 7f179e7fc700 1 --2- 172.21.15.69:0/296560407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 0x55dc1cb3ae00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f17940a2aa0 tx=0x7f1794009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29e214784bd400f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:20:18.474 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.470+0000 7f1785ffb700 1 -- 172.21.15.69:0/296560407 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179409eea0 con 0x55dc1cb3a870 2023-07-06T20:20:18.475 INFO:teuthology.orchestra.run.smithi069.stderr:26500000 of 50000000 tuples (53%) done (elapsed 55.28 s, remaining 49.02 s) 2023-07-06T20:20:18.478 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.470+0000 7f1785ffb700 1 -- 172.21.15.69:0/296560407 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f179409ec40 con 0x55dc1cb3a870 2023-07-06T20:20:18.478 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.470+0000 7f1785ffb700 1 -- 172.21.15.69:0/296560407 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179409ae70 con 0x55dc1cb3a870 2023-07-06T20:20:18.478 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.470+0000 7f17b925ab80 1 -- 172.21.15.69:0/296560407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 msgr2=0x55dc1cb3ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:18.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.470+0000 7f17b925ab80 1 --2- 172.21.15.69:0/296560407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 0x55dc1cb3ae00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f17940a2aa0 tx=0x7f1794009260 comp rx=0 tx=0).stop 2023-07-06T20:20:18.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.470+0000 7f17b925ab80 1 -- 172.21.15.69:0/296560407 shutdown_connections 2023-07-06T20:20:18.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.470+0000 7f17b925ab80 1 --2- 172.21.15.69:0/296560407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3b390 0x55dc1cb3b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.470+0000 7f17b925ab80 1 --2- 172.21.15.69:0/296560407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb3a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.470+0000 7f17b925ab80 1 --2- 172.21.15.69:0/296560407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 0x55dc1cb3ae00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.470+0000 7f17b925ab80 1 -- 172.21.15.69:0/296560407 >> 172.21.15.69:0/296560407 conn(0x55dc1cb2f640 msgr2=0x55dc1cb349f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:20:18.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.471+0000 7f17b925ab80 1 -- 172.21.15.69:0/296560407 shutdown_connections 2023-07-06T20:20:18.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.471+0000 7f17b925ab80 1 -- 172.21.15.69:0/296560407 wait complete. 2023-07-06T20:20:18.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.471+0000 7f17b925ab80 1 -- start start 2023-07-06T20:20:18.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.471+0000 7f17b925ab80 1 Processor -- start 2023-07-06T20:20:18.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f17b925ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb434c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f17b925ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 0x55dc1cb43a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f17b925ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3b390 0x55dc1cb4aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f17b925ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1ca2aac0 con 0x55dc1cb3b390 2023-07-06T20:20:18.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f17b925ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1ca2a7c0 con 0x55dc1cb3a870 2023-07-06T20:20:18.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f17b925ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1c90e920 con 0x55dc1cb39f20 2023-07-06T20:20:18.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f17b925ab80 10 client.0 _ll_register_callbacks cb 0x55dc1ca28760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:20:18.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f179e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 0x55dc1cb43a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f179f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3b390 0x55dc1cb4aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f179e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 0x55dc1cb43a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:53862/0 (socket says 172.21.15.69:53862) 2023-07-06T20:20:18.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f179e7fc700 1 -- 172.21.15.69:0/2417894504 learned_addr learned my addr 172.21.15.69:0/2417894504 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:20:18.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f179effd700 1 --2- 172.21.15.69:0/2417894504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb434c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f179e7fc700 1 -- 172.21.15.69:0/2417894504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 msgr2=0x55dc1cb434c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f179e7fc700 1 --2- 172.21.15.69:0/2417894504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb434c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f179e7fc700 1 -- 172.21.15.69:0/2417894504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3b390 msgr2=0x55dc1cb4aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f179e7fc700 1 --2- 172.21.15.69:0/2417894504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3b390 0x55dc1cb4aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.472+0000 7f179e7fc700 1 -- 172.21.15.69:0/2417894504 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1790009920 con 0x55dc1cb3a870 2023-07-06T20:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f179e7fc700 1 --2- 172.21.15.69:0/2417894504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 0x55dc1cb43a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f179400bfd0 tx=0x7f1794009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e10bb3080a0b07b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f177effd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f177effd700 1 -- 172.21.15.69:0/2417894504 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179409eea0 con 0x55dc1cb3a870 2023-07-06T20:20:18.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f17b925ab80 1 -- 172.21.15.69:0/2417894504 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc1c9e44f0 con 0x55dc1cb3a870 2023-07-06T20:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f177effd700 1 -- 172.21.15.69:0/2417894504 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f1794099070 con 0x55dc1cb3a870 2023-07-06T20:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f177effd700 1 -- 172.21.15.69:0/2417894504 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f179409a100 con 0x55dc1cb3a870 2023-07-06T20:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f17b925ab80 10 client.25457 fetch_fsmap learned FSMap version 27 2023-07-06T20:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f17b925ab80 1 -- 172.21.15.69:0/2417894504 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f17a01a0080 con 0x55dc1cb3a870 2023-07-06T20:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f177effd700 1 -- 172.21.15.69:0/2417894504 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc1c9e44f0 con 0x55dc1cb3a870 2023-07-06T20:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f177effd700 1 -- 172.21.15.69:0/2417894504 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f17940a2b40 con 0x55dc1cb3a870 2023-07-06T20:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f17b925ab80 10 client.25457 fetch_fsmap finished waiting for FSMap version 27 2023-07-06T20:20:18.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f17b925ab80 10 client.25457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:20:18.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f17b925ab80 20 client.25457 populate_metadata read hostname 'smithi069' 2023-07-06T20:20:18.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.473+0000 7f17b925ab80 1 --2- 172.21.15.69:0/2417894504 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55dc1cb4ee50 0x55dc1cb51260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.474+0000 7f17b925ab80 4 client.25457 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:20:18.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.474+0000 7f17b925ab80 1 -- 172.21.15.69:0/2417894504 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc1cb08d60 con 0x55dc1cb4ee50 2023-07-06T20:20:18.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.474+0000 7f179effd700 1 --2- 172.21.15.69:0/2417894504 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55dc1cb4ee50 0x55dc1cb51260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.474+0000 7f177effd700 10 client.25457 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:20:18.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.474+0000 7f179effd700 1 --2- 172.21.15.69:0/2417894504 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55dc1cb4ee50 0x55dc1cb51260 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf313f8ed7efdee server_cookie=1a47a86a43e81b7 in_seq=0 out_seq=0 2023-07-06T20:20:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.475+0000 7f177effd700 1 -- 172.21.15.69:0/2417894504 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dc1cb08d60 con 0x55dc1cb4ee50 2023-07-06T20:20:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.475+0000 7f177effd700 10 client.25457 handle_command_reply: tid=0 2023-07-06T20:20:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.475+0000 7f17b925ab80 1 client.25457 shutdown 2023-07-06T20:20:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.475+0000 7f17b925ab80 1 -- 172.21.15.69:0/2417894504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 msgr2=0x55dc1cb43a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:18.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.475+0000 7f17b925ab80 1 --2- 172.21.15.69:0/2417894504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 0x55dc1cb43a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f179400bfd0 tx=0x7f1794009400 comp rx=0 tx=0).stop 2023-07-06T20:20:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.475+0000 7f17b925ab80 1 -- 172.21.15.69:0/2417894504 shutdown_connections 2023-07-06T20:20:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.475+0000 7f17b925ab80 1 --2- 172.21.15.69:0/2417894504 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55dc1cb4ee50 0x55dc1cb51260 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.475+0000 7f17b925ab80 1 --2- 172.21.15.69:0/2417894504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3b390 0x55dc1cb4aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.476+0000 7f17b925ab80 1 --2- 172.21.15.69:0/2417894504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb434c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.476+0000 7f17b925ab80 1 --2- 172.21.15.69:0/2417894504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3a870 0x55dc1cb43a00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.476+0000 7f17b925ab80 1 -- 172.21.15.69:0/2417894504 >> 172.21.15.69:0/2417894504 conn(0x55dc1cb2f640 msgr2=0x55dc1cb340b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:20:18.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.476+0000 7f177effd700 0 client.25457 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:20:18.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.476+0000 7f17b925ab80 1 -- 172.21.15.69:0/2417894504 shutdown_connections 2023-07-06T20:20:18.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.476+0000 7f17b925ab80 1 -- 172.21.15.69:0/2417894504 wait complete. 2023-07-06T20:20:18.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.476+0000 7f17b925ab80 20 client.25457 trim_cache size 0 max 16384 2023-07-06T20:20:18.495 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.493+0000 7f17b925ab80 1 Processor -- start 2023-07-06T20:20:18.495 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.493+0000 7f17b925ab80 1 -- start start 2023-07-06T20:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.493+0000 7f17b925ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb41710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.493+0000 7f17b925ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3a870 0x55dc1cb41c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.493+0000 7f17b925ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3b390 0x55dc1cb4aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.493+0000 7f17b925ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1cb42970 con 0x55dc1cb3a870 2023-07-06T20:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.493+0000 7f17b925ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1cb43030 con 0x55dc1cb3b390 2023-07-06T20:20:18.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.493+0000 7f17b925ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1cb3ac40 con 0x55dc1cb39f20 2023-07-06T20:20:18.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3b390 0x55dc1cb4aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179e7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3a870 0x55dc1cb41c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3b390 0x55dc1cb4aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:53866/0 (socket says 172.21.15.69:53866) 2023-07-06T20:20:18.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179f7fe700 1 -- 172.21.15.69:0/2566002719 learned_addr learned my addr 172.21.15.69:0/2566002719 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:20:18.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179effd700 1 --2- 172.21.15.69:0/2566002719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb41710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179f7fe700 1 -- 172.21.15.69:0/2566002719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 msgr2=0x55dc1cb41710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179f7fe700 1 --2- 172.21.15.69:0/2566002719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb41710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179f7fe700 1 -- 172.21.15.69:0/2566002719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3a870 msgr2=0x55dc1cb41c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179f7fe700 1 --2- 172.21.15.69:0/2566002719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3a870 0x55dc1cb41c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179f7fe700 1 -- 172.21.15.69:0/2566002719 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc1cb3bc80 con 0x55dc1cb3b390 2023-07-06T20:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179effd700 1 --2- 172.21.15.69:0/2566002719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb41710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179e7fc700 1 --2- 172.21.15.69:0/2566002719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3a870 0x55dc1cb41c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.494+0000 7f179f7fe700 1 --2- 172.21.15.69:0/2566002719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3b390 0x55dc1cb4aa00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1798003c90 tx=0x7f179801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d7a7aff125960c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:20:18.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.495+0000 7f17867fc700 1 -- 172.21.15.69:0/2566002719 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179800e040 con 0x55dc1cb3b390 2023-07-06T20:20:18.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.495+0000 7f17867fc700 1 -- 172.21.15.69:0/2566002719 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f179800f040 con 0x55dc1cb3b390 2023-07-06T20:20:18.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.495+0000 7f17867fc700 1 -- 172.21.15.69:0/2566002719 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179801a6a0 con 0x55dc1cb3b390 2023-07-06T20:20:18.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.495+0000 7f17b925ab80 1 -- 172.21.15.69:0/2566002719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3b390 msgr2=0x55dc1cb4aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:18.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.495+0000 7f17b925ab80 1 --2- 172.21.15.69:0/2566002719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3b390 0x55dc1cb4aa00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1798003c90 tx=0x7f179801f000 comp rx=0 tx=0).stop 2023-07-06T20:20:18.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.495+0000 7f17b925ab80 1 -- 172.21.15.69:0/2566002719 shutdown_connections 2023-07-06T20:20:18.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.495+0000 7f17b925ab80 1 --2- 172.21.15.69:0/2566002719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb3b390 0x55dc1cb4aa00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.495+0000 7f17b925ab80 1 --2- 172.21.15.69:0/2566002719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb39f20 0x55dc1cb41710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.495+0000 7f17b925ab80 1 --2- 172.21.15.69:0/2566002719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3a870 0x55dc1cb41c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.495+0000 7f17b925ab80 1 -- 172.21.15.69:0/2566002719 >> 172.21.15.69:0/2566002719 conn(0x55dc1cb2f640 msgr2=0x55dc1cb340b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:20:18.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.496+0000 7f17b925ab80 1 -- 172.21.15.69:0/2566002719 shutdown_connections 2023-07-06T20:20:18.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.496+0000 7f17b925ab80 1 -- 172.21.15.69:0/2566002719 wait complete. 2023-07-06T20:20:18.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.496+0000 7f17b925ab80 1 -- start start 2023-07-06T20:20:18.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.496+0000 7f17b925ab80 1 Processor -- start 2023-07-06T20:20:18.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.496+0000 7f17b925ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb39f20 0x55dc1cb4a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f17b925ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3a870 0x55dc1cb472e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f17b925ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb3b390 0x55dc1cb47820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f17b925ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1cb47f00 con 0x55dc1cb3a870 2023-07-06T20:20:18.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f17b925ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1cba2cc0 con 0x55dc1cb39f20 2023-07-06T20:20:18.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f17b925ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1cba2e00 con 0x55dc1cb3b390 2023-07-06T20:20:18.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f17b925ab80 10 client.0 _ll_register_callbacks cb 0x7f1790009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:20:18.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f179effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb39f20 0x55dc1cb4a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f179effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb39f20 0x55dc1cb4a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:53870/0 (socket says 172.21.15.69:53870) 2023-07-06T20:20:18.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f179e7fc700 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3a870 0x55dc1cb472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f179effd700 1 -- 172.21.15.69:0/1952591941 learned_addr learned my addr 172.21.15.69:0/1952591941 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:20:18.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f179f7fe700 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb3b390 0x55dc1cb47820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f179effd700 1 -- 172.21.15.69:0/1952591941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb3b390 msgr2=0x55dc1cb47820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:18.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f179effd700 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb3b390 0x55dc1cb47820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f179effd700 1 -- 172.21.15.69:0/1952591941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3a870 msgr2=0x55dc1cb472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:18.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f179effd700 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3a870 0x55dc1cb472e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f179effd700 1 -- 172.21.15.69:0/1952591941 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1790004040 con 0x55dc1cb39f20 2023-07-06T20:20:18.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f179e7fc700 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3a870 0x55dc1cb472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:20:18.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f179effd700 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb39f20 0x55dc1cb4a840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1790004790 tx=0x7f1790021320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c46f2e1eacc04a4c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:20:18.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.497+0000 7f177effd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:20:18.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.498+0000 7f177effd700 1 -- 172.21.15.69:0/1952591941 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1790017070 con 0x55dc1cb39f20 2023-07-06T20:20:18.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.498+0000 7f17b925ab80 1 -- 172.21.15.69:0/1952591941 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f179409ec40 con 0x55dc1cb39f20 2023-07-06T20:20:18.506 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.498+0000 7f177effd700 1 -- 172.21.15.69:0/1952591941 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f1790021c50 con 0x55dc1cb39f20 2023-07-06T20:20:18.507 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.498+0000 7f177effd700 1 -- 172.21.15.69:0/1952591941 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17900122e0 con 0x55dc1cb39f20 2023-07-06T20:20:18.507 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.498+0000 7f17b925ab80 10 client.25463 fetch_fsmap learned FSMap version 27 2023-07-06T20:20:18.507 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.498+0000 7f17b925ab80 1 -- 172.21.15.69:0/1952591941 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f17a01a0080 con 0x55dc1cb39f20 2023-07-06T20:20:18.507 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.498+0000 7f177effd700 1 -- 172.21.15.69:0/1952591941 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f179409ec40 con 0x55dc1cb39f20 2023-07-06T20:20:18.507 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.498+0000 7f177effd700 1 -- 172.21.15.69:0/1952591941 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1790012690 con 0x55dc1cb39f20 2023-07-06T20:20:18.507 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.499+0000 7f17b925ab80 10 client.25463 fetch_fsmap finished waiting for FSMap version 27 2023-07-06T20:20:18.507 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.499+0000 7f17b925ab80 10 client.25463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:20:18.507 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.499+0000 7f17b925ab80 20 client.25463 populate_metadata read hostname 'smithi069' 2023-07-06T20:20:18.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.499+0000 7f17b925ab80 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55dc1cba48a0 0x55dc1cba6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:18.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.499+0000 7f17b925ab80 4 client.25463 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:20:18.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.499+0000 7f17b925ab80 1 -- 172.21.15.69:0/1952591941 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dc1ca83eb0 con 0x55dc1cba48a0 2023-07-06T20:20:18.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.499+0000 7f179e7fc700 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55dc1cba48a0 0x55dc1cba6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:18.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.499+0000 7f177effd700 10 client.25463 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:20:18.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.499+0000 7f179e7fc700 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55dc1cba48a0 0x55dc1cba6cb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe4bce17cc95f51 server_cookie=de7d47ac6be77212 in_seq=0 out_seq=0 2023-07-06T20:20:18.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f177effd700 1 -- 172.21.15.69:0/1952591941 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55dc1ca83eb0 con 0x55dc1cba48a0 2023-07-06T20:20:18.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f177effd700 10 client.25463 handle_command_reply: tid=0 2023-07-06T20:20:18.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f17b925ab80 1 client.25463 shutdown 2023-07-06T20:20:18.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f17b925ab80 1 -- 172.21.15.69:0/1952591941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb39f20 msgr2=0x55dc1cb4a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:18.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f17b925ab80 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb39f20 0x55dc1cb4a840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1790004790 tx=0x7f1790021320 comp rx=0 tx=0).stop 2023-07-06T20:20:18.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f17b925ab80 1 -- 172.21.15.69:0/1952591941 shutdown_connections 2023-07-06T20:20:18.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f17b925ab80 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55dc1cba48a0 0x55dc1cba6cb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f17b925ab80 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc1cb3b390 0x55dc1cb47820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f17b925ab80 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dc1cb39f20 0x55dc1cb4a840 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.510 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f177effd700 0 client.25463 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:20:18.510 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f17b925ab80 1 --2- 172.21.15.69:0/1952591941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55dc1cb3a870 0x55dc1cb472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.510 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f17b925ab80 1 -- 172.21.15.69:0/1952591941 >> 172.21.15.69:0/1952591941 conn(0x55dc1cb2f640 msgr2=0x55dc1cb41710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:20:18.510 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f17b925ab80 1 -- 172.21.15.69:0/1952591941 shutdown_connections 2023-07-06T20:20:18.510 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f17b925ab80 1 -- 172.21.15.69:0/1952591941 wait complete. 2023-07-06T20:20:18.510 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.500+0000 7f17b925ab80 20 client.25463 trim_cache size 0 max 16384 2023-07-06T20:20:18.510 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:20:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:20:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:20:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:20:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:20:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:20:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:20:18.511 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:20:18.512 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:20:18.512 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:20:18.512 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.501+0000 7f17a59ff700 1 -- 172.21.15.69:0/141418858 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f17880a1150 msgr2=0x7f17880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:18.513 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.501+0000 7f17a59ff700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f17880a1150 0x7f17880a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1798009c10 tx=0x7f179800fd50 comp rx=0 tx=0).stop 2023-07-06T20:20:18.513 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.501+0000 7f17a59ff700 1 -- 172.21.15.69:0/141418858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a0100e00 msgr2=0x7f17a0198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:18.513 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.501+0000 7f17a59ff700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a0100e00 0x7f17a0198120 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1794009230 tx=0x7f179400b730 comp rx=0 tx=0).stop 2023-07-06T20:20:18.513 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.501+0000 7f17a59ff700 1 -- 172.21.15.69:0/141418858 shutdown_connections 2023-07-06T20:20:18.513 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.501+0000 7f17a59ff700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f17880a1150 0x7f17880a3600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.513 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.501+0000 7f17a59ff700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17a0101790 0x7f17a019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.513 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.501+0000 7f17a59ff700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17a00ffc10 0x7f17a0197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.514 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.501+0000 7f17a59ff700 1 --2- 172.21.15.69:0/141418858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a0100e00 0x7f17a0198120 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:18.514 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.501+0000 7f17a59ff700 1 -- 172.21.15.69:0/141418858 >> 172.21.15.69:0/141418858 conn(0x7f17a00fb210 msgr2=0x7f17a00fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:20:18.514 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.501+0000 7f17a59ff700 1 -- 172.21.15.69:0/141418858 shutdown_connections 2023-07-06T20:20:18.514 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:18.501+0000 7f17a59ff700 1 -- 172.21.15.69:0/141418858 wait complete. 2023-07-06T20:20:18.521 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:20:18.712 INFO:teuthology.orchestra.run.smithi069.stderr:26600000 of 50000000 tuples (53%) done (elapsed 55.52 s, remaining 48.84 s) 2023-07-06T20:20:18.888 INFO:teuthology.orchestra.run.smithi069.stderr:26700000 of 50000000 tuples (53%) done (elapsed 55.70 s, remaining 48.61 s) 2023-07-06T20:20:19.005 INFO:teuthology.orchestra.run.smithi069.stderr:26800000 of 50000000 tuples (53%) done (elapsed 55.82 s, remaining 48.32 s) 2023-07-06T20:20:19.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:18 smithi156 ceph-mon[129280]: pgmap v357: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 125 MiB/s wr, 67 op/s 2023-07-06T20:20:19.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:18 smithi098 ceph-mon[128968]: pgmap v357: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 125 MiB/s wr, 67 op/s 2023-07-06T20:20:19.185 INFO:teuthology.orchestra.run.smithi069.stderr:26900000 of 50000000 tuples (53%) done (elapsed 56.00 s, remaining 48.09 s) 2023-07-06T20:20:19.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:18 smithi069 ceph-mon[126399]: pgmap v357: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 125 MiB/s wr, 67 op/s 2023-07-06T20:20:19.373 INFO:teuthology.orchestra.run.smithi069.stderr:27000000 of 50000000 tuples (54%) done (elapsed 56.18 s, remaining 47.86 s) 2023-07-06T20:20:19.571 INFO:teuthology.orchestra.run.smithi069.stderr:27100000 of 50000000 tuples (54%) done (elapsed 56.38 s, remaining 47.64 s) 2023-07-06T20:20:19.724 INFO:teuthology.orchestra.run.smithi069.stderr:27200000 of 50000000 tuples (54%) done (elapsed 56.54 s, remaining 47.39 s) 2023-07-06T20:20:19.878 INFO:teuthology.orchestra.run.smithi069.stderr:27300000 of 50000000 tuples (54%) done (elapsed 56.69 s, remaining 47.14 s) 2023-07-06T20:20:20.057 INFO:teuthology.orchestra.run.smithi069.stderr:27400000 of 50000000 tuples (54%) done (elapsed 56.87 s, remaining 46.91 s) 2023-07-06T20:20:20.235 INFO:teuthology.orchestra.run.smithi069.stderr:27500000 of 50000000 tuples (55%) done (elapsed 57.05 s, remaining 46.67 s) 2023-07-06T20:20:20.390 INFO:teuthology.orchestra.run.smithi069.stderr:27600000 of 50000000 tuples (55%) done (elapsed 57.20 s, remaining 46.42 s) 2023-07-06T20:20:20.509 INFO:teuthology.orchestra.run.smithi069.stderr:27700000 of 50000000 tuples (55%) done (elapsed 57.32 s, remaining 46.15 s) 2023-07-06T20:20:20.677 INFO:teuthology.orchestra.run.smithi069.stderr:27800000 of 50000000 tuples (55%) done (elapsed 57.49 s, remaining 45.91 s) 2023-07-06T20:20:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:27900000 of 50000000 tuples (55%) done (elapsed 57.65 s, remaining 45.67 s) 2023-07-06T20:20:20.974 INFO:teuthology.orchestra.run.smithi069.stderr:28000000 of 50000000 tuples (56%) done (elapsed 57.79 s, remaining 45.40 s) 2023-07-06T20:20:21.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:20 smithi156 ceph-mon[129280]: pgmap v358: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 127 MiB/s wr, 60 op/s 2023-07-06T20:20:21.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:20 smithi098 ceph-mon[128968]: pgmap v358: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 127 MiB/s wr, 60 op/s 2023-07-06T20:20:21.161 INFO:teuthology.orchestra.run.smithi069.stderr:28100000 of 50000000 tuples (56%) done (elapsed 57.97 s, remaining 45.18 s) 2023-07-06T20:20:21.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:20 smithi069 ceph-mon[126399]: pgmap v358: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 127 MiB/s wr, 60 op/s 2023-07-06T20:20:21.346 INFO:teuthology.orchestra.run.smithi069.stderr:28200000 of 50000000 tuples (56%) done (elapsed 58.16 s, remaining 44.96 s) 2023-07-06T20:20:21.484 INFO:teuthology.orchestra.run.smithi069.stderr:28300000 of 50000000 tuples (56%) done (elapsed 58.30 s, remaining 44.70 s) 2023-07-06T20:20:21.640 INFO:teuthology.orchestra.run.smithi069.stderr:28400000 of 50000000 tuples (56%) done (elapsed 58.45 s, remaining 44.46 s) 2023-07-06T20:20:21.825 INFO:teuthology.orchestra.run.smithi069.stderr:28500000 of 50000000 tuples (57%) done (elapsed 58.64 s, remaining 44.23 s) 2023-07-06T20:20:21.953 INFO:teuthology.orchestra.run.smithi069.stderr:28600000 of 50000000 tuples (57%) done (elapsed 58.77 s, remaining 43.97 s) 2023-07-06T20:20:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:28700000 of 50000000 tuples (57%) done (elapsed 58.94 s, remaining 43.74 s) 2023-07-06T20:20:22.277 INFO:teuthology.orchestra.run.smithi069.stderr:28800000 of 50000000 tuples (57%) done (elapsed 59.09 s, remaining 43.50 s) 2023-07-06T20:20:22.466 INFO:teuthology.orchestra.run.smithi069.stderr:28900000 of 50000000 tuples (57%) done (elapsed 59.28 s, remaining 43.28 s) 2023-07-06T20:20:22.606 INFO:teuthology.orchestra.run.smithi069.stderr:29000000 of 50000000 tuples (58%) done (elapsed 59.42 s, remaining 43.03 s) 2023-07-06T20:20:22.818 INFO:teuthology.orchestra.run.smithi069.stderr:29100000 of 50000000 tuples (58%) done (elapsed 59.63 s, remaining 42.83 s) 2023-07-06T20:20:22.981 INFO:teuthology.orchestra.run.smithi069.stderr:29200000 of 50000000 tuples (58%) done (elapsed 59.79 s, remaining 42.59 s) 2023-07-06T20:20:23.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:22 smithi156 ceph-mon[129280]: pgmap v359: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 139 MiB/s wr, 63 op/s 2023-07-06T20:20:23.120 INFO:teuthology.orchestra.run.smithi069.stderr:29300000 of 50000000 tuples (58%) done (elapsed 59.93 s, remaining 42.34 s) 2023-07-06T20:20:23.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:22 smithi098 ceph-mon[128968]: pgmap v359: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 139 MiB/s wr, 63 op/s 2023-07-06T20:20:23.268 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:22 smithi069 ceph-mon[126399]: pgmap v359: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 139 MiB/s wr, 63 op/s 2023-07-06T20:20:23.492 INFO:teuthology.orchestra.run.smithi069.stderr:29400000 of 50000000 tuples (58%) done (elapsed 60.30 s, remaining 42.25 s) 2023-07-06T20:20:23.962 INFO:teuthology.orchestra.run.smithi069.stderr:29500000 of 50000000 tuples (59%) done (elapsed 60.77 s, remaining 42.23 s) 2023-07-06T20:20:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:29600000 of 50000000 tuples (59%) done (elapsed 61.20 s, remaining 42.18 s) 2023-07-06T20:20:24.565 INFO:teuthology.orchestra.run.smithi069.stderr:29700000 of 50000000 tuples (59%) done (elapsed 61.38 s, remaining 41.95 s) 2023-07-06T20:20:24.743 INFO:teuthology.orchestra.run.smithi069.stderr:29800000 of 50000000 tuples (59%) done (elapsed 61.55 s, remaining 41.73 s) 2023-07-06T20:20:24.880 INFO:teuthology.orchestra.run.smithi069.stderr:29900000 of 50000000 tuples (59%) done (elapsed 61.69 s, remaining 41.47 s) 2023-07-06T20:20:25.080 INFO:teuthology.orchestra.run.smithi069.stderr:30000000 of 50000000 tuples (60%) done (elapsed 61.89 s, remaining 41.26 s) 2023-07-06T20:20:25.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:24 smithi156 ceph-mon[129280]: pgmap v360: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 124 MiB/s wr, 61 op/s 2023-07-06T20:20:25.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:24 smithi098 ceph-mon[128968]: pgmap v360: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 124 MiB/s wr, 61 op/s 2023-07-06T20:20:25.234 INFO:teuthology.orchestra.run.smithi069.stderr:30100000 of 50000000 tuples (60%) done (elapsed 62.05 s, remaining 41.02 s) 2023-07-06T20:20:25.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:24 smithi069 ceph-mon[126399]: pgmap v360: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 124 MiB/s wr, 61 op/s 2023-07-06T20:20:25.374 INFO:teuthology.orchestra.run.smithi069.stderr:30200000 of 50000000 tuples (60%) done (elapsed 62.19 s, remaining 40.77 s) 2023-07-06T20:20:25.570 INFO:teuthology.orchestra.run.smithi069.stderr:30300000 of 50000000 tuples (60%) done (elapsed 62.38 s, remaining 40.56 s) 2023-07-06T20:20:25.751 INFO:teuthology.orchestra.run.smithi069.stderr:30400000 of 50000000 tuples (60%) done (elapsed 62.56 s, remaining 40.34 s) 2023-07-06T20:20:25.903 INFO:teuthology.orchestra.run.smithi069.stderr:30500000 of 50000000 tuples (61%) done (elapsed 62.71 s, remaining 40.10 s) 2023-07-06T20:20:26.103 INFO:teuthology.orchestra.run.smithi069.stderr:30600000 of 50000000 tuples (61%) done (elapsed 62.92 s, remaining 39.89 s) 2023-07-06T20:20:26.259 INFO:teuthology.orchestra.run.smithi069.stderr:30700000 of 50000000 tuples (61%) done (elapsed 63.07 s, remaining 39.65 s) 2023-07-06T20:20:26.443 INFO:teuthology.orchestra.run.smithi069.stderr:30800000 of 50000000 tuples (61%) done (elapsed 63.25 s, remaining 39.43 s) 2023-07-06T20:20:26.615 INFO:teuthology.orchestra.run.smithi069.stderr:30900000 of 50000000 tuples (61%) done (elapsed 63.43 s, remaining 39.21 s) 2023-07-06T20:20:26.747 INFO:teuthology.orchestra.run.smithi069.stderr:31000000 of 50000000 tuples (62%) done (elapsed 63.56 s, remaining 38.96 s) 2023-07-06T20:20:26.905 INFO:teuthology.orchestra.run.smithi069.stderr:31100000 of 50000000 tuples (62%) done (elapsed 63.72 s, remaining 38.72 s) 2023-07-06T20:20:27.093 INFO:teuthology.orchestra.run.smithi069.stderr:31200000 of 50000000 tuples (62%) done (elapsed 63.90 s, remaining 38.51 s) 2023-07-06T20:20:27.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:26 smithi156 ceph-mon[129280]: pgmap v361: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 119 MiB/s wr, 52 op/s 2023-07-06T20:20:27.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:26 smithi098 ceph-mon[128968]: pgmap v361: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 119 MiB/s wr, 52 op/s 2023-07-06T20:20:27.206 INFO:teuthology.orchestra.run.smithi069.stderr:31300000 of 50000000 tuples (62%) done (elapsed 64.02 s, remaining 38.25 s) 2023-07-06T20:20:27.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:26 smithi069 ceph-mon[126399]: pgmap v361: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 119 MiB/s wr, 52 op/s 2023-07-06T20:20:27.405 INFO:teuthology.orchestra.run.smithi069.stderr:31400000 of 50000000 tuples (62%) done (elapsed 64.22 s, remaining 38.04 s) 2023-07-06T20:20:27.583 INFO:teuthology.orchestra.run.smithi069.stderr:31500000 of 50000000 tuples (63%) done (elapsed 64.39 s, remaining 37.82 s) 2023-07-06T20:20:27.716 INFO:teuthology.orchestra.run.smithi069.stderr:31600000 of 50000000 tuples (63%) done (elapsed 64.53 s, remaining 37.57 s) 2023-07-06T20:20:27.914 INFO:teuthology.orchestra.run.smithi069.stderr:31700000 of 50000000 tuples (63%) done (elapsed 64.73 s, remaining 37.37 s) 2023-07-06T20:20:28.073 INFO:teuthology.orchestra.run.smithi069.stderr:31800000 of 50000000 tuples (63%) done (elapsed 64.88 s, remaining 37.14 s) 2023-07-06T20:20:28.229 INFO:teuthology.orchestra.run.smithi069.stderr:31900000 of 50000000 tuples (63%) done (elapsed 65.04 s, remaining 36.90 s) 2023-07-06T20:20:28.431 INFO:teuthology.orchestra.run.smithi069.stderr:32000000 of 50000000 tuples (64%) done (elapsed 65.24 s, remaining 36.70 s) 2023-07-06T20:20:28.614 INFO:teuthology.orchestra.run.smithi069.stderr:32100000 of 50000000 tuples (64%) done (elapsed 65.43 s, remaining 36.48 s) 2023-07-06T20:20:28.806 INFO:teuthology.orchestra.run.smithi069.stderr:32200000 of 50000000 tuples (64%) done (elapsed 65.62 s, remaining 36.27 s) 2023-07-06T20:20:29.000 INFO:teuthology.orchestra.run.smithi069.stderr:32300000 of 50000000 tuples (64%) done (elapsed 65.81 s, remaining 36.06 s) 2023-07-06T20:20:29.105 INFO:teuthology.orchestra.run.smithi069.stderr:32400000 of 50000000 tuples (64%) done (elapsed 65.92 s, remaining 35.81 s) 2023-07-06T20:20:29.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:28 smithi156 ceph-mon[129280]: pgmap v362: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 139 MiB/s wr, 66 op/s 2023-07-06T20:20:29.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:28 smithi098 ceph-mon[128968]: pgmap v362: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 139 MiB/s wr, 66 op/s 2023-07-06T20:20:29.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:28 smithi069 ceph-mon[126399]: pgmap v362: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 139 MiB/s wr, 66 op/s 2023-07-06T20:20:29.288 INFO:teuthology.orchestra.run.smithi069.stderr:32500000 of 50000000 tuples (65%) done (elapsed 66.10 s, remaining 35.59 s) 2023-07-06T20:20:29.460 INFO:teuthology.orchestra.run.smithi069.stderr:32600000 of 50000000 tuples (65%) done (elapsed 66.27 s, remaining 35.37 s) 2023-07-06T20:20:29.604 INFO:teuthology.orchestra.run.smithi069.stderr:32700000 of 50000000 tuples (65%) done (elapsed 66.42 s, remaining 35.14 s) 2023-07-06T20:20:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:32800000 of 50000000 tuples (65%) done (elapsed 66.61 s, remaining 34.93 s) 2023-07-06T20:20:29.968 INFO:teuthology.orchestra.run.smithi069.stderr:32900000 of 50000000 tuples (65%) done (elapsed 66.78 s, remaining 34.71 s) 2023-07-06T20:20:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:33000000 of 50000000 tuples (66%) done (elapsed 66.93 s, remaining 34.48 s) 2023-07-06T20:20:30.266 INFO:teuthology.orchestra.run.smithi069.stderr:33100000 of 50000000 tuples (66%) done (elapsed 67.08 s, remaining 34.25 s) 2023-07-06T20:20:30.479 INFO:teuthology.orchestra.run.smithi069.stderr:33200000 of 50000000 tuples (66%) done (elapsed 67.29 s, remaining 34.05 s) 2023-07-06T20:20:30.861 INFO:teuthology.orchestra.run.smithi069.stderr:33300000 of 50000000 tuples (66%) done (elapsed 67.67 s, remaining 33.94 s) 2023-07-06T20:20:31.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:30 smithi156 ceph-mon[129280]: pgmap v363: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 124 MiB/s wr, 52 op/s 2023-07-06T20:20:31.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:30 smithi098 ceph-mon[128968]: pgmap v363: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 124 MiB/s wr, 52 op/s 2023-07-06T20:20:31.203 INFO:teuthology.orchestra.run.smithi069.stderr:33400000 of 50000000 tuples (66%) done (elapsed 68.02 s, remaining 33.80 s) 2023-07-06T20:20:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:30 smithi069 ceph-mon[126399]: pgmap v363: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 124 MiB/s wr, 52 op/s 2023-07-06T20:20:31.790 INFO:teuthology.orchestra.run.smithi069.stderr:33500000 of 50000000 tuples (67%) done (elapsed 68.60 s, remaining 33.79 s) 2023-07-06T20:20:31.981 INFO:teuthology.orchestra.run.smithi069.stderr:33600000 of 50000000 tuples (67%) done (elapsed 68.79 s, remaining 33.58 s) 2023-07-06T20:20:32.161 INFO:teuthology.orchestra.run.smithi069.stderr:33700000 of 50000000 tuples (67%) done (elapsed 68.97 s, remaining 33.36 s) 2023-07-06T20:20:32.289 INFO:teuthology.orchestra.run.smithi069.stderr:33800000 of 50000000 tuples (67%) done (elapsed 69.10 s, remaining 33.12 s) 2023-07-06T20:20:32.517 INFO:teuthology.orchestra.run.smithi069.stderr:33900000 of 50000000 tuples (67%) done (elapsed 69.33 s, remaining 32.93 s) 2023-07-06T20:20:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:34000000 of 50000000 tuples (68%) done (elapsed 69.51 s, remaining 32.71 s) 2023-07-06T20:20:32.887 INFO:teuthology.orchestra.run.smithi069.stderr:34100000 of 50000000 tuples (68%) done (elapsed 69.70 s, remaining 32.50 s) 2023-07-06T20:20:33.054 INFO:teuthology.orchestra.run.smithi069.stderr:34200000 of 50000000 tuples (68%) done (elapsed 69.87 s, remaining 32.28 s) 2023-07-06T20:20:33.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:32 smithi156 ceph-mon[129280]: pgmap v364: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 118 MiB/s wr, 50 op/s 2023-07-06T20:20:33.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:32 smithi098 ceph-mon[128968]: pgmap v364: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 118 MiB/s wr, 50 op/s 2023-07-06T20:20:33.207 INFO:teuthology.orchestra.run.smithi069.stderr:34300000 of 50000000 tuples (68%) done (elapsed 70.02 s, remaining 32.05 s) 2023-07-06T20:20:33.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:32 smithi069 ceph-mon[126399]: pgmap v364: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 118 MiB/s wr, 50 op/s 2023-07-06T20:20:33.381 INFO:teuthology.orchestra.run.smithi069.stderr:34400000 of 50000000 tuples (68%) done (elapsed 70.19 s, remaining 31.83 s) 2023-07-06T20:20:33.567 INFO:teuthology.orchestra.run.smithi069.stderr:34500000 of 50000000 tuples (69%) done (elapsed 70.38 s, remaining 31.62 s) 2023-07-06T20:20:33.675 INFO:teuthology.orchestra.run.smithi069.stderr:34600000 of 50000000 tuples (69%) done (elapsed 70.49 s, remaining 31.37 s) 2023-07-06T20:20:33.854 INFO:teuthology.orchestra.run.smithi069.stderr:34700000 of 50000000 tuples (69%) done (elapsed 70.67 s, remaining 31.16 s) 2023-07-06T20:20:34.020 INFO:teuthology.orchestra.run.smithi069.stderr:34800000 of 50000000 tuples (69%) done (elapsed 70.83 s, remaining 30.94 s) 2023-07-06T20:20:34.159 INFO:teuthology.orchestra.run.smithi069.stderr:34900000 of 50000000 tuples (69%) done (elapsed 70.97 s, remaining 30.71 s) 2023-07-06T20:20:34.353 INFO:teuthology.orchestra.run.smithi069.stderr:35000000 of 50000000 tuples (70%) done (elapsed 71.17 s, remaining 30.50 s) 2023-07-06T20:20:34.505 INFO:teuthology.orchestra.run.smithi069.stderr:35100000 of 50000000 tuples (70%) done (elapsed 71.32 s, remaining 30.27 s) 2023-07-06T20:20:34.655 INFO:teuthology.orchestra.run.smithi069.stderr:35200000 of 50000000 tuples (70%) done (elapsed 71.47 s, remaining 30.05 s) 2023-07-06T20:20:34.853 INFO:teuthology.orchestra.run.smithi069.stderr:35300000 of 50000000 tuples (70%) done (elapsed 71.67 s, remaining 29.84 s) 2023-07-06T20:20:35.022 INFO:teuthology.orchestra.run.smithi069.stderr:35400000 of 50000000 tuples (70%) done (elapsed 71.83 s, remaining 29.63 s) 2023-07-06T20:20:35.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:34 smithi156 ceph-mon[129280]: pgmap v365: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 121 MiB/s wr, 61 op/s 2023-07-06T20:20:35.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:34 smithi098 ceph-mon[128968]: pgmap v365: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 121 MiB/s wr, 61 op/s 2023-07-06T20:20:35.169 INFO:teuthology.orchestra.run.smithi069.stderr:35500000 of 50000000 tuples (71%) done (elapsed 71.98 s, remaining 29.40 s) 2023-07-06T20:20:35.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:34 smithi069 ceph-mon[126399]: pgmap v365: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 121 MiB/s wr, 61 op/s 2023-07-06T20:20:35.318 INFO:teuthology.orchestra.run.smithi069.stderr:35600000 of 50000000 tuples (71%) done (elapsed 72.13 s, remaining 29.18 s) 2023-07-06T20:20:35.505 INFO:teuthology.orchestra.run.smithi069.stderr:35700000 of 50000000 tuples (71%) done (elapsed 72.32 s, remaining 28.97 s) 2023-07-06T20:20:35.651 INFO:teuthology.orchestra.run.smithi069.stderr:35800000 of 50000000 tuples (71%) done (elapsed 72.46 s, remaining 28.74 s) 2023-07-06T20:20:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:35900000 of 50000000 tuples (71%) done (elapsed 72.62 s, remaining 28.52 s) 2023-07-06T20:20:36.013 INFO:teuthology.orchestra.run.smithi069.stderr:36000000 of 50000000 tuples (72%) done (elapsed 72.83 s, remaining 28.32 s) 2023-07-06T20:20:36.193 INFO:teuthology.orchestra.run.smithi069.stderr:36100000 of 50000000 tuples (72%) done (elapsed 73.00 s, remaining 28.11 s) 2023-07-06T20:20:36.367 INFO:teuthology.orchestra.run.smithi069.stderr:36200000 of 50000000 tuples (72%) done (elapsed 73.18 s, remaining 27.90 s) 2023-07-06T20:20:36.472 INFO:teuthology.orchestra.run.smithi069.stderr:36300000 of 50000000 tuples (72%) done (elapsed 73.28 s, remaining 27.66 s) 2023-07-06T20:20:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:36400000 of 50000000 tuples (72%) done (elapsed 73.47 s, remaining 27.45 s) 2023-07-06T20:20:36.834 INFO:teuthology.orchestra.run.smithi069.stderr:36500000 of 50000000 tuples (73%) done (elapsed 73.65 s, remaining 27.24 s) 2023-07-06T20:20:36.957 INFO:teuthology.orchestra.run.smithi069.stderr:36600000 of 50000000 tuples (73%) done (elapsed 73.77 s, remaining 27.01 s) 2023-07-06T20:20:37.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:36 smithi156 ceph-mon[129280]: pgmap v366: 97 pgs: 97 active+clean; 5.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 135 MiB/s wr, 65 op/s 2023-07-06T20:20:37.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:36 smithi098 ceph-mon[128968]: pgmap v366: 97 pgs: 97 active+clean; 5.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 135 MiB/s wr, 65 op/s 2023-07-06T20:20:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:36700000 of 50000000 tuples (73%) done (elapsed 73.97 s, remaining 26.81 s) 2023-07-06T20:20:37.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:36 smithi069 ceph-mon[126399]: pgmap v366: 97 pgs: 97 active+clean; 5.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 135 MiB/s wr, 65 op/s 2023-07-06T20:20:37.324 INFO:teuthology.orchestra.run.smithi069.stderr:36800000 of 50000000 tuples (73%) done (elapsed 74.14 s, remaining 26.59 s) 2023-07-06T20:20:37.460 INFO:teuthology.orchestra.run.smithi069.stderr:36900000 of 50000000 tuples (73%) done (elapsed 74.27 s, remaining 26.37 s) 2023-07-06T20:20:37.628 INFO:teuthology.orchestra.run.smithi069.stderr:37000000 of 50000000 tuples (74%) done (elapsed 74.44 s, remaining 26.15 s) 2023-07-06T20:20:37.892 INFO:teuthology.orchestra.run.smithi069.stderr:37100000 of 50000000 tuples (74%) done (elapsed 74.70 s, remaining 25.97 s) 2023-07-06T20:20:38.280 INFO:teuthology.orchestra.run.smithi069.stderr:37200000 of 50000000 tuples (74%) done (elapsed 75.09 s, remaining 25.84 s) 2023-07-06T20:20:38.772 INFO:teuthology.orchestra.run.smithi069.stderr:37300000 of 50000000 tuples (74%) done (elapsed 75.58 s, remaining 25.73 s) 2023-07-06T20:20:38.899 INFO:teuthology.orchestra.run.smithi069.stderr:37400000 of 50000000 tuples (74%) done (elapsed 75.71 s, remaining 25.51 s) 2023-07-06T20:20:39.080 INFO:teuthology.orchestra.run.smithi069.stderr:37500000 of 50000000 tuples (75%) done (elapsed 75.89 s, remaining 25.30 s) 2023-07-06T20:20:39.245 INFO:teuthology.orchestra.run.smithi069.stderr:37600000 of 50000000 tuples (75%) done (elapsed 76.06 s, remaining 25.08 s) 2023-07-06T20:20:39.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:38 smithi069 ceph-mon[126399]: pgmap v367: 97 pgs: 97 active+clean; 5.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 128 MiB/s wr, 64 op/s 2023-07-06T20:20:39.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:38 smithi156 ceph-mon[129280]: pgmap v367: 97 pgs: 97 active+clean; 5.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 128 MiB/s wr, 64 op/s 2023-07-06T20:20:39.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:38 smithi098 ceph-mon[128968]: pgmap v367: 97 pgs: 97 active+clean; 5.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 128 MiB/s wr, 64 op/s 2023-07-06T20:20:39.394 INFO:teuthology.orchestra.run.smithi069.stderr:37700000 of 50000000 tuples (75%) done (elapsed 76.21 s, remaining 24.86 s) 2023-07-06T20:20:39.596 INFO:teuthology.orchestra.run.smithi069.stderr:37800000 of 50000000 tuples (75%) done (elapsed 76.41 s, remaining 24.66 s) 2023-07-06T20:20:39.771 INFO:teuthology.orchestra.run.smithi069.stderr:37900000 of 50000000 tuples (75%) done (elapsed 76.58 s, remaining 24.45 s) 2023-07-06T20:20:39.973 INFO:teuthology.orchestra.run.smithi069.stderr:38000000 of 50000000 tuples (76%) done (elapsed 76.78 s, remaining 24.25 s) 2023-07-06T20:20:40.128 INFO:teuthology.orchestra.run.smithi069.stderr:38100000 of 50000000 tuples (76%) done (elapsed 76.94 s, remaining 24.03 s) 2023-07-06T20:20:40.282 INFO:teuthology.orchestra.run.smithi069.stderr:38200000 of 50000000 tuples (76%) done (elapsed 77.09 s, remaining 23.81 s) 2023-07-06T20:20:40.477 INFO:teuthology.orchestra.run.smithi069.stderr:38300000 of 50000000 tuples (76%) done (elapsed 77.29 s, remaining 23.61 s) 2023-07-06T20:20:40.658 INFO:teuthology.orchestra.run.smithi069.stderr:38400000 of 50000000 tuples (76%) done (elapsed 77.47 s, remaining 23.40 s) 2023-07-06T20:20:40.772 INFO:teuthology.orchestra.run.smithi069.stderr:38500000 of 50000000 tuples (77%) done (elapsed 77.58 s, remaining 23.17 s) 2023-07-06T20:20:40.980 INFO:teuthology.orchestra.run.smithi069.stderr:38600000 of 50000000 tuples (77%) done (elapsed 77.79 s, remaining 22.97 s) 2023-07-06T20:20:41.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:40 smithi156 ceph-mon[129280]: pgmap v368: 97 pgs: 97 active+clean; 5.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 111 MiB/s wr, 50 op/s 2023-07-06T20:20:41.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:40 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:20:41.163 INFO:teuthology.orchestra.run.smithi069.stderr:38700000 of 50000000 tuples (77%) done (elapsed 77.97 s, remaining 22.77 s) 2023-07-06T20:20:41.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:40 smithi069 ceph-mon[126399]: pgmap v368: 97 pgs: 97 active+clean; 5.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 111 MiB/s wr, 50 op/s 2023-07-06T20:20:41.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:40 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:20:41.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:40 smithi098 ceph-mon[128968]: pgmap v368: 97 pgs: 97 active+clean; 5.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 111 MiB/s wr, 50 op/s 2023-07-06T20:20:41.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:40 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:20:41.396 INFO:teuthology.orchestra.run.smithi069.stderr:38800000 of 50000000 tuples (77%) done (elapsed 78.21 s, remaining 22.58 s) 2023-07-06T20:20:41.570 INFO:teuthology.orchestra.run.smithi069.stderr:38900000 of 50000000 tuples (77%) done (elapsed 78.38 s, remaining 22.37 s) 2023-07-06T20:20:41.721 INFO:teuthology.orchestra.run.smithi069.stderr:39000000 of 50000000 tuples (78%) done (elapsed 78.53 s, remaining 22.15 s) 2023-07-06T20:20:41.884 INFO:teuthology.orchestra.run.smithi069.stderr:39100000 of 50000000 tuples (78%) done (elapsed 78.70 s, remaining 21.94 s) 2023-07-06T20:20:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:39200000 of 50000000 tuples (78%) done (elapsed 78.87 s, remaining 21.73 s) 2023-07-06T20:20:42.221 INFO:teuthology.orchestra.run.smithi069.stderr:39300000 of 50000000 tuples (78%) done (elapsed 79.03 s, remaining 21.52 s) 2023-07-06T20:20:42.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:41 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:20:42.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:41 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:20:42.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:41 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:20:42.343 INFO:teuthology.orchestra.run.smithi069.stderr:39400000 of 50000000 tuples (78%) done (elapsed 79.15 s, remaining 21.30 s) 2023-07-06T20:20:42.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:41 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:20:42.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:41 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:20:42.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:41 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:20:42.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:41 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:20:42.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:41 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:20:42.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:41 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:20:42.505 INFO:teuthology.orchestra.run.smithi069.stderr:39500000 of 50000000 tuples (79%) done (elapsed 79.32 s, remaining 21.08 s) 2023-07-06T20:20:42.709 INFO:teuthology.orchestra.run.smithi069.stderr:39600000 of 50000000 tuples (79%) done (elapsed 79.52 s, remaining 20.88 s) 2023-07-06T20:20:42.837 INFO:teuthology.orchestra.run.smithi069.stderr:39700000 of 50000000 tuples (79%) done (elapsed 79.65 s, remaining 20.66 s) 2023-07-06T20:20:43.021 INFO:teuthology.orchestra.run.smithi069.stderr:39800000 of 50000000 tuples (79%) done (elapsed 79.83 s, remaining 20.46 s) 2023-07-06T20:20:43.213 INFO:teuthology.orchestra.run.smithi069.stderr:39900000 of 50000000 tuples (79%) done (elapsed 80.02 s, remaining 20.26 s) 2023-07-06T20:20:43.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:42 smithi069 ceph-mon[126399]: pgmap v369: 97 pgs: 97 active+clean; 5.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 126 MiB/s wr, 55 op/s 2023-07-06T20:20:43.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:42 smithi156 ceph-mon[129280]: pgmap v369: 97 pgs: 97 active+clean; 5.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 126 MiB/s wr, 55 op/s 2023-07-06T20:20:43.374 INFO:teuthology.orchestra.run.smithi069.stderr:40000000 of 50000000 tuples (80%) done (elapsed 80.19 s, remaining 20.05 s) 2023-07-06T20:20:43.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:42 smithi098 ceph-mon[128968]: pgmap v369: 97 pgs: 97 active+clean; 5.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 126 MiB/s wr, 55 op/s 2023-07-06T20:20:43.552 INFO:teuthology.orchestra.run.smithi069.stderr:40100000 of 50000000 tuples (80%) done (elapsed 80.36 s, remaining 19.84 s) 2023-07-06T20:20:43.738 INFO:teuthology.orchestra.run.smithi069.stderr:40200000 of 50000000 tuples (80%) done (elapsed 80.55 s, remaining 19.64 s) 2023-07-06T20:20:43.913 INFO:teuthology.orchestra.run.smithi069.stderr:40300000 of 50000000 tuples (80%) done (elapsed 80.73 s, remaining 19.43 s) 2023-07-06T20:20:44.026 INFO:teuthology.orchestra.run.smithi069.stderr:40400000 of 50000000 tuples (80%) done (elapsed 80.84 s, remaining 19.21 s) 2023-07-06T20:20:44.186 INFO:teuthology.orchestra.run.smithi069.stderr:40500000 of 50000000 tuples (81%) done (elapsed 81.00 s, remaining 19.00 s) 2023-07-06T20:20:44.357 INFO:teuthology.orchestra.run.smithi069.stderr:40600000 of 50000000 tuples (81%) done (elapsed 81.17 s, remaining 18.79 s) 2023-07-06T20:20:44.464 INFO:teuthology.orchestra.run.smithi069.stderr:40700000 of 50000000 tuples (81%) done (elapsed 81.28 s, remaining 18.57 s) 2023-07-06T20:20:44.671 INFO:teuthology.orchestra.run.smithi069.stderr:40800000 of 50000000 tuples (81%) done (elapsed 81.48 s, remaining 18.37 s) 2023-07-06T20:20:44.841 INFO:teuthology.orchestra.run.smithi069.stderr:40900000 of 50000000 tuples (81%) done (elapsed 81.65 s, remaining 18.17 s) 2023-07-06T20:20:45.074 INFO:teuthology.orchestra.run.smithi069.stderr:41000000 of 50000000 tuples (82%) done (elapsed 81.88 s, remaining 17.97 s) 2023-07-06T20:20:45.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:44 smithi069 ceph-mon[126399]: pgmap v370: 97 pgs: 97 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 138 MiB/s wr, 67 op/s 2023-07-06T20:20:45.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:44 smithi156 ceph-mon[129280]: pgmap v370: 97 pgs: 97 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 138 MiB/s wr, 67 op/s 2023-07-06T20:20:45.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:44 smithi098 ceph-mon[128968]: pgmap v370: 97 pgs: 97 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 138 MiB/s wr, 67 op/s 2023-07-06T20:20:45.627 INFO:teuthology.orchestra.run.smithi069.stderr:41100000 of 50000000 tuples (82%) done (elapsed 82.44 s, remaining 17.85 s) 2023-07-06T20:20:45.983 INFO:teuthology.orchestra.run.smithi069.stderr:41200000 of 50000000 tuples (82%) done (elapsed 82.80 s, remaining 17.68 s) 2023-07-06T20:20:46.121 INFO:teuthology.orchestra.run.smithi069.stderr:41300000 of 50000000 tuples (82%) done (elapsed 82.93 s, remaining 17.47 s) 2023-07-06T20:20:46.338 INFO:teuthology.orchestra.run.smithi069.stderr:41400000 of 50000000 tuples (82%) done (elapsed 83.15 s, remaining 17.27 s) 2023-07-06T20:20:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:41500000 of 50000000 tuples (83%) done (elapsed 83.31 s, remaining 17.06 s) 2023-07-06T20:20:46.704 INFO:teuthology.orchestra.run.smithi069.stderr:41600000 of 50000000 tuples (83%) done (elapsed 83.52 s, remaining 16.86 s) 2023-07-06T20:20:46.861 INFO:teuthology.orchestra.run.smithi069.stderr:41700000 of 50000000 tuples (83%) done (elapsed 83.67 s, remaining 16.65 s) 2023-07-06T20:20:47.006 INFO:teuthology.orchestra.run.smithi069.stderr:41800000 of 50000000 tuples (83%) done (elapsed 83.82 s, remaining 16.44 s) 2023-07-06T20:20:47.208 INFO:teuthology.orchestra.run.smithi069.stderr:41900000 of 50000000 tuples (83%) done (elapsed 84.02 s, remaining 16.24 s) 2023-07-06T20:20:47.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:46 smithi069 ceph-mon[126399]: pgmap v371: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 123 MiB/s wr, 54 op/s 2023-07-06T20:20:47.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:46 smithi156 ceph-mon[129280]: pgmap v371: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 123 MiB/s wr, 54 op/s 2023-07-06T20:20:47.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:46 smithi098 ceph-mon[128968]: pgmap v371: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 123 MiB/s wr, 54 op/s 2023-07-06T20:20:47.382 INFO:teuthology.orchestra.run.smithi069.stderr:42000000 of 50000000 tuples (84%) done (elapsed 84.19 s, remaining 16.04 s) 2023-07-06T20:20:47.497 INFO:teuthology.orchestra.run.smithi069.stderr:42100000 of 50000000 tuples (84%) done (elapsed 84.31 s, remaining 15.82 s) 2023-07-06T20:20:47.655 INFO:teuthology.orchestra.run.smithi069.stderr:42200000 of 50000000 tuples (84%) done (elapsed 84.47 s, remaining 15.61 s) 2023-07-06T20:20:47.826 INFO:teuthology.orchestra.run.smithi069.stderr:42300000 of 50000000 tuples (84%) done (elapsed 84.64 s, remaining 15.41 s) 2023-07-06T20:20:47.938 INFO:teuthology.orchestra.run.smithi069.stderr:42400000 of 50000000 tuples (84%) done (elapsed 84.75 s, remaining 15.19 s) 2023-07-06T20:20:48.127 INFO:teuthology.orchestra.run.smithi069.stderr:42500000 of 50000000 tuples (85%) done (elapsed 84.94 s, remaining 14.99 s) 2023-07-06T20:20:48.296 INFO:teuthology.orchestra.run.smithi069.stderr:42600000 of 50000000 tuples (85%) done (elapsed 85.11 s, remaining 14.78 s) 2023-07-06T20:20:48.442 INFO:teuthology.orchestra.run.smithi069.stderr:42700000 of 50000000 tuples (85%) done (elapsed 85.25 s, remaining 14.58 s) 2023-07-06T20:20:48.522 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:20:48.643 INFO:teuthology.orchestra.run.smithi069.stderr:42800000 of 50000000 tuples (85%) done (elapsed 85.45 s, remaining 14.38 s) 2023-07-06T20:20:48.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.751+0000 7fe7b27ff700 1 -- 172.21.15.69:0/1752454842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac074680 msgr2=0x7fe7ac072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:48.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.751+0000 7fe7b27ff700 1 --2- 172.21.15.69:0/1752454842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac074680 0x7fe7ac072ad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe79c009990 tx=0x7fe79c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.751+0000 7fe7b27ff700 1 -- 172.21.15.69:0/1752454842 shutdown_connections 2023-07-06T20:20:48.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.751+0000 7fe7b27ff700 1 --2- 172.21.15.69:0/1752454842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7ac107ae0 0x7fe7ac109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.751+0000 7fe7b27ff700 1 --2- 172.21.15.69:0/1752454842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac074680 0x7fe7ac072ad0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.751+0000 7fe7b27ff700 1 --2- 172.21.15.69:0/1752454842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac0730a0 0x7fe7ac073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.751+0000 7fe7b27ff700 1 -- 172.21.15.69:0/1752454842 >> 172.21.15.69:0/1752454842 conn(0x7fe7ac0fb260 msgr2=0x7fe7ac0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:20:48.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.751+0000 7fe7b27ff700 1 -- 172.21.15.69:0/1752454842 shutdown_connections 2023-07-06T20:20:48.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.751+0000 7fe7b27ff700 1 -- 172.21.15.69:0/1752454842 wait complete. 2023-07-06T20:20:48.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.752+0000 7fe7b27ff700 1 Processor -- start 2023-07-06T20:20:48.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.752+0000 7fe7b27ff700 1 -- start start 2023-07-06T20:20:48.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.752+0000 7fe7b27ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac0730a0 0x7fe7ac197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7b27ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7ac074680 0x7fe7ac198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7b27ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac107ae0 0x7fe7ac19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7b27ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac19f960 con 0x7fe7ac074680 2023-07-06T20:20:48.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7b27ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac19faa0 con 0x7fe7ac0730a0 2023-07-06T20:20:48.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7b27ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac19fc30 con 0x7fe7ac107ae0 2023-07-06T20:20:48.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7abfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac0730a0 0x7fe7ac197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7ab7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7ac074680 0x7fe7ac198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7ab7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7ac074680 0x7fe7ac198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36888/0 (socket says 172.21.15.69:36888) 2023-07-06T20:20:48.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7ab7fe700 1 -- 172.21.15.69:0/4221099928 learned_addr learned my addr 172.21.15.69:0/4221099928 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:20:48.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7abfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac0730a0 0x7fe7ac197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:46092/0 (socket says 172.21.15.69:46092) 2023-07-06T20:20:48.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7b0d9c700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac107ae0 0x7fe7ac19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7ab7fe700 1 -- 172.21.15.69:0/4221099928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac107ae0 msgr2=0x7fe7ac19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:48.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7ab7fe700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac107ae0 0x7fe7ac19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7ab7fe700 1 -- 172.21.15.69:0/4221099928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac0730a0 msgr2=0x7fe7ac197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:48.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7ab7fe700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac0730a0 0x7fe7ac197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7ab7fe700 1 -- 172.21.15.69:0/4221099928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c0095f0 con 0x7fe7ac074680 2023-07-06T20:20:48.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.753+0000 7fe7abfff700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac0730a0 0x7fe7ac197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:20:48.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.754+0000 7fe7ab7fe700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7ac074680 0x7fe7ac198320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000e970 tx=0x7fe7a000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:20:48.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.754+0000 7fe7a97fa700 1 -- 172.21.15.69:0/4221099928 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0009790 con 0x7fe7ac074680 2023-07-06T20:20:48.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.754+0000 7fe7b27ff700 1 -- 172.21.15.69:0/4221099928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ac19ff10 con 0x7fe7ac074680 2023-07-06T20:20:48.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.754+0000 7fe7a97fa700 1 -- 172.21.15.69:0/4221099928 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7a000c7e0 con 0x7fe7ac074680 2023-07-06T20:20:48.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.754+0000 7fe7a97fa700 1 -- 172.21.15.69:0/4221099928 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0003c70 con 0x7fe7ac074680 2023-07-06T20:20:48.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.754+0000 7fe7b27ff700 1 -- 172.21.15.69:0/4221099928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ac1a0390 con 0x7fe7ac074680 2023-07-06T20:20:48.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.757+0000 7fe7a97fa700 1 -- 172.21.15.69:0/4221099928 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe7a000cc50 con 0x7fe7ac074680 2023-07-06T20:20:48.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.758+0000 7fe7c61c7b80 1 Processor -- start 2023-07-06T20:20:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7c61c7b80 1 -- start start 2023-07-06T20:20:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7a97fa700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe7940a1150 0x7fe7940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7c61c7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc8906330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7c61c7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8906870 0x55bcc8906e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7a97fa700 1 -- 172.21.15.69:0/4221099928 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fe7a0014070 con 0x7fe7ac074680 2023-07-06T20:20:48.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7c61c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8907390 0x55bcc8907830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7c61c7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc880acb0 con 0x55bcc8906870 2023-07-06T20:20:48.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7c61c7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc885da40 con 0x55bcc8905f20 2023-07-06T20:20:48.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7c61c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc8856ed0 con 0x55bcc8907390 2023-07-06T20:20:48.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7abfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc8906330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7abfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc8906330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:46104/0 (socket says 172.21.15.69:46104) 2023-07-06T20:20:48.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7b0d9c700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe7940a1150 0x7fe7940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7abfff700 1 -- 172.21.15.69:0/2681052155 learned_addr learned my addr 172.21.15.69:0/2681052155 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:20:48.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7b0d9c700 1 --2- 172.21.15.69:0/2681052155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8907390 0x55bcc8907830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.759+0000 7fe7ab7fe700 1 --2- 172.21.15.69:0/2681052155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8906870 0x55bcc8906e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.760+0000 7fe7abfff700 1 -- 172.21.15.69:0/2681052155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8907390 msgr2=0x55bcc8907830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:48.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.760+0000 7fe7abfff700 1 --2- 172.21.15.69:0/2681052155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8907390 0x55bcc8907830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.760+0000 7fe7abfff700 1 -- 172.21.15.69:0/2681052155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8906870 msgr2=0x55bcc8906e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:48.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.760+0000 7fe7abfff700 1 --2- 172.21.15.69:0/2681052155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8906870 0x55bcc8906e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.760+0000 7fe7abfff700 1 -- 172.21.15.69:0/2681052155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcc8684de0 con 0x55bcc8905f20 2023-07-06T20:20:48.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.760+0000 7fe7ab7fe700 1 --2- 172.21.15.69:0/2681052155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8906870 0x55bcc8906e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:20:48.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.760+0000 7fe7abfff700 1 --2- 172.21.15.69:0/2681052155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc8906330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe79c009990 tx=0x7fe79c005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e69d72af3b39447 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:20:48.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.760+0000 7fe7b0d9c700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe7940a1150 0x7fe7940a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009ec0 tx=0x7fe7a4009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:20:48.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.760+0000 7fe7927fc700 1 -- 172.21.15.69:0/2681052155 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c01d050 con 0x55bcc8905f20 2023-07-06T20:20:48.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.760+0000 7fe7927fc700 1 -- 172.21.15.69:0/2681052155 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe79c01e040 con 0x55bcc8905f20 2023-07-06T20:20:48.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.760+0000 7fe7927fc700 1 -- 172.21.15.69:0/2681052155 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c005660 con 0x55bcc8905f20 2023-07-06T20:20:48.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.761+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2681052155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 msgr2=0x55bcc8906330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:48.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.761+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/2681052155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc8906330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe79c009990 tx=0x7fe79c005ea0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.761+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2681052155 shutdown_connections 2023-07-06T20:20:48.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.761+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/2681052155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8907390 0x55bcc8907830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.761+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/2681052155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc8906330 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.761+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/2681052155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8906870 0x55bcc8906e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.761+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2681052155 >> 172.21.15.69:0/2681052155 conn(0x55bcc88fb640 msgr2=0x55bcc89009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:20:48.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.761+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2681052155 shutdown_connections 2023-07-06T20:20:48.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.761+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2681052155 wait complete. 2023-07-06T20:20:48.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.762+0000 7fe7c61c7b80 1 -- start start 2023-07-06T20:20:48.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.762+0000 7fe7c61c7b80 1 Processor -- start 2023-07-06T20:20:48.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7c61c7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc890f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7c61c7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8906870 0x55bcc890fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7c61c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8907390 0x55bcc8916d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7c61c7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc87f6ac0 con 0x55bcc8906870 2023-07-06T20:20:48.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7c61c7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc87f67c0 con 0x55bcc8905f20 2023-07-06T20:20:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7c61c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc86da920 con 0x55bcc8907390 2023-07-06T20:20:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7c61c7b80 10 client.0 _ll_register_callbacks cb 0x55bcc87f4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:20:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7abfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc890f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7abfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc890f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:46110/0 (socket says 172.21.15.69:46110) 2023-07-06T20:20:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7abfff700 1 -- 172.21.15.69:0/2290040773 learned_addr learned my addr 172.21.15.69:0/2290040773 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:20:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7b0d9c700 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8907390 0x55bcc8916d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7ab7fe700 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8906870 0x55bcc890fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7abfff700 1 -- 172.21.15.69:0/2290040773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8907390 msgr2=0x55bcc8916d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:48.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7abfff700 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8907390 0x55bcc8916d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7abfff700 1 -- 172.21.15.69:0/2290040773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8906870 msgr2=0x55bcc890fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:48.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7abfff700 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8906870 0x55bcc890fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.763+0000 7fe7abfff700 1 -- 172.21.15.69:0/2290040773 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ac10ab80 con 0x55bcc8905f20 2023-07-06T20:20:48.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.764+0000 7fe7b0d9c700 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8907390 0x55bcc8916d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:20:48.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.764+0000 7fe7abfff700 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc890f800 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe79c005f50 tx=0x7fe79c005510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2770bb4116690f8b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:20:48.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.764+0000 7fe78b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:20:48.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.764+0000 7fe78b7fe700 1 -- 172.21.15.69:0/2290040773 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c004cb0 con 0x55bcc8905f20 2023-07-06T20:20:48.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.764+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2290040773 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcc87b04f0 con 0x55bcc8905f20 2023-07-06T20:20:48.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.764+0000 7fe78b7fe700 1 -- 172.21.15.69:0/2290040773 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fe79c023460 con 0x55bcc8905f20 2023-07-06T20:20:48.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.764+0000 7fe78b7fe700 1 -- 172.21.15.69:0/2290040773 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe79c01d020 con 0x55bcc8905f20 2023-07-06T20:20:48.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.764+0000 7fe7c61c7b80 10 client.25475 fetch_fsmap learned FSMap version 27 2023-07-06T20:20:48.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.764+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2290040773 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bcc866dc60 con 0x55bcc8905f20 2023-07-06T20:20:48.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.765+0000 7fe78b7fe700 1 -- 172.21.15.69:0/2290040773 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bcc87b04f0 con 0x55bcc8905f20 2023-07-06T20:20:48.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.765+0000 7fe78b7fe700 1 -- 172.21.15.69:0/2290040773 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe79c00f610 con 0x55bcc8905f20 2023-07-06T20:20:48.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.765+0000 7fe7c61c7b80 10 client.25475 fetch_fsmap finished waiting for FSMap version 27 2023-07-06T20:20:48.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.765+0000 7fe7c61c7b80 10 client.25475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:20:48.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.765+0000 7fe7c61c7b80 20 client.25475 populate_metadata read hostname 'smithi069' 2023-07-06T20:20:48.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.765+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bcc891afb0 0x55bcc891d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.765+0000 7fe7c61c7b80 4 client.25475 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:20:48.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.765+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2290040773 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcc88d4d60 con 0x55bcc891afb0 2023-07-06T20:20:48.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.765+0000 7fe7ab7fe700 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bcc891afb0 0x55bcc891d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.766+0000 7fe7ab7fe700 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bcc891afb0 0x55bcc891d3c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca41a3a80dbba684 server_cookie=df60c3a19617e873 in_seq=0 out_seq=0 2023-07-06T20:20:48.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.766+0000 7fe78b7fe700 10 client.25475 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:20:48.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.766+0000 7fe78b7fe700 1 -- 172.21.15.69:0/2290040773 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bcc88d4d60 con 0x55bcc891afb0 2023-07-06T20:20:48.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.766+0000 7fe78b7fe700 10 client.25475 handle_command_reply: tid=0 2023-07-06T20:20:48.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.766+0000 7fe7c61c7b80 1 client.25475 shutdown 2023-07-06T20:20:48.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.767+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2290040773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 msgr2=0x55bcc890f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:48.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.767+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc890f800 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe79c005f50 tx=0x7fe79c005510 comp rx=0 tx=0).stop 2023-07-06T20:20:48.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.767+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2290040773 shutdown_connections 2023-07-06T20:20:48.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.767+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bcc891afb0 0x55bcc891d3c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.767+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8907390 0x55bcc8916d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.767+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc890f800 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.767+0000 7fe78b7fe700 0 client.25475 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:20:48.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.767+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/2290040773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8906870 0x55bcc890fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.767+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2290040773 >> 172.21.15.69:0/2290040773 conn(0x55bcc88fb640 msgr2=0x55bcc89000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:20:48.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.767+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2290040773 shutdown_connections 2023-07-06T20:20:48.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.767+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/2290040773 wait complete. 2023-07-06T20:20:48.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.767+0000 7fe7c61c7b80 20 client.25475 trim_cache size 0 max 16384 2023-07-06T20:20:48.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7c61c7b80 1 Processor -- start 2023-07-06T20:20:48.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7c61c7b80 1 -- start start 2023-07-06T20:20:48.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7c61c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8905f20 0x55bcc8917670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7c61c7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8906870 0x55bcc8916d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7c61c7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8907390 0x55bcc890a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7c61c7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc890f3d0 con 0x55bcc8907390 2023-07-06T20:20:48.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7c61c7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc88fb210 con 0x55bcc8906870 2023-07-06T20:20:48.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7c61c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc88fb450 con 0x55bcc8905f20 2023-07-06T20:20:48.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7ab7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8906870 0x55bcc8916d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7b0d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8907390 0x55bcc890a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7ab7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8906870 0x55bcc8916d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:46114/0 (socket says 172.21.15.69:46114) 2023-07-06T20:20:48.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7ab7fe700 1 -- 172.21.15.69:0/166613328 learned_addr learned my addr 172.21.15.69:0/166613328 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:20:48.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.785+0000 7fe7abfff700 1 --2- 172.21.15.69:0/166613328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8905f20 0x55bcc8917670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.786+0000 7fe7abfff700 1 -- 172.21.15.69:0/166613328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8906870 msgr2=0x55bcc8916d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:48.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.786+0000 7fe7abfff700 1 --2- 172.21.15.69:0/166613328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8906870 0x55bcc8916d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.786+0000 7fe7abfff700 1 -- 172.21.15.69:0/166613328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8907390 msgr2=0x55bcc890a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:48.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.786+0000 7fe7abfff700 1 --2- 172.21.15.69:0/166613328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8907390 0x55bcc890a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.786+0000 7fe7abfff700 1 -- 172.21.15.69:0/166613328 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcc86ab250 con 0x55bcc8905f20 2023-07-06T20:20:48.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.786+0000 7fe7ab7fe700 1 --2- 172.21.15.69:0/166613328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8906870 0x55bcc8916d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:20:48.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.786+0000 7fe7abfff700 1 --2- 172.21.15.69:0/166613328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8905f20 0x55bcc8917670 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00fa20 tx=0x7fe79c005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73b745353b6620e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:20:48.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.786+0000 7fe792ffd700 1 -- 172.21.15.69:0/166613328 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c036da0 con 0x55bcc8905f20 2023-07-06T20:20:48.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.786+0000 7fe792ffd700 1 -- 172.21.15.69:0/166613328 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe79c0051a0 con 0x55bcc8905f20 2023-07-06T20:20:48.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.786+0000 7fe792ffd700 1 -- 172.21.15.69:0/166613328 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c004da0 con 0x55bcc8905f20 2023-07-06T20:20:48.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.787+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/166613328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8905f20 msgr2=0x55bcc8917670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:48.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.787+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/166613328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8905f20 0x55bcc8917670 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00fa20 tx=0x7fe79c005fb0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.787+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/166613328 shutdown_connections 2023-07-06T20:20:48.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.787+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/166613328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8907390 0x55bcc890a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.787+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/166613328 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8905f20 0x55bcc8917670 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.787+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/166613328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8906870 0x55bcc8916d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.787+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/166613328 >> 172.21.15.69:0/166613328 conn(0x55bcc88fb640 msgr2=0x55bcc8921d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:20:48.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.787+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/166613328 shutdown_connections 2023-07-06T20:20:48.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.787+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/166613328 wait complete. 2023-07-06T20:20:48.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.787+0000 7fe7c61c7b80 1 -- start start 2023-07-06T20:20:48.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7c61c7b80 1 Processor -- start 2023-07-06T20:20:48.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7c61c7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc8912a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7c61c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8906870 0x55bcc8912f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7c61c7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8907390 0x55bcc8915f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7c61c7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc89164b0 con 0x55bcc8907390 2023-07-06T20:20:48.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7c61c7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc8916620 con 0x55bcc8905f20 2023-07-06T20:20:48.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7c61c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc89167e0 con 0x55bcc8906870 2023-07-06T20:20:48.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7c61c7b80 10 client.0 _ll_register_callbacks cb 0x7fe79c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:20:48.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7abfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc8912a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7ab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8906870 0x55bcc8912f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7abfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc8912a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:46116/0 (socket says 172.21.15.69:46116) 2023-07-06T20:20:48.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7ab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8906870 0x55bcc8912f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43350/0 (socket says 172.21.15.69:43350) 2023-07-06T20:20:48.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7abfff700 1 -- 172.21.15.69:0/3136181436 learned_addr learned my addr 172.21.15.69:0/3136181436 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:20:48.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.788+0000 7fe7b0d9c700 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8907390 0x55bcc8915f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.789+0000 7fe7abfff700 1 -- 172.21.15.69:0/3136181436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8906870 msgr2=0x55bcc8912f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:48.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.789+0000 7fe7abfff700 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8906870 0x55bcc8912f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.789+0000 7fe7abfff700 1 -- 172.21.15.69:0/3136181436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8907390 msgr2=0x55bcc8915f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:20:48.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.789+0000 7fe7abfff700 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8907390 0x55bcc8915f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.789+0000 7fe7abfff700 1 -- 172.21.15.69:0/3136181436 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ac1a0390 con 0x55bcc8905f20 2023-07-06T20:20:48.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.789+0000 7fe7ab7fe700 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8906870 0x55bcc8912f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:20:48.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.789+0000 7fe7abfff700 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc8912a10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00b360 tx=0x7fe79c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90c60d403209f48b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:20:48.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.789+0000 7fe78b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:20:48.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.789+0000 7fe78b7fe700 1 -- 172.21.15.69:0/3136181436 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c01d050 con 0x55bcc8905f20 2023-07-06T20:20:48.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.789+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/3136181436 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe79c01e040 con 0x55bcc8905f20 2023-07-06T20:20:48.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.789+0000 7fe78b7fe700 1 -- 172.21.15.69:0/3136181436 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fe79c000e40 con 0x55bcc8905f20 2023-07-06T20:20:48.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.790+0000 7fe78b7fe700 1 -- 172.21.15.69:0/3136181436 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe79c00f620 con 0x55bcc8905f20 2023-07-06T20:20:48.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.790+0000 7fe7c61c7b80 10 client.25481 fetch_fsmap learned FSMap version 27 2023-07-06T20:20:48.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.790+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/3136181436 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bcc866dc60 con 0x55bcc8905f20 2023-07-06T20:20:48.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.790+0000 7fe78b7fe700 1 -- 172.21.15.69:0/3136181436 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe79c01e040 con 0x55bcc8905f20 2023-07-06T20:20:48.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.790+0000 7fe78b7fe700 1 -- 172.21.15.69:0/3136181436 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe79c0236a0 con 0x55bcc8905f20 2023-07-06T20:20:48.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.790+0000 7fe7c61c7b80 10 client.25481 fetch_fsmap finished waiting for FSMap version 27 2023-07-06T20:20:48.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.790+0000 7fe7c61c7b80 10 client.25481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:20:48.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.790+0000 7fe7c61c7b80 20 client.25481 populate_metadata read hostname 'smithi069' 2023-07-06T20:20:48.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.790+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bcc896f950 0x55bcc8971d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:20:48.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.790+0000 7fe7c61c7b80 4 client.25481 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:20:48.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.790+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/3136181436 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bcc884feb0 con 0x55bcc896f950 2023-07-06T20:20:48.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.790+0000 7fe7ab7fe700 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bcc896f950 0x55bcc8971d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:20:48.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe7ab7fe700 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bcc896f950 0x55bcc8971d30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d5f688bb02660f server_cookie=41a429477a94cf19 in_seq=0 out_seq=0 2023-07-06T20:20:48.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe78b7fe700 10 client.25481 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:20:48.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe78b7fe700 1 -- 172.21.15.69:0/3136181436 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55bcc884feb0 con 0x55bcc896f950 2023-07-06T20:20:48.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe78b7fe700 10 client.25481 handle_command_reply: tid=0 2023-07-06T20:20:48.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe7c61c7b80 1 client.25481 shutdown 2023-07-06T20:20:48.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/3136181436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 msgr2=0x55bcc8912a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:48.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc8912a10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00b360 tx=0x7fe79c009310 comp rx=0 tx=0).stop 2023-07-06T20:20:48.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/3136181436 shutdown_connections 2023-07-06T20:20:48.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bcc896f950 0x55bcc8971d30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc8907390 0x55bcc8915f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcc8905f20 0x55bcc8912a10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe7c61c7b80 1 --2- 172.21.15.69:0/3136181436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bcc8906870 0x55bcc8912f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/3136181436 >> 172.21.15.69:0/3136181436 conn(0x55bcc88fb640 msgr2=0x55bcc8917670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:20:48.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe78b7fe700 0 client.25481 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:20:48.802 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.791+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/3136181436 shutdown_connections 2023-07-06T20:20:48.802 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7c61c7b80 1 -- 172.21.15.69:0/3136181436 wait complete. 2023-07-06T20:20:48.802 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7c61c7b80 20 client.25481 trim_cache size 0 max 16384 2023-07-06T20:20:48.802 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:20:48.802 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:20:48.802 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:20:48.803 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:20:48.803 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:20:48.803 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:20:48.803 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:20:48.803 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:20:48.803 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:20:48.803 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:20:48.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7b27ff700 1 -- 172.21.15.69:0/4221099928 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe7940a1150 msgr2=0x7fe7940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:48.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7b27ff700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe7940a1150 0x7fe7940a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009ec0 tx=0x7fe7a4009920 comp rx=0 tx=0).stop 2023-07-06T20:20:48.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7b27ff700 1 -- 172.21.15.69:0/4221099928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7ac074680 msgr2=0x7fe7ac198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:20:48.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7b27ff700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7ac074680 0x7fe7ac198320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000e970 tx=0x7fe7a000ec80 comp rx=0 tx=0).stop 2023-07-06T20:20:48.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7b27ff700 1 -- 172.21.15.69:0/4221099928 shutdown_connections 2023-07-06T20:20:48.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7b27ff700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fe7940a1150 0x7fe7940a3600 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7b27ff700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7ac107ae0 0x7fe7ac19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7b27ff700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac0730a0 0x7fe7ac197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7b27ff700 1 --2- 172.21.15.69:0/4221099928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7ac074680 0x7fe7ac198320 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:20:48.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7b27ff700 1 -- 172.21.15.69:0/4221099928 >> 172.21.15.69:0/4221099928 conn(0x7fe7ac0fb260 msgr2=0x7fe7ac109e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:20:48.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.792+0000 7fe7b27ff700 1 -- 172.21.15.69:0/4221099928 shutdown_connections 2023-07-06T20:20:48.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:20:48.793+0000 7fe7b27ff700 1 -- 172.21.15.69:0/4221099928 wait complete. 2023-07-06T20:20:48.806 INFO:teuthology.orchestra.run.smithi069.stderr:42900000 of 50000000 tuples (85%) done (elapsed 85.61 s, remaining 14.17 s) 2023-07-06T20:20:48.813 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:20:48.983 INFO:teuthology.orchestra.run.smithi069.stderr:43000000 of 50000000 tuples (86%) done (elapsed 85.80 s, remaining 13.97 s) 2023-07-06T20:20:49.179 INFO:teuthology.orchestra.run.smithi069.stderr:43100000 of 50000000 tuples (86%) done (elapsed 85.99 s, remaining 13.77 s) 2023-07-06T20:20:49.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:48 smithi069 ceph-mon[126399]: pgmap v372: 97 pgs: 97 active+clean; 6.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 130 MiB/s wr, 64 op/s 2023-07-06T20:20:49.308 INFO:teuthology.orchestra.run.smithi069.stderr:43200000 of 50000000 tuples (86%) done (elapsed 86.12 s, remaining 13.56 s) 2023-07-06T20:20:49.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:48 smithi156 ceph-mon[129280]: pgmap v372: 97 pgs: 97 active+clean; 6.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 130 MiB/s wr, 64 op/s 2023-07-06T20:20:49.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:48 smithi098 ceph-mon[128968]: pgmap v372: 97 pgs: 97 active+clean; 6.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 130 MiB/s wr, 64 op/s 2023-07-06T20:20:49.505 INFO:teuthology.orchestra.run.smithi069.stderr:43300000 of 50000000 tuples (86%) done (elapsed 86.32 s, remaining 13.36 s) 2023-07-06T20:20:49.683 INFO:teuthology.orchestra.run.smithi069.stderr:43400000 of 50000000 tuples (86%) done (elapsed 86.49 s, remaining 13.15 s) 2023-07-06T20:20:49.874 INFO:teuthology.orchestra.run.smithi069.stderr:43500000 of 50000000 tuples (87%) done (elapsed 86.69 s, remaining 12.95 s) 2023-07-06T20:20:50.034 INFO:teuthology.orchestra.run.smithi069.stderr:43600000 of 50000000 tuples (87%) done (elapsed 86.85 s, remaining 12.75 s) 2023-07-06T20:20:50.191 INFO:teuthology.orchestra.run.smithi069.stderr:43700000 of 50000000 tuples (87%) done (elapsed 87.00 s, remaining 12.54 s) 2023-07-06T20:20:50.394 INFO:teuthology.orchestra.run.smithi069.stderr:43800000 of 50000000 tuples (87%) done (elapsed 87.21 s, remaining 12.34 s) 2023-07-06T20:20:50.574 INFO:teuthology.orchestra.run.smithi069.stderr:43900000 of 50000000 tuples (87%) done (elapsed 87.39 s, remaining 12.14 s) 2023-07-06T20:20:50.753 INFO:teuthology.orchestra.run.smithi069.stderr:44000000 of 50000000 tuples (88%) done (elapsed 87.57 s, remaining 11.94 s) 2023-07-06T20:20:50.866 INFO:teuthology.orchestra.run.smithi069.stderr:44100000 of 50000000 tuples (88%) done (elapsed 87.68 s, remaining 11.73 s) 2023-07-06T20:20:51.070 INFO:teuthology.orchestra.run.smithi069.stderr:44200000 of 50000000 tuples (88%) done (elapsed 87.88 s, remaining 11.53 s) 2023-07-06T20:20:51.248 INFO:teuthology.orchestra.run.smithi069.stderr:44300000 of 50000000 tuples (88%) done (elapsed 88.06 s, remaining 11.33 s) 2023-07-06T20:20:51.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:50 smithi069 ceph-mon[126399]: pgmap v373: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 131 MiB/s wr, 62 op/s 2023-07-06T20:20:51.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:50 smithi156 ceph-mon[129280]: pgmap v373: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 131 MiB/s wr, 62 op/s 2023-07-06T20:20:51.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:50 smithi098 ceph-mon[128968]: pgmap v373: 97 pgs: 97 active+clean; 6.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 131 MiB/s wr, 62 op/s 2023-07-06T20:20:51.387 INFO:teuthology.orchestra.run.smithi069.stderr:44400000 of 50000000 tuples (88%) done (elapsed 88.20 s, remaining 11.12 s) 2023-07-06T20:20:51.549 INFO:teuthology.orchestra.run.smithi069.stderr:44500000 of 50000000 tuples (89%) done (elapsed 88.36 s, remaining 10.92 s) 2023-07-06T20:20:51.742 INFO:teuthology.orchestra.run.smithi069.stderr:44600000 of 50000000 tuples (89%) done (elapsed 88.55 s, remaining 10.72 s) 2023-07-06T20:20:51.879 INFO:teuthology.orchestra.run.smithi069.stderr:44700000 of 50000000 tuples (89%) done (elapsed 88.69 s, remaining 10.52 s) 2023-07-06T20:20:52.073 INFO:teuthology.orchestra.run.smithi069.stderr:44800000 of 50000000 tuples (89%) done (elapsed 88.88 s, remaining 10.32 s) 2023-07-06T20:20:52.269 INFO:teuthology.orchestra.run.smithi069.stderr:44900000 of 50000000 tuples (89%) done (elapsed 89.08 s, remaining 10.12 s) 2023-07-06T20:20:52.953 INFO:teuthology.orchestra.run.smithi069.stderr:45000000 of 50000000 tuples (90%) done (elapsed 89.76 s, remaining 9.97 s) 2023-07-06T20:20:53.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:52 smithi069 ceph-mon[126399]: pgmap v374: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 134 MiB/s wr, 64 op/s 2023-07-06T20:20:53.275 INFO:teuthology.orchestra.run.smithi069.stderr:45100000 of 50000000 tuples (90%) done (elapsed 90.09 s, remaining 9.79 s) 2023-07-06T20:20:53.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:52 smithi156 ceph-mon[129280]: pgmap v374: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 134 MiB/s wr, 64 op/s 2023-07-06T20:20:53.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:52 smithi098 ceph-mon[128968]: pgmap v374: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 134 MiB/s wr, 64 op/s 2023-07-06T20:20:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:45200000 of 50000000 tuples (90%) done (elapsed 90.23 s, remaining 9.58 s) 2023-07-06T20:20:53.606 INFO:teuthology.orchestra.run.smithi069.stderr:45300000 of 50000000 tuples (90%) done (elapsed 90.42 s, remaining 9.38 s) 2023-07-06T20:20:53.765 INFO:teuthology.orchestra.run.smithi069.stderr:45400000 of 50000000 tuples (90%) done (elapsed 90.58 s, remaining 9.18 s) 2023-07-06T20:20:53.910 INFO:teuthology.orchestra.run.smithi069.stderr:45500000 of 50000000 tuples (91%) done (elapsed 90.72 s, remaining 8.97 s) 2023-07-06T20:20:54.115 INFO:teuthology.orchestra.run.smithi069.stderr:45600000 of 50000000 tuples (91%) done (elapsed 90.93 s, remaining 8.77 s) 2023-07-06T20:20:54.265 INFO:teuthology.orchestra.run.smithi069.stderr:45700000 of 50000000 tuples (91%) done (elapsed 91.08 s, remaining 8.57 s) 2023-07-06T20:20:54.450 INFO:teuthology.orchestra.run.smithi069.stderr:45800000 of 50000000 tuples (91%) done (elapsed 91.26 s, remaining 8.37 s) 2023-07-06T20:20:54.639 INFO:teuthology.orchestra.run.smithi069.stderr:45900000 of 50000000 tuples (91%) done (elapsed 91.45 s, remaining 8.17 s) 2023-07-06T20:20:54.746 INFO:teuthology.orchestra.run.smithi069.stderr:46000000 of 50000000 tuples (92%) done (elapsed 91.56 s, remaining 7.96 s) 2023-07-06T20:20:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:46100000 of 50000000 tuples (92%) done (elapsed 91.73 s, remaining 7.76 s) 2023-07-06T20:20:55.093 INFO:teuthology.orchestra.run.smithi069.stderr:46200000 of 50000000 tuples (92%) done (elapsed 91.91 s, remaining 7.56 s) 2023-07-06T20:20:55.237 INFO:teuthology.orchestra.run.smithi069.stderr:46300000 of 50000000 tuples (92%) done (elapsed 92.05 s, remaining 7.36 s) 2023-07-06T20:20:55.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:54 smithi069 ceph-mon[126399]: pgmap v375: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 123 MiB/s wr, 62 op/s 2023-07-06T20:20:55.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:54 smithi156 ceph-mon[129280]: pgmap v375: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 123 MiB/s wr, 62 op/s 2023-07-06T20:20:55.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:54 smithi098 ceph-mon[128968]: pgmap v375: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 123 MiB/s wr, 62 op/s 2023-07-06T20:20:55.439 INFO:teuthology.orchestra.run.smithi069.stderr:46400000 of 50000000 tuples (92%) done (elapsed 92.25 s, remaining 7.16 s) 2023-07-06T20:20:55.605 INFO:teuthology.orchestra.run.smithi069.stderr:46500000 of 50000000 tuples (93%) done (elapsed 92.42 s, remaining 6.96 s) 2023-07-06T20:20:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:46600000 of 50000000 tuples (93%) done (elapsed 92.62 s, remaining 6.76 s) 2023-07-06T20:20:56.004 INFO:teuthology.orchestra.run.smithi069.stderr:46700000 of 50000000 tuples (93%) done (elapsed 92.82 s, remaining 6.56 s) 2023-07-06T20:20:56.116 INFO:teuthology.orchestra.run.smithi069.stderr:46800000 of 50000000 tuples (93%) done (elapsed 92.93 s, remaining 6.35 s) 2023-07-06T20:20:56.282 INFO:teuthology.orchestra.run.smithi069.stderr:46900000 of 50000000 tuples (93%) done (elapsed 93.09 s, remaining 6.15 s) 2023-07-06T20:20:56.486 INFO:teuthology.orchestra.run.smithi069.stderr:47000000 of 50000000 tuples (94%) done (elapsed 93.30 s, remaining 5.96 s) 2023-07-06T20:20:56.668 INFO:teuthology.orchestra.run.smithi069.stderr:47100000 of 50000000 tuples (94%) done (elapsed 93.48 s, remaining 5.76 s) 2023-07-06T20:20:56.813 INFO:teuthology.orchestra.run.smithi069.stderr:47200000 of 50000000 tuples (94%) done (elapsed 93.62 s, remaining 5.55 s) 2023-07-06T20:20:56.988 INFO:teuthology.orchestra.run.smithi069.stderr:47300000 of 50000000 tuples (94%) done (elapsed 93.80 s, remaining 5.35 s) 2023-07-06T20:20:57.173 INFO:teuthology.orchestra.run.smithi069.stderr:47400000 of 50000000 tuples (94%) done (elapsed 93.98 s, remaining 5.16 s) 2023-07-06T20:20:57.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:56 smithi069 ceph-mon[126399]: pgmap v376: 97 pgs: 97 active+clean; 6.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 118 MiB/s wr, 52 op/s 2023-07-06T20:20:57.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:56 smithi156 ceph-mon[129280]: pgmap v376: 97 pgs: 97 active+clean; 6.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 118 MiB/s wr, 52 op/s 2023-07-06T20:20:57.374 INFO:teuthology.orchestra.run.smithi069.stderr:47500000 of 50000000 tuples (95%) done (elapsed 94.19 s, remaining 4.96 s) 2023-07-06T20:20:57.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:56 smithi098 ceph-mon[128968]: pgmap v376: 97 pgs: 97 active+clean; 6.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 118 MiB/s wr, 52 op/s 2023-07-06T20:20:57.546 INFO:teuthology.orchestra.run.smithi069.stderr:47600000 of 50000000 tuples (95%) done (elapsed 94.36 s, remaining 4.76 s) 2023-07-06T20:20:57.660 INFO:teuthology.orchestra.run.smithi069.stderr:47700000 of 50000000 tuples (95%) done (elapsed 94.47 s, remaining 4.56 s) 2023-07-06T20:20:57.826 INFO:teuthology.orchestra.run.smithi069.stderr:47800000 of 50000000 tuples (95%) done (elapsed 94.64 s, remaining 4.36 s) 2023-07-06T20:20:58.008 INFO:teuthology.orchestra.run.smithi069.stderr:47900000 of 50000000 tuples (95%) done (elapsed 94.82 s, remaining 4.16 s) 2023-07-06T20:20:58.156 INFO:teuthology.orchestra.run.smithi069.stderr:48000000 of 50000000 tuples (96%) done (elapsed 94.97 s, remaining 3.96 s) 2023-07-06T20:20:58.357 INFO:teuthology.orchestra.run.smithi069.stderr:48100000 of 50000000 tuples (96%) done (elapsed 95.17 s, remaining 3.76 s) 2023-07-06T20:20:58.536 INFO:teuthology.orchestra.run.smithi069.stderr:48200000 of 50000000 tuples (96%) done (elapsed 95.35 s, remaining 3.56 s) 2023-07-06T20:20:58.667 INFO:teuthology.orchestra.run.smithi069.stderr:48300000 of 50000000 tuples (96%) done (elapsed 95.48 s, remaining 3.36 s) 2023-07-06T20:20:58.860 INFO:teuthology.orchestra.run.smithi069.stderr:48400000 of 50000000 tuples (96%) done (elapsed 95.67 s, remaining 3.16 s) 2023-07-06T20:20:59.057 INFO:teuthology.orchestra.run.smithi069.stderr:48500000 of 50000000 tuples (97%) done (elapsed 95.87 s, remaining 2.97 s) 2023-07-06T20:20:59.185 INFO:teuthology.orchestra.run.smithi069.stderr:48600000 of 50000000 tuples (97%) done (elapsed 96.00 s, remaining 2.77 s) 2023-07-06T20:20:59.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:20:58 smithi069 ceph-mon[126399]: pgmap v377: 97 pgs: 97 active+clean; 6.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 134 MiB/s wr, 64 op/s 2023-07-06T20:20:59.346 INFO:teuthology.orchestra.run.smithi069.stderr:48700000 of 50000000 tuples (97%) done (elapsed 96.16 s, remaining 2.57 s) 2023-07-06T20:20:59.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:20:58 smithi156 ceph-mon[129280]: pgmap v377: 97 pgs: 97 active+clean; 6.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 134 MiB/s wr, 64 op/s 2023-07-06T20:20:59.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:20:58 smithi098 ceph-mon[128968]: pgmap v377: 97 pgs: 97 active+clean; 6.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 134 MiB/s wr, 64 op/s 2023-07-06T20:20:59.480 INFO:teuthology.orchestra.run.smithi069.stderr:48800000 of 50000000 tuples (97%) done (elapsed 96.29 s, remaining 2.37 s) 2023-07-06T20:21:00.084 INFO:teuthology.orchestra.run.smithi069.stderr:48900000 of 50000000 tuples (97%) done (elapsed 96.89 s, remaining 2.18 s) 2023-07-06T20:21:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:49000000 of 50000000 tuples (98%) done (elapsed 97.23 s, remaining 1.98 s) 2023-07-06T20:21:00.690 INFO:teuthology.orchestra.run.smithi069.stderr:49100000 of 50000000 tuples (98%) done (elapsed 97.50 s, remaining 1.79 s) 2023-07-06T20:21:00.883 INFO:teuthology.orchestra.run.smithi069.stderr:49200000 of 50000000 tuples (98%) done (elapsed 97.70 s, remaining 1.59 s) 2023-07-06T20:21:01.067 INFO:teuthology.orchestra.run.smithi069.stderr:49300000 of 50000000 tuples (98%) done (elapsed 97.88 s, remaining 1.39 s) 2023-07-06T20:21:01.205 INFO:teuthology.orchestra.run.smithi069.stderr:49400000 of 50000000 tuples (98%) done (elapsed 98.02 s, remaining 1.19 s) 2023-07-06T20:21:01.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:00 smithi069 ceph-mon[126399]: pgmap v378: 97 pgs: 97 active+clean; 6.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 113 MiB/s wr, 49 op/s 2023-07-06T20:21:01.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:00 smithi156 ceph-mon[129280]: pgmap v378: 97 pgs: 97 active+clean; 6.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 113 MiB/s wr, 49 op/s 2023-07-06T20:21:01.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:00 smithi098 ceph-mon[128968]: pgmap v378: 97 pgs: 97 active+clean; 6.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 113 MiB/s wr, 49 op/s 2023-07-06T20:21:01.379 INFO:teuthology.orchestra.run.smithi069.stderr:49500000 of 50000000 tuples (99%) done (elapsed 98.19 s, remaining 0.99 s) 2023-07-06T20:21:01.572 INFO:teuthology.orchestra.run.smithi069.stderr:49600000 of 50000000 tuples (99%) done (elapsed 98.38 s, remaining 0.79 s) 2023-07-06T20:21:01.723 INFO:teuthology.orchestra.run.smithi069.stderr:49700000 of 50000000 tuples (99%) done (elapsed 98.53 s, remaining 0.59 s) 2023-07-06T20:21:01.877 INFO:teuthology.orchestra.run.smithi069.stderr:49800000 of 50000000 tuples (99%) done (elapsed 98.69 s, remaining 0.40 s) 2023-07-06T20:21:02.084 INFO:teuthology.orchestra.run.smithi069.stderr:49900000 of 50000000 tuples (99%) done (elapsed 98.90 s, remaining 0.20 s) 2023-07-06T20:21:02.230 INFO:teuthology.orchestra.run.smithi069.stderr:50000000 of 50000000 tuples (100%) done (elapsed 99.04 s, remaining 0.00 s) 2023-07-06T20:21:02.297 INFO:teuthology.orchestra.run.smithi069.stderr:vacuum... 2023-07-06T20:21:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:02 smithi069 ceph-mon[126399]: pgmap v379: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 127 MiB/s wr, 53 op/s 2023-07-06T20:21:03.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:02 smithi156 ceph-mon[129280]: pgmap v379: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 127 MiB/s wr, 53 op/s 2023-07-06T20:21:03.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:02 smithi098 ceph-mon[128968]: pgmap v379: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 127 MiB/s wr, 53 op/s 2023-07-06T20:21:05.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:04 smithi069 ceph-mon[126399]: pgmap v380: 97 pgs: 97 active+clean; 7.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 135 MiB/s wr, 67 op/s 2023-07-06T20:21:05.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:04 smithi156 ceph-mon[129280]: pgmap v380: 97 pgs: 97 active+clean; 7.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 135 MiB/s wr, 67 op/s 2023-07-06T20:21:05.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:04 smithi098 ceph-mon[128968]: pgmap v380: 97 pgs: 97 active+clean; 7.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 135 MiB/s wr, 67 op/s 2023-07-06T20:21:07.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:06 smithi069 ceph-mon[126399]: pgmap v381: 97 pgs: 97 active+clean; 7.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 129 MiB/s wr, 65 op/s 2023-07-06T20:21:07.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:06 smithi156 ceph-mon[129280]: pgmap v381: 97 pgs: 97 active+clean; 7.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 129 MiB/s wr, 65 op/s 2023-07-06T20:21:07.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:06 smithi098 ceph-mon[128968]: pgmap v381: 97 pgs: 97 active+clean; 7.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 129 MiB/s wr, 65 op/s 2023-07-06T20:21:09.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:09 smithi156 ceph-mon[129280]: pgmap v382: 97 pgs: 97 active+clean; 7.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 108 MiB/s wr, 62 op/s 2023-07-06T20:21:09.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:09 smithi098 ceph-mon[128968]: pgmap v382: 97 pgs: 97 active+clean; 7.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 108 MiB/s wr, 62 op/s 2023-07-06T20:21:09.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:09 smithi069 ceph-mon[126399]: pgmap v382: 97 pgs: 97 active+clean; 7.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 108 MiB/s wr, 62 op/s 2023-07-06T20:21:11.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:11 smithi069 ceph-mon[126399]: pgmap v383: 97 pgs: 97 active+clean; 7.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 91 MiB/s wr, 49 op/s 2023-07-06T20:21:11.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:11 smithi156 ceph-mon[129280]: pgmap v383: 97 pgs: 97 active+clean; 7.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 91 MiB/s wr, 49 op/s 2023-07-06T20:21:11.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:11 smithi098 ceph-mon[128968]: pgmap v383: 97 pgs: 97 active+clean; 7.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 91 MiB/s wr, 49 op/s 2023-07-06T20:21:13.517 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:13 smithi069 ceph-mon[126399]: pgmap v384: 97 pgs: 97 active+clean; 7.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 125 MiB/s wr, 59 op/s 2023-07-06T20:21:13.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:13 smithi156 ceph-mon[129280]: pgmap v384: 97 pgs: 97 active+clean; 7.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 125 MiB/s wr, 59 op/s 2023-07-06T20:21:13.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:13 smithi098 ceph-mon[128968]: pgmap v384: 97 pgs: 97 active+clean; 7.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 125 MiB/s wr, 59 op/s 2023-07-06T20:21:15.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:15 smithi069 ceph-mon[126399]: pgmap v385: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 160 MiB/s wr, 69 op/s 2023-07-06T20:21:15.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:15 smithi156 ceph-mon[129280]: pgmap v385: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 160 MiB/s wr, 69 op/s 2023-07-06T20:21:15.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:15 smithi098 ceph-mon[128968]: pgmap v385: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 160 MiB/s wr, 69 op/s 2023-07-06T20:21:16.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:16 smithi156 ceph-mon[129280]: pgmap v386: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 190 MiB/s wr, 64 op/s 2023-07-06T20:21:16.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:16 smithi098 ceph-mon[128968]: pgmap v386: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 190 MiB/s wr, 64 op/s 2023-07-06T20:21:16.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:16 smithi069 ceph-mon[126399]: pgmap v386: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 190 MiB/s wr, 64 op/s 2023-07-06T20:21:17.303 INFO:teuthology.orchestra.run.smithi069.stderr:set primary keys... 2023-07-06T20:21:18.814 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:21:19.015 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:18 smithi069 ceph-mon[126399]: pgmap v387: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 256 MiB/s wr, 83 op/s 2023-07-06T20:21:19.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:18 smithi156 ceph-mon[129280]: pgmap v387: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 256 MiB/s wr, 83 op/s 2023-07-06T20:21:19.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:18 smithi098 ceph-mon[128968]: pgmap v387: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 256 MiB/s wr, 83 op/s 2023-07-06T20:21:19.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.635+0000 7f15fadff700 1 -- 172.21.15.69:0/4039561893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15f4071960 msgr2=0x7f15f410b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:19.639 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.635+0000 7f15fadff700 1 --2- 172.21.15.69:0/4039561893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15f4071960 0x7f15f410b900 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009990 tx=0x7f15e0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.648 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.646+0000 7f15fadff700 1 -- 172.21.15.69:0/4039561893 shutdown_connections 2023-07-06T20:21:19.648 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.646+0000 7f15fadff700 1 --2- 172.21.15.69:0/4039561893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15f410be90 0x7f15f410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.649 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.646+0000 7f15fadff700 1 --2- 172.21.15.69:0/4039561893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4071010 0x7f15f4071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.649 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.646+0000 7f15fadff700 1 --2- 172.21.15.69:0/4039561893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15f4071960 0x7f15f410b900 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.649 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.646+0000 7f15fadff700 1 -- 172.21.15.69:0/4039561893 >> 172.21.15.69:0/4039561893 conn(0x7f15f406cbc0 msgr2=0x7f15f406eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:21:19.653 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.648+0000 7f15fadff700 1 -- 172.21.15.69:0/4039561893 shutdown_connections 2023-07-06T20:21:19.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.651+0000 7f15fadff700 1 -- 172.21.15.69:0/4039561893 wait complete. 2023-07-06T20:21:19.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.653+0000 7f15fadff700 1 Processor -- start 2023-07-06T20:21:19.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.657+0000 7f15fadff700 1 -- start start 2023-07-06T20:21:19.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.658+0000 7f15fadff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15f4071010 0x7f15f4113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.658+0000 7f15fadff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4071960 0x7f15f41140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.658+0000 7f15fadff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15f410be90 0x7f15f41b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.658+0000 7f15fadff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f41171c0 con 0x7f15f4071010 2023-07-06T20:21:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.658+0000 7f15fadff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4117330 con 0x7f15f410be90 2023-07-06T20:21:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.658+0000 7f15fadff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f41b4f10 con 0x7f15f4071960 2023-07-06T20:21:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.658+0000 7f15f939c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15f410be90 0x7f15f41b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.658+0000 7f15f939c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15f410be90 0x7f15f41b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:55806/0 (socket says 172.21.15.69:55806) 2023-07-06T20:21:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.658+0000 7f15f939c700 1 -- 172.21.15.69:0/1053944187 learned_addr learned my addr 172.21.15.69:0/1053944187 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:21:19.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.659+0000 7f15f939c700 1 -- 172.21.15.69:0/1053944187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4071960 msgr2=0x7f15f41140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:19.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.659+0000 7f15f939c700 1 --2- 172.21.15.69:0/1053944187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4071960 0x7f15f41140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.659+0000 7f15f939c700 1 -- 172.21.15.69:0/1053944187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15f4071010 msgr2=0x7f15f4113ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T20:21:19.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.659+0000 7f15f939c700 1 --2- 172.21.15.69:0/1053944187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15f4071010 0x7f15f4113ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.659+0000 7f15f939c700 1 -- 172.21.15.69:0/1053944187 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e00095f0 con 0x7f15f410be90 2023-07-06T20:21:19.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.660+0000 7f15f939c700 1 --2- 172.21.15.69:0/1053944187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15f410be90 0x7f15f41b49d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00e940 tx=0x7f15ec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:21:19.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.660+0000 7f15f1ffb700 1 -- 172.21.15.69:0/1053944187 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0097b0 con 0x7f15f410be90 2023-07-06T20:21:19.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.660+0000 7f15f1ffb700 1 -- 172.21.15.69:0/1053944187 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15ec004830 con 0x7f15f410be90 2023-07-06T20:21:19.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.660+0000 7f15fadff700 1 -- 172.21.15.69:0/1053944187 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f4072870 con 0x7f15f410be90 2023-07-06T20:21:19.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.660+0000 7f15f1ffb700 1 -- 172.21.15.69:0/1053944187 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0055c0 con 0x7f15f410be90 2023-07-06T20:21:19.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.660+0000 7f15fadff700 1 -- 172.21.15.69:0/1053944187 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f4072d80 con 0x7f15f410be90 2023-07-06T20:21:19.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.667+0000 7f160e6a5b80 1 Processor -- start 2023-07-06T20:21:19.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.667+0000 7f15f1ffb700 1 -- 172.21.15.69:0/1053944187 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f15ec004c80 con 0x7f15f410be90 2023-07-06T20:21:19.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.670+0000 7f15f1ffb700 1 --2- 172.21.15.69:0/1053944187 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f15e40c32b0 0x7f15e40c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.670+0000 7f15f1ffb700 1 -- 172.21.15.69:0/1053944187 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f15ec014070 con 0x7f15f410be90 2023-07-06T20:21:19.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.672+0000 7f160e6a5b80 1 -- start start 2023-07-06T20:21:19.679 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.672+0000 7f160e6a5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5df20 0x55baafc5e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.672+0000 7f160e6a5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5e870 0x55baafc5ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.672+0000 7f160e6a5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 0x55baafc5f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.672+0000 7f160e6a5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baafb62cb0 con 0x55baafc5df20 2023-07-06T20:21:19.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.672+0000 7f160e6a5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baafbb5a40 con 0x55baafc5f390 2023-07-06T20:21:19.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.672+0000 7f160e6a5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baafbaeed0 con 0x55baafc5e870 2023-07-06T20:21:19.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.674+0000 7f15f939c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 0x55baafc5f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:19.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.674+0000 7f15f939c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 0x55baafc5f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:55816/0 (socket says 172.21.15.69:55816) 2023-07-06T20:21:19.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.674+0000 7f15f939c700 1 -- 172.21.15.69:0/3927105868 learned_addr learned my addr 172.21.15.69:0/3927105868 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:21:19.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.675+0000 7f15f939c700 1 -- 172.21.15.69:0/3927105868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5e870 msgr2=0x55baafc5ee00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:21:19.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.675+0000 7f15f939c700 1 --2- 172.21.15.69:0/3927105868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5e870 0x55baafc5ee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.675+0000 7f15f939c700 1 -- 172.21.15.69:0/3927105868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5df20 msgr2=0x55baafc5e330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:21:19.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.675+0000 7f15f939c700 1 --2- 172.21.15.69:0/3927105868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5df20 0x55baafc5e330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.675+0000 7f15f939c700 1 -- 172.21.15.69:0/3927105868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baaf9dcde0 con 0x55baafc5f390 2023-07-06T20:21:19.683 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.675+0000 7f15f3fff700 1 --2- 172.21.15.69:0/1053944187 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f15e40c32b0 0x7f15e40c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:19.685 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.675+0000 7f15f939c700 1 --2- 172.21.15.69:0/3927105868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 0x55baafc5f830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f15ec099f50 tx=0x7f15ec099920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40fdb94f9f2c88be server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:21:19.685 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.676+0000 7f15deffd700 1 -- 172.21.15.69:0/3927105868 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec099fc0 con 0x55baafc5f390 2023-07-06T20:21:19.685 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.676+0000 7f15deffd700 1 -- 172.21.15.69:0/3927105868 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15ec09dff0 con 0x55baafc5f390 2023-07-06T20:21:19.685 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.676+0000 7f15deffd700 1 -- 172.21.15.69:0/3927105868 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0996b0 con 0x55baafc5f390 2023-07-06T20:21:19.685 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.677+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3927105868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 msgr2=0x55baafc5f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:19.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.677+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/3927105868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 0x55baafc5f830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f15ec099f50 tx=0x7f15ec099920 comp rx=0 tx=0).stop 2023-07-06T20:21:19.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.677+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3927105868 shutdown_connections 2023-07-06T20:21:19.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.677+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/3927105868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 0x55baafc5f830 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.677+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/3927105868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5df20 0x55baafc5e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.677+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/3927105868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5e870 0x55baafc5ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.677+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3927105868 >> 172.21.15.69:0/3927105868 conn(0x55baafc53640 msgr2=0x55baafc589f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:21:19.687 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.682+0000 7f15f3fff700 1 --2- 172.21.15.69:0/1053944187 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f15e40c32b0 0x7f15e40c5760 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f15e000b680 tx=0x7f15e0005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:21:19.687 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.683+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3927105868 shutdown_connections 2023-07-06T20:21:19.688 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.686+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3927105868 wait complete. 2023-07-06T20:21:19.689 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.687+0000 7f160e6a5b80 1 -- start start 2023-07-06T20:21:19.689 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.687+0000 7f160e6a5b80 1 Processor -- start 2023-07-06T20:21:19.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.692+0000 7f160e6a5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5df20 0x55baafc898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.692+0000 7f160e6a5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5e870 0x55baafc89de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.692+0000 7f160e6a5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5f390 0x55baafc90e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.692+0000 7f160e6a5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baafb4eac0 con 0x55baafc5e870 2023-07-06T20:21:19.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.692+0000 7f160e6a5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baafb4e7c0 con 0x55baafc5df20 2023-07-06T20:21:19.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.692+0000 7f160e6a5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baafa32920 con 0x55baafc5f390 2023-07-06T20:21:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.692+0000 7f160e6a5b80 10 client.0 _ll_register_callbacks cb 0x55baafb4c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:21:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.694+0000 7f15f939c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5f390 0x55baafc90e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.694+0000 7f15f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5e870 0x55baafc89de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.694+0000 7f15f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5e870 0x55baafc89de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45390/0 (socket says 172.21.15.69:45390) 2023-07-06T20:21:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.694+0000 7f15f3fff700 1 -- 172.21.15.69:0/3472227151 learned_addr learned my addr 172.21.15.69:0/3472227151 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:21:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.694+0000 7f15f3fff700 1 -- 172.21.15.69:0/3472227151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5f390 msgr2=0x55baafc90e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:21:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.694+0000 7f15f3fff700 1 --2- 172.21.15.69:0/3472227151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5f390 0x55baafc90e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.694+0000 7f15f3fff700 1 -- 172.21.15.69:0/3472227151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5df20 msgr2=0x55baafc898a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:21:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.694+0000 7f15f3fff700 1 --2- 172.21.15.69:0/3472227151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5df20 0x55baafc898a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.694+0000 7f15f3fff700 1 -- 172.21.15.69:0/3472227151 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f4072d80 con 0x55baafc5e870 2023-07-06T20:21:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.695+0000 7f15f3fff700 1 --2- 172.21.15.69:0/3472227151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5e870 0x55baafc89de0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f15e0015000 tx=0x7f15e0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e9d7b0fcfaa69e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:21:19.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.697+0000 7f15d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:21:19.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.697+0000 7f15d3fff700 1 -- 172.21.15.69:0/3472227151 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e001d070 con 0x55baafc5e870 2023-07-06T20:21:19.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.697+0000 7f15d3fff700 1 -- 172.21.15.69:0/3472227151 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f15e000fde0 con 0x55baafc5e870 2023-07-06T20:21:19.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.698+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3472227151 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55baafb084f0 con 0x55baafc5e870 2023-07-06T20:21:19.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.699+0000 7f15d3fff700 1 -- 172.21.15.69:0/3472227151 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e0025450 con 0x55baafc5e870 2023-07-06T20:21:19.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.703+0000 7f160e6a5b80 10 client.15429 fetch_fsmap learned FSMap version 27 2023-07-06T20:21:19.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.703+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3472227151 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55baaf9c5c60 con 0x55baafc5e870 2023-07-06T20:21:19.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.705+0000 7f15d3fff700 1 -- 172.21.15.69:0/3472227151 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55baafb084f0 con 0x55baafc5e870 2023-07-06T20:21:19.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.705+0000 7f15d3fff700 1 -- 172.21.15.69:0/3472227151 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f15e0026770 con 0x55baafc5e870 2023-07-06T20:21:19.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.707+0000 7f160e6a5b80 10 client.15429 fetch_fsmap finished waiting for FSMap version 27 2023-07-06T20:21:19.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.707+0000 7f160e6a5b80 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:21:19.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.707+0000 7f160e6a5b80 20 client.15429 populate_metadata read hostname 'smithi069' 2023-07-06T20:21:19.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.707+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/3472227151 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55baafc956d0 0x55baafc97ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.707+0000 7f160e6a5b80 4 client.15429 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:21:19.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.707+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3472227151 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55baafc2cd60 con 0x55baafc956d0 2023-07-06T20:21:19.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.707+0000 7f15f8b9b700 1 --2- 172.21.15.69:0/3472227151 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55baafc956d0 0x55baafc97ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:19.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.710+0000 7f15d3fff700 10 client.15429 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:21:19.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.710+0000 7f15f8b9b700 1 --2- 172.21.15.69:0/3472227151 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55baafc956d0 0x55baafc97ae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0fa7549d295caf server_cookie=b508319685c11fe3 in_seq=0 out_seq=0 2023-07-06T20:21:19.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.736+0000 7f15d3fff700 1 -- 172.21.15.69:0/3472227151 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55baafc2cd60 con 0x55baafc956d0 2023-07-06T20:21:19.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.736+0000 7f15d3fff700 10 client.15429 handle_command_reply: tid=0 2023-07-06T20:21:19.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.736+0000 7f160e6a5b80 1 client.15429 shutdown 2023-07-06T20:21:19.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.737+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3472227151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5e870 msgr2=0x55baafc89de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:19.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.737+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/3472227151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5e870 0x55baafc89de0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f15e0015000 tx=0x7f15e0009310 comp rx=0 tx=0).stop 2023-07-06T20:21:19.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.744+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3472227151 shutdown_connections 2023-07-06T20:21:19.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.744+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/3472227151 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55baafc956d0 0x55baafc97ae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.744+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/3472227151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5f390 0x55baafc90e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.744+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/3472227151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5df20 0x55baafc898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.744+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/3472227151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5e870 0x55baafc89de0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.744+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3472227151 >> 172.21.15.69:0/3472227151 conn(0x55baafc53640 msgr2=0x55baafc580b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:21:19.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.744+0000 7f15d3fff700 0 client.15429 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:21:19.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.756+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3472227151 shutdown_connections 2023-07-06T20:21:19.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.756+0000 7f160e6a5b80 1 -- 172.21.15.69:0/3472227151 wait complete. 2023-07-06T20:21:19.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.757+0000 7f160e6a5b80 20 client.15429 trim_cache size 0 max 16384 2023-07-06T20:21:19.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.795+0000 7f160e6a5b80 1 Processor -- start 2023-07-06T20:21:19.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.795+0000 7f160e6a5b80 1 -- start start 2023-07-06T20:21:19.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.796+0000 7f160e6a5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5df20 0x55baafc91650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.796+0000 7f160e6a5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5e870 0x55baafc90de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.796+0000 7f160e6a5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5f390 0x55baafc65780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.796+0000 7f160e6a5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baafc89470 con 0x55baafc5e870 2023-07-06T20:21:19.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.796+0000 7f160e6a5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baafc8a7a0 con 0x55baafc5df20 2023-07-06T20:21:19.802 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.796+0000 7f160e6a5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baafc53450 con 0x55baafc5f390 2023-07-06T20:21:19.802 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.796+0000 7f15f8b9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5df20 0x55baafc91650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:19.802 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.798+0000 7f15f8b9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5df20 0x55baafc91650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:55832/0 (socket says 172.21.15.69:55832) 2023-07-06T20:21:19.802 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.798+0000 7f15f8b9b700 1 -- 172.21.15.69:0/1834034240 learned_addr learned my addr 172.21.15.69:0/1834034240 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:21:19.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.799+0000 7f15f939c700 1 --2- 172.21.15.69:0/1834034240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5f390 0x55baafc65780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:19.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.800+0000 7f15f8b9b700 1 -- 172.21.15.69:0/1834034240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5f390 msgr2=0x55baafc65780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:21:19.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.800+0000 7f15f8b9b700 1 --2- 172.21.15.69:0/1834034240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5f390 0x55baafc65780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.800+0000 7f15f8b9b700 1 -- 172.21.15.69:0/1834034240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5e870 msgr2=0x55baafc90de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:21:19.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.800+0000 7f15f8b9b700 1 --2- 172.21.15.69:0/1834034240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5e870 0x55baafc90de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.800+0000 7f15f8b9b700 1 -- 172.21.15.69:0/1834034240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baafa03250 con 0x55baafc5df20 2023-07-06T20:21:19.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.802+0000 7f15f8b9b700 1 --2- 172.21.15.69:0/1834034240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5df20 0x55baafc91650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55baafc98380 tx=0x7f15e8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=226c6a6a6dcba73f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:21:19.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.804+0000 7f15df7fe700 1 -- 172.21.15.69:0/1834034240 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e80090d0 con 0x55baafc5df20 2023-07-06T20:21:19.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.805+0000 7f15df7fe700 1 -- 172.21.15.69:0/1834034240 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e800f040 con 0x55baafc5df20 2023-07-06T20:21:19.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.805+0000 7f15df7fe700 1 -- 172.21.15.69:0/1834034240 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8005420 con 0x55baafc5df20 2023-07-06T20:21:19.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.806+0000 7f160e6a5b80 1 -- 172.21.15.69:0/1834034240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5df20 msgr2=0x55baafc91650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:19.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.806+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/1834034240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5df20 0x55baafc91650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55baafc98380 tx=0x7f15e8009920 comp rx=0 tx=0).stop 2023-07-06T20:21:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.808+0000 7f160e6a5b80 1 -- 172.21.15.69:0/1834034240 shutdown_connections 2023-07-06T20:21:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.809+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/1834034240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5f390 0x55baafc65780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.809+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/1834034240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5df20 0x55baafc91650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.809+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/1834034240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5e870 0x55baafc90de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.809+0000 7f160e6a5b80 1 -- 172.21.15.69:0/1834034240 >> 172.21.15.69:0/1834034240 conn(0x55baafc53640 msgr2=0x55baafc885b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:21:19.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.810+0000 7f160e6a5b80 1 -- 172.21.15.69:0/1834034240 shutdown_connections 2023-07-06T20:21:19.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.811+0000 7f160e6a5b80 1 -- 172.21.15.69:0/1834034240 wait complete. 2023-07-06T20:21:19.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.811+0000 7f160e6a5b80 1 -- start start 2023-07-06T20:21:19.814 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.812+0000 7f160e6a5b80 1 Processor -- start 2023-07-06T20:21:19.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.814+0000 7f160e6a5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5df20 0x55baafc90a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.814+0000 7f160e6a5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5e870 0x55baafc8d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.819 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.814+0000 7f160e6a5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 0x55baafc8da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.814+0000 7f160e6a5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baafc8df40 con 0x55baafc5df20 2023-07-06T20:21:19.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.814+0000 7f160e6a5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baafc8e0b0 con 0x55baafc5f390 2023-07-06T20:21:19.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.814+0000 7f160e6a5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baafce9540 con 0x55baafc5e870 2023-07-06T20:21:19.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.814+0000 7f160e6a5b80 10 client.0 _ll_register_callbacks cb 0x7f15e000be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:21:19.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.814+0000 7f15f3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5e870 0x55baafc8d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:19.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.814+0000 7f15f3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5e870 0x55baafc8d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:50224/0 (socket says 172.21.15.69:50224) 2023-07-06T20:21:19.820 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.814+0000 7f15f3fff700 1 -- 172.21.15.69:0/2788706956 learned_addr learned my addr 172.21.15.69:0/2788706956 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:21:19.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.815+0000 7f15f939c700 1 --2- 172.21.15.69:0/2788706956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 0x55baafc8da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:19.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.815+0000 7f15f939c700 1 -- 172.21.15.69:0/2788706956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5e870 msgr2=0x55baafc8d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:21:19.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.815+0000 7f15f939c700 1 --2- 172.21.15.69:0/2788706956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5e870 0x55baafc8d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.815+0000 7f15f939c700 1 -- 172.21.15.69:0/2788706956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5df20 msgr2=0x55baafc90a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:21:19.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.815+0000 7f15f939c700 1 --2- 172.21.15.69:0/2788706956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5df20 0x55baafc90a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.815+0000 7f15f939c700 1 -- 172.21.15.69:0/2788706956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e8004b40 con 0x55baafc5f390 2023-07-06T20:21:19.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.816+0000 7f15f939c700 1 --2- 172.21.15.69:0/2788706956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 0x55baafc8da00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f15ec099cc0 tx=0x7f15ec0092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba47ffdab0bfd293 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:21:19.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.816+0000 7f15d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:21:19.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.816+0000 7f15d3fff700 1 -- 172.21.15.69:0/2788706956 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0b8400 con 0x55baafc5f390 2023-07-06T20:21:19.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.816+0000 7f15d3fff700 1 -- 172.21.15.69:0/2788706956 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f15ec0a2410 con 0x55baafc5f390 2023-07-06T20:21:19.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.816+0000 7f160e6a5b80 1 -- 172.21.15.69:0/2788706956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15ec09dff0 con 0x55baafc5f390 2023-07-06T20:21:19.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.817+0000 7f15d3fff700 1 -- 172.21.15.69:0/2788706956 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15ec0994b0 con 0x55baafc5f390 2023-07-06T20:21:19.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.817+0000 7f160e6a5b80 10 client.25496 fetch_fsmap learned FSMap version 27 2023-07-06T20:21:19.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.817+0000 7f160e6a5b80 1 -- 172.21.15.69:0/2788706956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55baaf9db2b0 con 0x55baafc5f390 2023-07-06T20:21:19.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.822+0000 7f15d3fff700 1 -- 172.21.15.69:0/2788706956 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15ec09dff0 con 0x55baafc5f390 2023-07-06T20:21:19.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.823+0000 7f15d3fff700 1 -- 172.21.15.69:0/2788706956 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f15ec0a1f80 con 0x55baafc5f390 2023-07-06T20:21:19.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.824+0000 7f160e6a5b80 10 client.25496 fetch_fsmap finished waiting for FSMap version 27 2023-07-06T20:21:19.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.824+0000 7f160e6a5b80 10 client.25496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:21:19.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.824+0000 7f160e6a5b80 20 client.25496 populate_metadata read hostname 'smithi069' 2023-07-06T20:21:19.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.824+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/2788706956 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55baafceab60 0x55baafcecf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:19.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.824+0000 7f160e6a5b80 4 client.25496 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:21:19.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.824+0000 7f160e6a5b80 1 -- 172.21.15.69:0/2788706956 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55baafba7eb0 con 0x55baafceab60 2023-07-06T20:21:19.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.827+0000 7f15f8b9b700 1 --2- 172.21.15.69:0/2788706956 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55baafceab60 0x55baafcecf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:19.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.844+0000 7f15f8b9b700 1 --2- 172.21.15.69:0/2788706956 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55baafceab60 0x55baafcecf70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91e5cdfac4d75cc1 server_cookie=a9f64c02aa19fbc0 in_seq=0 out_seq=0 2023-07-06T20:21:19.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.845+0000 7f15d3fff700 10 client.25496 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:21:19.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.845+0000 7f15d3fff700 1 -- 172.21.15.69:0/2788706956 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55baafba7eb0 con 0x55baafceab60 2023-07-06T20:21:19.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.845+0000 7f15d3fff700 10 client.25496 handle_command_reply: tid=0 2023-07-06T20:21:19.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.845+0000 7f160e6a5b80 1 client.25496 shutdown 2023-07-06T20:21:19.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.845+0000 7f160e6a5b80 1 -- 172.21.15.69:0/2788706956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 msgr2=0x55baafc8da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:19.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.845+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/2788706956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 0x55baafc8da00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f15ec099cc0 tx=0x7f15ec0092d0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.846+0000 7f160e6a5b80 1 -- 172.21.15.69:0/2788706956 shutdown_connections 2023-07-06T20:21:19.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.846+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/2788706956 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55baafceab60 0x55baafcecf70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.846+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/2788706956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55baafc5f390 0x55baafc8da00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.846+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/2788706956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55baafc5df20 0x55baafc90a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.846+0000 7f160e6a5b80 1 --2- 172.21.15.69:0/2788706956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55baafc5e870 0x55baafc8d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.846+0000 7f160e6a5b80 1 -- 172.21.15.69:0/2788706956 >> 172.21.15.69:0/2788706956 conn(0x55baafc53640 msgr2=0x55baafc91650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:21:19.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.846+0000 7f15d3fff700 0 client.25496 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:21:19.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.846+0000 7f160e6a5b80 1 -- 172.21.15.69:0/2788706956 shutdown_connections 2023-07-06T20:21:19.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.846+0000 7f160e6a5b80 1 -- 172.21.15.69:0/2788706956 wait complete. 2023-07-06T20:21:19.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.847+0000 7f160e6a5b80 20 client.25496 trim_cache size 0 max 16384 2023-07-06T20:21:19.850 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:21:19.850 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:21:19.850 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:21:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:21:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:21:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:21:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:21:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:21:19.851 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:21:19.851 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:21:19.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.848+0000 7f15fadff700 1 -- 172.21.15.69:0/1053944187 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f15e40c32b0 msgr2=0x7f15e40c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:19.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.848+0000 7f15fadff700 1 --2- 172.21.15.69:0/1053944187 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f15e40c32b0 0x7f15e40c5760 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f15e000b680 tx=0x7f15e0005be0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.848+0000 7f15fadff700 1 -- 172.21.15.69:0/1053944187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15f410be90 msgr2=0x7f15f41b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:19.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.848+0000 7f15fadff700 1 --2- 172.21.15.69:0/1053944187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15f410be90 0x7f15f41b49d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00e940 tx=0x7f15ec00ed00 comp rx=0 tx=0).stop 2023-07-06T20:21:19.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.848+0000 7f15fadff700 1 -- 172.21.15.69:0/1053944187 shutdown_connections 2023-07-06T20:21:19.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.848+0000 7f15fadff700 1 --2- 172.21.15.69:0/1053944187 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f15e40c32b0 0x7f15e40c5760 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.848+0000 7f15fadff700 1 --2- 172.21.15.69:0/1053944187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15f410be90 0x7f15f41b49d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.848+0000 7f15fadff700 1 --2- 172.21.15.69:0/1053944187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15f4071010 0x7f15f4113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.848+0000 7f15fadff700 1 --2- 172.21.15.69:0/1053944187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f4071960 0x7f15f41140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:19.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.848+0000 7f15fadff700 1 -- 172.21.15.69:0/1053944187 >> 172.21.15.69:0/1053944187 conn(0x7f15f406cbc0 msgr2=0x7f15f406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:21:19.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.849+0000 7f15fadff700 1 -- 172.21.15.69:0/1053944187 shutdown_connections 2023-07-06T20:21:19.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:19.849+0000 7f15fadff700 1 -- 172.21.15.69:0/1053944187 wait complete. 2023-07-06T20:21:19.892 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:21:21.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:20 smithi156 ceph-mon[129280]: pgmap v388: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 293 MiB/s wr, 90 op/s 2023-07-06T20:21:21.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:20 smithi098 ceph-mon[128968]: pgmap v388: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 293 MiB/s wr, 90 op/s 2023-07-06T20:21:21.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:20 smithi069 ceph-mon[126399]: pgmap v388: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 293 MiB/s wr, 90 op/s 2023-07-06T20:21:23.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:22 smithi156 ceph-mon[129280]: pgmap v389: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 339 MiB/s wr, 101 op/s 2023-07-06T20:21:23.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:22 smithi098 ceph-mon[128968]: pgmap v389: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 339 MiB/s wr, 101 op/s 2023-07-06T20:21:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:22 smithi069 ceph-mon[126399]: pgmap v389: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 339 MiB/s wr, 101 op/s 2023-07-06T20:21:25.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:24 smithi156 ceph-mon[129280]: pgmap v390: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 333 MiB/s wr, 101 op/s 2023-07-06T20:21:25.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:24 smithi098 ceph-mon[128968]: pgmap v390: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 333 MiB/s wr, 101 op/s 2023-07-06T20:21:25.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:24 smithi069 ceph-mon[126399]: pgmap v390: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 333 MiB/s wr, 101 op/s 2023-07-06T20:21:27.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:26 smithi156 ceph-mon[129280]: pgmap v391: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 278 MiB/s wr, 85 op/s 2023-07-06T20:21:27.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:26 smithi098 ceph-mon[128968]: pgmap v391: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 278 MiB/s wr, 85 op/s 2023-07-06T20:21:27.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:26 smithi069 ceph-mon[126399]: pgmap v391: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 278 MiB/s wr, 85 op/s 2023-07-06T20:21:29.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:28 smithi156 ceph-mon[129280]: pgmap v392: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 222 MiB/s wr, 72 op/s 2023-07-06T20:21:29.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:28 smithi098 ceph-mon[128968]: pgmap v392: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 222 MiB/s wr, 72 op/s 2023-07-06T20:21:29.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:28 smithi069 ceph-mon[126399]: pgmap v392: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 222 MiB/s wr, 72 op/s 2023-07-06T20:21:31.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:30 smithi156 ceph-mon[129280]: pgmap v393: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 146 MiB/s wr, 48 op/s 2023-07-06T20:21:31.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:30 smithi098 ceph-mon[128968]: pgmap v393: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 146 MiB/s wr, 48 op/s 2023-07-06T20:21:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:30 smithi069 ceph-mon[126399]: pgmap v393: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 146 MiB/s wr, 48 op/s 2023-07-06T20:21:33.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:32 smithi156 ceph-mon[129280]: pgmap v394: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 107 MiB/s wr, 37 op/s 2023-07-06T20:21:33.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:32 smithi098 ceph-mon[128968]: pgmap v394: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 107 MiB/s wr, 37 op/s 2023-07-06T20:21:33.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:32 smithi069 ceph-mon[126399]: pgmap v394: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 107 MiB/s wr, 37 op/s 2023-07-06T20:21:35.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:34 smithi156 ceph-mon[129280]: pgmap v395: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 54 MiB/s wr, 29 op/s 2023-07-06T20:21:35.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:34 smithi098 ceph-mon[128968]: pgmap v395: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 54 MiB/s wr, 29 op/s 2023-07-06T20:21:35.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:34 smithi069 ceph-mon[126399]: pgmap v395: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 54 MiB/s wr, 29 op/s 2023-07-06T20:21:37.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:36 smithi156 ceph-mon[129280]: pgmap v396: 97 pgs: 97 active+clean; 7.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 37 MiB/s wr, 26 op/s 2023-07-06T20:21:37.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:36 smithi098 ceph-mon[128968]: pgmap v396: 97 pgs: 97 active+clean; 7.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 37 MiB/s wr, 26 op/s 2023-07-06T20:21:37.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:36 smithi069 ceph-mon[126399]: pgmap v396: 97 pgs: 97 active+clean; 7.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 37 MiB/s wr, 26 op/s 2023-07-06T20:21:39.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:38 smithi156 ceph-mon[129280]: pgmap v397: 97 pgs: 97 active+clean; 7.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 110 MiB/s wr, 55 op/s 2023-07-06T20:21:39.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:38 smithi098 ceph-mon[128968]: pgmap v397: 97 pgs: 97 active+clean; 7.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 110 MiB/s wr, 55 op/s 2023-07-06T20:21:39.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:38 smithi069 ceph-mon[126399]: pgmap v397: 97 pgs: 97 active+clean; 7.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 110 MiB/s wr, 55 op/s 2023-07-06T20:21:41.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:40 smithi098 ceph-mon[128968]: pgmap v398: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 168 MiB/s wr, 70 op/s 2023-07-06T20:21:41.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:40 smithi069 ceph-mon[126399]: pgmap v398: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 168 MiB/s wr, 70 op/s 2023-07-06T20:21:41.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:40 smithi156 ceph-mon[129280]: pgmap v398: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 168 MiB/s wr, 70 op/s 2023-07-06T20:21:42.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:41 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:21:42.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:41 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:21:42.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:41 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:21:43.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:43 smithi069 ceph-mon[126399]: pgmap v399: 97 pgs: 97 active+clean; 8.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 246 MiB/s wr, 91 op/s 2023-07-06T20:21:43.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:43 smithi156 ceph-mon[129280]: pgmap v399: 97 pgs: 97 active+clean; 8.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 246 MiB/s wr, 91 op/s 2023-07-06T20:21:43.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:43 smithi098 ceph-mon[128968]: pgmap v399: 97 pgs: 97 active+clean; 8.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 246 MiB/s wr, 91 op/s 2023-07-06T20:21:43.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:43 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:43 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:43 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:43 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:43 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:43.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:43 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:44.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:44 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:21:44.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:44 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:21:44.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:44 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:44.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:44 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:21:44.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:44 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:21:44.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:44 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:44.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:44 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:21:44.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:44 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:21:44.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:44 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:21:45.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:45 smithi069 ceph-mon[126399]: pgmap v400: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 299 MiB/s wr, 119 op/s 2023-07-06T20:21:45.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:45 smithi156 ceph-mon[129280]: pgmap v400: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 299 MiB/s wr, 119 op/s 2023-07-06T20:21:45.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:45 smithi098 ceph-mon[128968]: pgmap v400: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 299 MiB/s wr, 119 op/s 2023-07-06T20:21:47.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:47 smithi069 ceph-mon[126399]: pgmap v401: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 322 MiB/s wr, 121 op/s 2023-07-06T20:21:47.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:47 smithi156 ceph-mon[129280]: pgmap v401: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 322 MiB/s wr, 121 op/s 2023-07-06T20:21:47.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:47 smithi098 ceph-mon[128968]: pgmap v401: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 322 MiB/s wr, 121 op/s 2023-07-06T20:21:48.888 INFO:teuthology.orchestra.run.smithi069.stderr:done. 2023-07-06T20:21:48.895 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-07-06T20:21:49.228 INFO:teuthology.orchestra.run.smithi069.stderr:starting vacuum...end. 2023-07-06T20:21:49.514 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:49 smithi069 ceph-mon[126399]: pgmap v402: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 325 MiB/s wr, 127 op/s 2023-07-06T20:21:49.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:49 smithi156 ceph-mon[129280]: pgmap v402: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 325 MiB/s wr, 127 op/s 2023-07-06T20:21:49.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:49 smithi098 ceph-mon[128968]: pgmap v402: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 325 MiB/s wr, 127 op/s 2023-07-06T20:21:49.893 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:21:50.430 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:50 smithi069 ceph-mon[126399]: pgmap v403: 97 pgs: 97 active+clean; 9.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 264 MiB/s wr, 101 op/s 2023-07-06T20:21:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.535+0000 7f67df7ff700 1 -- 172.21.15.69:0/1339872129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d8100e00 msgr2=0x7f67d8101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.535+0000 7f67df7ff700 1 --2- 172.21.15.69:0/1339872129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d8100e00 0x7f67d8101250 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009990 tx=0x7f67c8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.542+0000 7f67df7ff700 1 -- 172.21.15.69:0/1339872129 shutdown_connections 2023-07-06T20:21:50.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.542+0000 7f67df7ff700 1 --2- 172.21.15.69:0/1339872129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67d8101790 0x7f67d8108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.542+0000 7f67df7ff700 1 --2- 172.21.15.69:0/1339872129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80ffc10 0x7f67d8100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.542+0000 7f67df7ff700 1 --2- 172.21.15.69:0/1339872129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d8100e00 0x7f67d8101250 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.542+0000 7f67df7ff700 1 -- 172.21.15.69:0/1339872129 >> 172.21.15.69:0/1339872129 conn(0x7f67d80fb210 msgr2=0x7f67d80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:21:50.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.553+0000 7f67df7ff700 1 -- 172.21.15.69:0/1339872129 shutdown_connections 2023-07-06T20:21:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.553+0000 7f67df7ff700 1 -- 172.21.15.69:0/1339872129 wait complete. 2023-07-06T20:21:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.554+0000 7f67df7ff700 1 Processor -- start 2023-07-06T20:21:50.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.577+0000 7f67df7ff700 1 -- start start 2023-07-06T20:21:50.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.577+0000 7f67df7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80ffc10 0x7f67d806ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.577+0000 7f67df7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d8100e00 0x7f67d806b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.577+0000 7f67df7ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67d8101790 0x7f67d81ac400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.578+0000 7f67df7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80723d0 con 0x7f67d80ffc10 2023-07-06T20:21:50.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.578+0000 7f67df7ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8072510 con 0x7f67d8101790 2023-07-06T20:21:50.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.578+0000 7f67df7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80726a0 con 0x7f67d8100e00 2023-07-06T20:21:50.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.591+0000 7f67ddd9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67d8101790 0x7f67d81ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.591+0000 7f67ddd9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67d8101790 0x7f67d81ac400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:37314/0 (socket says 172.21.15.69:37314) 2023-07-06T20:21:50.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.591+0000 7f67ddd9c700 1 -- 172.21.15.69:0/1001530881 learned_addr learned my addr 172.21.15.69:0/1001530881 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:21:50.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.591+0000 7f67dcd9a700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d8100e00 0x7f67d806b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.600 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.592+0000 7f67dd59b700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80ffc10 0x7f67d806ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.597+0000 7f67ddd9c700 1 -- 172.21.15.69:0/1001530881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d8100e00 msgr2=0x7f67d806b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:50.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.597+0000 7f67ddd9c700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d8100e00 0x7f67d806b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.598+0000 7f67ddd9c700 1 -- 172.21.15.69:0/1001530881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80ffc10 msgr2=0x7f67d806ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:50.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.598+0000 7f67ddd9c700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80ffc10 0x7f67d806ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.601 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.598+0000 7f67ddd9c700 1 -- 172.21.15.69:0/1001530881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c80095f0 con 0x7f67d8101790 2023-07-06T20:21:50.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.614+0000 7f67ddd9c700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67d8101790 0x7f67d81ac400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67d400b4d0 tx=0x7f67d400b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:21:50.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.618+0000 7f67ce7fc700 1 -- 172.21.15.69:0/1001530881 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4011830 con 0x7f67d8101790 2023-07-06T20:21:50.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.618+0000 7f67ce7fc700 1 -- 172.21.15.69:0/1001530881 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d4011e50 con 0x7f67d8101790 2023-07-06T20:21:50.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.618+0000 7f67ce7fc700 1 -- 172.21.15.69:0/1001530881 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400f2e0 con 0x7f67d8101790 2023-07-06T20:21:50.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.627+0000 7f67df7ff700 1 -- 172.21.15.69:0/1001530881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d810dff0 con 0x7f67d8101790 2023-07-06T20:21:50.632 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.627+0000 7f67df7ff700 1 -- 172.21.15.69:0/1001530881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d810e500 con 0x7f67d8101790 2023-07-06T20:21:50.648 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.639+0000 7f67f303db80 1 Processor -- start 2023-07-06T20:21:50.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.649+0000 7f67f303db80 1 -- start start 2023-07-06T20:21:50.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.649+0000 7f67f303db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a38787ff20 0x55a387880330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.649+0000 7f67f303db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a387880870 0x55a387880e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.649+0000 7f67f303db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387881390 0x55a387881830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.649+0000 7f67f303db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a387784cb0 con 0x55a387880870 2023-07-06T20:21:50.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.649+0000 7f67f303db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3877d7a40 con 0x55a387881390 2023-07-06T20:21:50.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.649+0000 7f67f303db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3877d0ed0 con 0x55a38787ff20 2023-07-06T20:21:50.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.651+0000 7f67dcd9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a38787ff20 0x55a387880330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.651+0000 7f67dcd9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387881390 0x55a387881830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.659+0000 7f67ce7fc700 1 -- 172.21.15.69:0/1001530881 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f67d4010580 con 0x7f67d8101790 2023-07-06T20:21:50.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.659+0000 7f67dd59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a387880870 0x55a387880e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.659+0000 7f67dd59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a387880870 0x55a387880e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40602/0 (socket says 172.21.15.69:40602) 2023-07-06T20:21:50.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.659+0000 7f67dd59b700 1 -- 172.21.15.69:0/647206632 learned_addr learned my addr 172.21.15.69:0/647206632 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:21:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.660+0000 7f67dd59b700 1 -- 172.21.15.69:0/647206632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a38787ff20 msgr2=0x55a387880330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:21:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.660+0000 7f67dd59b700 1 --2- 172.21.15.69:0/647206632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a38787ff20 0x55a387880330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.660+0000 7f67dd59b700 1 -- 172.21.15.69:0/647206632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387881390 msgr2=0x55a387881830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:21:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.660+0000 7f67dd59b700 1 --2- 172.21.15.69:0/647206632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387881390 0x55a387881830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.660+0000 7f67dd59b700 1 -- 172.21.15.69:0/647206632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3875fede0 con 0x55a387880870 2023-07-06T20:21:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.661+0000 7f67dd59b700 1 --2- 172.21.15.69:0/647206632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a387880870 0x55a387880e00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009f50 tx=0x7f67d0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e276e3ee4d18263 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:21:50.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.661+0000 7f67c37fe700 1 -- 172.21.15.69:0/647206632 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0011070 con 0x55a387880870 2023-07-06T20:21:50.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.661+0000 7f67c37fe700 1 -- 172.21.15.69:0/647206632 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d0015410 con 0x55a387880870 2023-07-06T20:21:50.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.661+0000 7f67c37fe700 1 -- 172.21.15.69:0/647206632 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d000a660 con 0x55a387880870 2023-07-06T20:21:50.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.662+0000 7f67ce7fc700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f67c40a1350 0x7f67c40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.662+0000 7f67f303db80 1 -- 172.21.15.69:0/647206632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a387880870 msgr2=0x55a387880e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:50.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.662+0000 7f67f303db80 1 --2- 172.21.15.69:0/647206632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a387880870 0x55a387880e00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009f50 tx=0x7f67d0009f80 comp rx=0 tx=0).stop 2023-07-06T20:21:50.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.662+0000 7f67ce7fc700 1 -- 172.21.15.69:0/1001530881 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f67d40cc350 con 0x7f67d8101790 2023-07-06T20:21:50.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.696+0000 7f67dcd9a700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f67c40a1350 0x7f67c40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.697+0000 7f67f303db80 1 -- 172.21.15.69:0/647206632 shutdown_connections 2023-07-06T20:21:50.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.697+0000 7f67f303db80 1 --2- 172.21.15.69:0/647206632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387881390 0x55a387881830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.697+0000 7f67f303db80 1 --2- 172.21.15.69:0/647206632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a38787ff20 0x55a387880330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.697+0000 7f67f303db80 1 --2- 172.21.15.69:0/647206632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a387880870 0x55a387880e00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.697+0000 7f67f303db80 1 -- 172.21.15.69:0/647206632 >> 172.21.15.69:0/647206632 conn(0x55a387875640 msgr2=0x55a38787a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:21:50.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.702+0000 7f67dcd9a700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f67c40a1350 0x7f67c40a3800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f67c800b5c0 tx=0x7f67c8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:21:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.712+0000 7f67f303db80 1 -- 172.21.15.69:0/647206632 shutdown_connections 2023-07-06T20:21:50.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.733+0000 7f67f303db80 1 -- 172.21.15.69:0/647206632 wait complete. 2023-07-06T20:21:50.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.733+0000 7f67f303db80 1 -- start start 2023-07-06T20:21:50.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.734+0000 7f67f303db80 1 Processor -- start 2023-07-06T20:21:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.734+0000 7f67f303db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a38787ff20 0x55a387889860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.735+0000 7f67f303db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a387880870 0x55a387889da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.735+0000 7f67f303db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387881390 0x55a387890de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.735+0000 7f67f303db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a387770ac0 con 0x55a387880870 2023-07-06T20:21:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.735+0000 7f67f303db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3877707c0 con 0x55a387881390 2023-07-06T20:21:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.735+0000 7f67f303db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a387654920 con 0x55a38787ff20 2023-07-06T20:21:50.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.735+0000 7f67f303db80 10 client.0 _ll_register_callbacks cb 0x55a38776e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:21:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.739+0000 7f67ddd9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387881390 0x55a387890de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.740+0000 7f67ddd9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387881390 0x55a387890de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:37320/0 (socket says 172.21.15.69:37320) 2023-07-06T20:21:50.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.740+0000 7f67ddd9c700 1 -- 172.21.15.69:0/4077372234 learned_addr learned my addr 172.21.15.69:0/4077372234 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:21:50.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.743+0000 7f67dd59b700 1 --2- 172.21.15.69:0/4077372234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a38787ff20 0x55a387889860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.754+0000 7f67dd59b700 1 -- 172.21.15.69:0/4077372234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387881390 msgr2=0x55a387890de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:21:50.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.754+0000 7f67dd59b700 1 --2- 172.21.15.69:0/4077372234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387881390 0x55a387890de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.754+0000 7f67dd59b700 1 -- 172.21.15.69:0/4077372234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a387880870 msgr2=0x55a387889da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:21:50.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.754+0000 7f67dd59b700 1 --2- 172.21.15.69:0/4077372234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a387880870 0x55a387889da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.754+0000 7f67dd59b700 1 -- 172.21.15.69:0/4077372234 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d0009c30 con 0x55a38787ff20 2023-07-06T20:21:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.757+0000 7f67dd59b700 1 --2- 172.21.15.69:0/4077372234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a38787ff20 0x55a387889860 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f67d0017000 tx=0x7f67d000cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68d527aa60eb6a90 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:21:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.757+0000 7f67c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:21:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.758+0000 7f67c0ff9700 1 -- 172.21.15.69:0/4077372234 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0011050 con 0x55a38787ff20 2023-07-06T20:21:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.758+0000 7f67f303db80 1 -- 172.21.15.69:0/4077372234 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a38772a4f0 con 0x55a38787ff20 2023-07-06T20:21:50.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.758+0000 7f67c0ff9700 1 -- 172.21.15.69:0/4077372234 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f67d0006e40 con 0x55a38787ff20 2023-07-06T20:21:50.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.759+0000 7f67c0ff9700 1 -- 172.21.15.69:0/4077372234 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67d0015b80 con 0x55a38787ff20 2023-07-06T20:21:50.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.759+0000 7f67f303db80 10 client.25441 fetch_fsmap learned FSMap version 27 2023-07-06T20:21:50.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.759+0000 7f67f303db80 1 -- 172.21.15.69:0/4077372234 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a3875e7c60 con 0x55a38787ff20 2023-07-06T20:21:50.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.759+0000 7f67c0ff9700 1 -- 172.21.15.69:0/4077372234 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a38772a4f0 con 0x55a38787ff20 2023-07-06T20:21:50.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.760+0000 7f67c0ff9700 1 -- 172.21.15.69:0/4077372234 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f67d001e830 con 0x55a38787ff20 2023-07-06T20:21:50.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.760+0000 7f67f303db80 10 client.25441 fetch_fsmap finished waiting for FSMap version 27 2023-07-06T20:21:50.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.760+0000 7f67f303db80 10 client.25441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:21:50.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.760+0000 7f67f303db80 20 client.25441 populate_metadata read hostname 'smithi069' 2023-07-06T20:21:50.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.760+0000 7f67f303db80 1 --2- 172.21.15.69:0/4077372234 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a3878954f0 0x55a387897900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.760+0000 7f67f303db80 4 client.25441 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:21:50.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.760+0000 7f67f303db80 1 -- 172.21.15.69:0/4077372234 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a38784ed60 con 0x55a3878954f0 2023-07-06T20:21:50.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.761+0000 7f67dcd9a700 1 --2- 172.21.15.69:0/4077372234 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a3878954f0 0x55a387897900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.763+0000 7f67dcd9a700 1 --2- 172.21.15.69:0/4077372234 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a3878954f0 0x55a387897900 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24586bc740b6a51a server_cookie=95ecec279356df3f in_seq=0 out_seq=0 2023-07-06T20:21:50.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.763+0000 7f67c0ff9700 10 client.25441 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:21:50.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.773+0000 7f67c0ff9700 1 -- 172.21.15.69:0/4077372234 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a38784ed60 con 0x55a3878954f0 2023-07-06T20:21:50.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.773+0000 7f67c0ff9700 10 client.25441 handle_command_reply: tid=0 2023-07-06T20:21:50.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.773+0000 7f67f303db80 1 client.25441 shutdown 2023-07-06T20:21:50.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.773+0000 7f67f303db80 1 -- 172.21.15.69:0/4077372234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a38787ff20 msgr2=0x55a387889860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:50.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.773+0000 7f67f303db80 1 --2- 172.21.15.69:0/4077372234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a38787ff20 0x55a387889860 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f67d0017000 tx=0x7f67d000cd10 comp rx=0 tx=0).stop 2023-07-06T20:21:50.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.780+0000 7f67f303db80 1 -- 172.21.15.69:0/4077372234 shutdown_connections 2023-07-06T20:21:50.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.780+0000 7f67f303db80 1 --2- 172.21.15.69:0/4077372234 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a3878954f0 0x55a387897900 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.780+0000 7f67f303db80 1 --2- 172.21.15.69:0/4077372234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387881390 0x55a387890de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.780+0000 7f67f303db80 1 --2- 172.21.15.69:0/4077372234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a38787ff20 0x55a387889860 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.780+0000 7f67f303db80 1 --2- 172.21.15.69:0/4077372234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a387880870 0x55a387889da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.780+0000 7f67f303db80 1 -- 172.21.15.69:0/4077372234 >> 172.21.15.69:0/4077372234 conn(0x55a387875640 msgr2=0x55a38787a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:21:50.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.785+0000 7f67c0ff9700 0 client.25441 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:21:50.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.785+0000 7f67f303db80 1 -- 172.21.15.69:0/4077372234 shutdown_connections 2023-07-06T20:21:50.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.795+0000 7f67f303db80 1 -- 172.21.15.69:0/4077372234 wait complete. 2023-07-06T20:21:50.824 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.822+0000 7f67f303db80 20 client.25441 trim_cache size 0 max 16384 2023-07-06T20:21:50.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:50 smithi156 ceph-mon[129280]: pgmap v403: 97 pgs: 97 active+clean; 9.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 264 MiB/s wr, 101 op/s 2023-07-06T20:21:50.870 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.860+0000 7f67f303db80 1 Processor -- start 2023-07-06T20:21:50.870 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.860+0000 7f67f303db80 1 -- start start 2023-07-06T20:21:50.870 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.861+0000 7f67f303db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 0x55a3878916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.870 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.861+0000 7f67f303db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38787ff20 0x55a387890da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.870 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.861+0000 7f67f303db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a387880870 0x55a3878876a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.870 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.861+0000 7f67f303db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a387889430 con 0x55a38787ff20 2023-07-06T20:21:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.861+0000 7f67f303db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a387888ce0 con 0x55a387875640 2023-07-06T20:21:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.861+0000 7f67f303db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a387872940 con 0x55a387880870 2023-07-06T20:21:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.865+0000 7f67ddd9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a387880870 0x55a3878876a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.865+0000 7f67dd59b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 0x55a3878916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.865+0000 7f67ddd9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a387880870 0x55a3878876a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:53134/0 (socket says 172.21.15.69:53134) 2023-07-06T20:21:50.871 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.865+0000 7f67ddd9c700 1 -- 172.21.15.69:0/4033394871 learned_addr learned my addr 172.21.15.69:0/4033394871 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:21:50.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:50 smithi098 ceph-mon[128968]: pgmap v403: 97 pgs: 97 active+clean; 9.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 264 MiB/s wr, 101 op/s 2023-07-06T20:21:50.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.872+0000 7f67dd59b700 1 -- 172.21.15.69:0/4033394871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a387880870 msgr2=0x55a3878876a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:21:50.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.872+0000 7f67dd59b700 1 --2- 172.21.15.69:0/4033394871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a387880870 0x55a3878876a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.872+0000 7f67dd59b700 1 -- 172.21.15.69:0/4033394871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38787ff20 msgr2=0x55a387890da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:21:50.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.872+0000 7f67dd59b700 1 --2- 172.21.15.69:0/4033394871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38787ff20 0x55a387890da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.873+0000 7f67dd59b700 1 -- 172.21.15.69:0/4033394871 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a387625250 con 0x55a387875640 2023-07-06T20:21:50.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.888+0000 7f67dd59b700 1 --2- 172.21.15.69:0/4033394871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 0x55a3878916d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009ad0 tx=0x7f67d001ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b528efda50bba4a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:21:50.902 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.898+0000 7f67c3fff700 1 -- 172.21.15.69:0/4033394871 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0011070 con 0x55a387875640 2023-07-06T20:21:50.902 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.899+0000 7f67c3fff700 1 -- 172.21.15.69:0/4033394871 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d0012040 con 0x55a387875640 2023-07-06T20:21:50.902 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.899+0000 7f67c3fff700 1 -- 172.21.15.69:0/4033394871 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d001fe90 con 0x55a387875640 2023-07-06T20:21:50.902 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.900+0000 7f67f303db80 1 -- 172.21.15.69:0/4033394871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 msgr2=0x55a3878916d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.900+0000 7f67f303db80 1 --2- 172.21.15.69:0/4033394871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 0x55a3878916d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009ad0 tx=0x7f67d001ed50 comp rx=0 tx=0).stop 2023-07-06T20:21:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.900+0000 7f67f303db80 1 -- 172.21.15.69:0/4033394871 shutdown_connections 2023-07-06T20:21:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.900+0000 7f67f303db80 1 --2- 172.21.15.69:0/4033394871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a387880870 0x55a3878876a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.900+0000 7f67f303db80 1 --2- 172.21.15.69:0/4033394871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 0x55a3878916d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.904 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.900+0000 7f67f303db80 1 --2- 172.21.15.69:0/4033394871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38787ff20 0x55a387890da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.904 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.900+0000 7f67f303db80 1 -- 172.21.15.69:0/4033394871 >> 172.21.15.69:0/4033394871 conn(0x55a387888910 msgr2=0x55a38789c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:21:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.905+0000 7f67f303db80 1 -- 172.21.15.69:0/4033394871 shutdown_connections 2023-07-06T20:21:50.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.921+0000 7f67f303db80 1 -- 172.21.15.69:0/4033394871 wait complete. 2023-07-06T20:21:50.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.922+0000 7f67f303db80 1 -- start start 2023-07-06T20:21:50.925 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.923+0000 7f67f303db80 1 Processor -- start 2023-07-06T20:21:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.924+0000 7f67f303db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 0x55a38788cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.925+0000 7f67f303db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38787ff20 0x55a38788d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.925+0000 7f67f303db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a387880870 0x55a38788fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.925+0000 7f67f303db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a387890370 con 0x55a38787ff20 2023-07-06T20:21:50.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.925+0000 7f67f303db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3878904e0 con 0x55a387875640 2023-07-06T20:21:50.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.925+0000 7f67f303db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3878906a0 con 0x55a387880870 2023-07-06T20:21:50.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.925+0000 7f67f303db80 10 client.0 _ll_register_callbacks cb 0x7f67d0009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:21:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.926+0000 7f67ddd9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a387880870 0x55a38788fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.929+0000 7f67dd59b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 0x55a38788cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.930+0000 7f67dd59b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 0x55a38788cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:37340/0 (socket says 172.21.15.69:37340) 2023-07-06T20:21:50.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.930+0000 7f67dd59b700 1 -- 172.21.15.69:0/3190508435 learned_addr learned my addr 172.21.15.69:0/3190508435 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:21:50.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.932+0000 7f67dcd9a700 1 --2- 172.21.15.69:0/3190508435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38787ff20 0x55a38788d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.933+0000 7f67dd59b700 1 -- 172.21.15.69:0/3190508435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a387880870 msgr2=0x55a38788fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:21:50.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.933+0000 7f67dd59b700 1 --2- 172.21.15.69:0/3190508435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a387880870 0x55a38788fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.933+0000 7f67dd59b700 1 -- 172.21.15.69:0/3190508435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38787ff20 msgr2=0x55a38788d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:21:50.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.933+0000 7f67dd59b700 1 --2- 172.21.15.69:0/3190508435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38787ff20 0x55a38788d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:50.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.933+0000 7f67dd59b700 1 -- 172.21.15.69:0/3190508435 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3875e7c60 con 0x55a387875640 2023-07-06T20:21:50.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.934+0000 7f67dd59b700 1 --2- 172.21.15.69:0/3190508435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 0x55a38788cfd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f67d0002230 tx=0x7f67d001fef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6455286a144d9bcd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:21:50.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.935+0000 7f67c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:21:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.937+0000 7f67c0ff9700 1 -- 172.21.15.69:0/3190508435 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0011070 con 0x55a387875640 2023-07-06T20:21:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.937+0000 7f67c0ff9700 1 -- 172.21.15.69:0/3190508435 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f67d001e840 con 0x55a387875640 2023-07-06T20:21:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.938+0000 7f67f303db80 1 -- 172.21.15.69:0/3190508435 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67d0015410 con 0x55a387875640 2023-07-06T20:21:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.942+0000 7f67c0ff9700 1 -- 172.21.15.69:0/3190508435 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67d001f3d0 con 0x55a387875640 2023-07-06T20:21:50.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.942+0000 7f67f303db80 10 client.25508 fetch_fsmap learned FSMap version 27 2023-07-06T20:21:50.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.942+0000 7f67f303db80 1 -- 172.21.15.69:0/3190508435 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a3875fd2b0 con 0x55a387875640 2023-07-06T20:21:50.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.945+0000 7f67c0ff9700 1 -- 172.21.15.69:0/3190508435 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d0015410 con 0x55a387875640 2023-07-06T20:21:50.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.945+0000 7f67c0ff9700 1 -- 172.21.15.69:0/3190508435 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f67d002bc10 con 0x55a387875640 2023-07-06T20:21:50.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.946+0000 7f67f303db80 10 client.25508 fetch_fsmap finished waiting for FSMap version 27 2023-07-06T20:21:50.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.946+0000 7f67f303db80 10 client.25508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:21:50.950 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.946+0000 7f67f303db80 20 client.25508 populate_metadata read hostname 'smithi069' 2023-07-06T20:21:50.950 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.946+0000 7f67f303db80 1 --2- 172.21.15.69:0/3190508435 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a387881390 0x55a3878ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:21:50.950 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.946+0000 7f67f303db80 4 client.25508 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:21:50.950 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.946+0000 7f67f303db80 1 -- 172.21.15.69:0/3190508435 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a3877c9eb0 con 0x55a387881390 2023-07-06T20:21:50.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.951+0000 7f67dcd9a700 1 --2- 172.21.15.69:0/3190508435 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a387881390 0x55a3878ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:21:50.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.990+0000 7f67dcd9a700 1 --2- 172.21.15.69:0/3190508435 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a387881390 0x55a3878ebaf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ecc34d7e6143cde server_cookie=2886e8c1401ab7a8 in_seq=0 out_seq=0 2023-07-06T20:21:50.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:50.991+0000 7f67c0ff9700 10 client.25508 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:21:51.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.004+0000 7f67c0ff9700 1 -- 172.21.15.69:0/3190508435 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a3877c9eb0 con 0x55a387881390 2023-07-06T20:21:51.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.004+0000 7f67c0ff9700 10 client.25508 handle_command_reply: tid=0 2023-07-06T20:21:51.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.004+0000 7f67f303db80 1 client.25508 shutdown 2023-07-06T20:21:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.008+0000 7f67f303db80 1 -- 172.21.15.69:0/3190508435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 msgr2=0x55a38788cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.008+0000 7f67f303db80 1 --2- 172.21.15.69:0/3190508435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 0x55a38788cfd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f67d0002230 tx=0x7f67d001fef0 comp rx=0 tx=0).stop 2023-07-06T20:21:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.008+0000 7f67f303db80 1 -- 172.21.15.69:0/3190508435 shutdown_connections 2023-07-06T20:21:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.008+0000 7f67f303db80 1 --2- 172.21.15.69:0/3190508435 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a387881390 0x55a3878ebaf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.008+0000 7f67f303db80 1 --2- 172.21.15.69:0/3190508435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a387880870 0x55a38788fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.008+0000 7f67f303db80 1 --2- 172.21.15.69:0/3190508435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a387875640 0x55a38788cfd0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f67d0002230 tx=0x7f67d001fef0 comp rx=0 tx=0).stop 2023-07-06T20:21:51.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.008+0000 7f67f303db80 1 --2- 172.21.15.69:0/3190508435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a38787ff20 0x55a38788d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:51.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.008+0000 7f67f303db80 1 -- 172.21.15.69:0/3190508435 >> 172.21.15.69:0/3190508435 conn(0x55a387888910 msgr2=0x55a3878916d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:21:51.014 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.012+0000 7f67c0ff9700 0 client.25508 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:21:51.042 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.039+0000 7f67f303db80 1 -- 172.21.15.69:0/3190508435 shutdown_connections 2023-07-06T20:21:51.042 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.040+0000 7f67f303db80 1 -- 172.21.15.69:0/3190508435 wait complete. 2023-07-06T20:21:51.043 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.040+0000 7f67f303db80 20 client.25508 trim_cache size 0 max 16384 2023-07-06T20:21:51.043 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:21:51.044 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:21:51.044 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:21:51.044 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:21:51.044 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:21:51.044 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:21:51.044 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:21:51.045 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:21:51.045 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:21:51.045 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:21:51.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.052+0000 7f67df7ff700 1 -- 172.21.15.69:0/1001530881 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f67c40a1350 msgr2=0x7f67c40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:51.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.052+0000 7f67df7ff700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f67c40a1350 0x7f67c40a3800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f67c800b5c0 tx=0x7f67c8009310 comp rx=0 tx=0).stop 2023-07-06T20:21:51.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.052+0000 7f67df7ff700 1 -- 172.21.15.69:0/1001530881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67d8101790 msgr2=0x7f67d81ac400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:21:51.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.052+0000 7f67df7ff700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67d8101790 0x7f67d81ac400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67d400b4d0 tx=0x7f67d400b890 comp rx=0 tx=0).stop 2023-07-06T20:21:51.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.057+0000 7f67df7ff700 1 -- 172.21.15.69:0/1001530881 shutdown_connections 2023-07-06T20:21:51.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.057+0000 7f67df7ff700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f67c40a1350 0x7f67c40a3800 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:51.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.057+0000 7f67df7ff700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67d8101790 0x7f67d81ac400 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:51.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.057+0000 7f67df7ff700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80ffc10 0x7f67d806ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:51.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.057+0000 7f67df7ff700 1 --2- 172.21.15.69:0/1001530881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67d8100e00 0x7f67d806b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:21:51.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.057+0000 7f67df7ff700 1 -- 172.21.15.69:0/1001530881 >> 172.21.15.69:0/1001530881 conn(0x7f67d80fb210 msgr2=0x7f67d810c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:21:51.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.064+0000 7f67df7ff700 1 -- 172.21.15.69:0/1001530881 shutdown_connections 2023-07-06T20:21:51.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:21:51.069+0000 7f67df7ff700 1 -- 172.21.15.69:0/1001530881 wait complete. 2023-07-06T20:21:51.110 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:21:53.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:52 smithi069 ceph-mon[126399]: pgmap v404: 97 pgs: 97 active+clean; 9.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s rd, 232 MiB/s wr, 111 op/s 2023-07-06T20:21:53.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:52 smithi156 ceph-mon[129280]: pgmap v404: 97 pgs: 97 active+clean; 9.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s rd, 232 MiB/s wr, 111 op/s 2023-07-06T20:21:53.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:52 smithi098 ceph-mon[128968]: pgmap v404: 97 pgs: 97 active+clean; 9.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s rd, 232 MiB/s wr, 111 op/s 2023-07-06T20:21:54.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 5.0 s, 765.0 tps, lat 124.995 ms stddev 133.262 2023-07-06T20:21:55.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:54 smithi156 ceph-mon[129280]: pgmap v405: 97 pgs: 97 active+clean; 8.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 214 MiB/s rd, 159 MiB/s wr, 143 op/s 2023-07-06T20:21:55.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:54 smithi098 ceph-mon[128968]: pgmap v405: 97 pgs: 97 active+clean; 8.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 214 MiB/s rd, 159 MiB/s wr, 143 op/s 2023-07-06T20:21:55.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:54 smithi069 ceph-mon[126399]: pgmap v405: 97 pgs: 97 active+clean; 8.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 214 MiB/s rd, 159 MiB/s wr, 143 op/s 2023-07-06T20:21:57.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:56 smithi156 ceph-mon[129280]: pgmap v406: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 371 MiB/s rd, 109 MiB/s wr, 170 op/s 2023-07-06T20:21:57.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:56 smithi098 ceph-mon[128968]: pgmap v406: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 371 MiB/s rd, 109 MiB/s wr, 170 op/s 2023-07-06T20:21:57.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:56 smithi069 ceph-mon[126399]: pgmap v406: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 371 MiB/s rd, 109 MiB/s wr, 170 op/s 2023-07-06T20:21:59.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:21:58 smithi156 ceph-mon[129280]: pgmap v407: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 502 MiB/s rd, 90 MiB/s wr, 246 op/s 2023-07-06T20:21:59.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:21:58 smithi098 ceph-mon[128968]: pgmap v407: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 502 MiB/s rd, 90 MiB/s wr, 246 op/s 2023-07-06T20:21:59.234 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 10.0 s, 3587.3 tps, lat 27.917 ms stddev 19.695 2023-07-06T20:21:59.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:21:58 smithi069 ceph-mon[126399]: pgmap v407: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 502 MiB/s rd, 90 MiB/s wr, 246 op/s 2023-07-06T20:22:01.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:00 smithi156 ceph-mon[129280]: pgmap v408: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 531 MiB/s rd, 72 MiB/s wr, 329 op/s 2023-07-06T20:22:01.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:00 smithi098 ceph-mon[128968]: pgmap v408: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 531 MiB/s rd, 72 MiB/s wr, 329 op/s 2023-07-06T20:22:01.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:00 smithi069 ceph-mon[126399]: pgmap v408: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 531 MiB/s rd, 72 MiB/s wr, 329 op/s 2023-07-06T20:22:03.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:02 smithi098 ceph-mon[128968]: pgmap v409: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 533 MiB/s rd, 59 MiB/s wr, 556 op/s 2023-07-06T20:22:03.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:02 smithi069 ceph-mon[126399]: pgmap v409: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 533 MiB/s rd, 59 MiB/s wr, 556 op/s 2023-07-06T20:22:03.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:02 smithi156 ceph-mon[129280]: pgmap v409: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 533 MiB/s rd, 59 MiB/s wr, 556 op/s 2023-07-06T20:22:04.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 15.0 s, 2449.6 tps, lat 40.913 ms stddev 63.002 2023-07-06T20:22:05.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:04 smithi069 ceph-mon[126399]: pgmap v410: 97 pgs: 97 active+clean; 8.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 465 MiB/s rd, 46 MiB/s wr, 1.16k op/s 2023-07-06T20:22:05.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:04 smithi156 ceph-mon[129280]: pgmap v410: 97 pgs: 97 active+clean; 8.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 465 MiB/s rd, 46 MiB/s wr, 1.16k op/s 2023-07-06T20:22:05.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:04 smithi098 ceph-mon[128968]: pgmap v410: 97 pgs: 97 active+clean; 8.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 465 MiB/s rd, 46 MiB/s wr, 1.16k op/s 2023-07-06T20:22:07.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:06 smithi069 ceph-mon[126399]: pgmap v411: 97 pgs: 97 active+clean; 8.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 319 MiB/s rd, 57 MiB/s wr, 2.28k op/s 2023-07-06T20:22:07.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:06 smithi156 ceph-mon[129280]: pgmap v411: 97 pgs: 97 active+clean; 8.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 319 MiB/s rd, 57 MiB/s wr, 2.28k op/s 2023-07-06T20:22:07.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:06 smithi098 ceph-mon[128968]: pgmap v411: 97 pgs: 97 active+clean; 8.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 319 MiB/s rd, 57 MiB/s wr, 2.28k op/s 2023-07-06T20:22:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 20.0 s, 3264.0 tps, lat 30.594 ms stddev 21.687 2023-07-06T20:22:09.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:08 smithi069 ceph-mon[126399]: pgmap v412: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 162 MiB/s rd, 76 MiB/s wr, 3.63k op/s 2023-07-06T20:22:09.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:08 smithi156 ceph-mon[129280]: pgmap v412: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 162 MiB/s rd, 76 MiB/s wr, 3.63k op/s 2023-07-06T20:22:09.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:08 smithi098 ceph-mon[128968]: pgmap v412: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 162 MiB/s rd, 76 MiB/s wr, 3.63k op/s 2023-07-06T20:22:11.263 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:10 smithi069 ceph-mon[126399]: pgmap v413: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 77 MiB/s wr, 4.00k op/s 2023-07-06T20:22:11.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:10 smithi156 ceph-mon[129280]: pgmap v413: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 77 MiB/s wr, 4.00k op/s 2023-07-06T20:22:11.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:10 smithi098 ceph-mon[128968]: pgmap v413: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 77 MiB/s wr, 4.00k op/s 2023-07-06T20:22:13.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:12 smithi069 ceph-mon[126399]: pgmap v414: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 80 MiB/s wr, 4.11k op/s 2023-07-06T20:22:13.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:12 smithi156 ceph-mon[129280]: pgmap v414: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 80 MiB/s wr, 4.11k op/s 2023-07-06T20:22:13.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:12 smithi098 ceph-mon[128968]: pgmap v414: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 80 MiB/s wr, 4.11k op/s 2023-07-06T20:22:14.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 25.0 s, 2357.8 tps, lat 42.423 ms stddev 62.654 2023-07-06T20:22:14.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:14 smithi156 ceph-mon[129280]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:22:14.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:14 smithi098 ceph-mon[128968]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:22:14.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:14 smithi069 ceph-mon[126399]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:22:15.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:15 smithi156 ceph-mon[129280]: pgmap v415: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 87 MiB/s wr, 4.82k op/s 2023-07-06T20:22:15.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:15 smithi098 ceph-mon[128968]: pgmap v415: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 87 MiB/s wr, 4.82k op/s 2023-07-06T20:22:15.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:15 smithi069 ceph-mon[126399]: pgmap v415: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 87 MiB/s wr, 4.82k op/s 2023-07-06T20:22:17.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:17 smithi156 ceph-mon[129280]: pgmap v416: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 89 MiB/s wr, 5.41k op/s 2023-07-06T20:22:17.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:17 smithi098 ceph-mon[128968]: pgmap v416: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 89 MiB/s wr, 5.41k op/s 2023-07-06T20:22:17.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:17 smithi069 ceph-mon[126399]: pgmap v416: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 89 MiB/s wr, 5.41k op/s 2023-07-06T20:22:19.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 30.0 s, 3276.1 tps, lat 30.492 ms stddev 22.175 2023-07-06T20:22:19.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:19 smithi156 ceph-mon[129280]: pgmap v417: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 91 MiB/s wr, 5.51k op/s 2023-07-06T20:22:19.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:19 smithi156 ceph-mon[129280]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:22:19.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:19 smithi098 ceph-mon[128968]: pgmap v417: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 91 MiB/s wr, 5.51k op/s 2023-07-06T20:22:19.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:19 smithi098 ceph-mon[128968]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:22:19.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:19 smithi069 ceph-mon[126399]: pgmap v417: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 91 MiB/s wr, 5.51k op/s 2023-07-06T20:22:19.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:19 smithi069 ceph-mon[126399]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:22:21.111 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:22:21.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:21 smithi156 ceph-mon[129280]: pgmap v418: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 MiB/s wr, 4.48k op/s 2023-07-06T20:22:21.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:21 smithi098 ceph-mon[128968]: pgmap v418: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 MiB/s wr, 4.48k op/s 2023-07-06T20:22:21.460 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.458+0000 7f582d5ff700 1 -- 172.21.15.69:0/114776225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58280feb10 msgr2=0x7f58280fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:21.460 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.458+0000 7f582d5ff700 1 --2- 172.21.15.69:0/114776225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58280feb10 0x7f58280fef20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f581c009990 tx=0x7f581c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.463+0000 7f582d5ff700 1 -- 172.21.15.69:0/114776225 shutdown_connections 2023-07-06T20:22:21.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.463+0000 7f582d5ff700 1 --2- 172.21.15.69:0/114776225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5828100690 0x7f5828106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.465 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.463+0000 7f582d5ff700 1 --2- 172.21.15.69:0/114776225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58280feb10 0x7f58280fef20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.463+0000 7f582d5ff700 1 --2- 172.21.15.69:0/114776225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58280ffd00 0x7f5828100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.463+0000 7f582d5ff700 1 -- 172.21.15.69:0/114776225 >> 172.21.15.69:0/114776225 conn(0x7f58280fa130 msgr2=0x7f58280fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:22:21.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.463+0000 7f582d5ff700 1 -- 172.21.15.69:0/114776225 shutdown_connections 2023-07-06T20:22:21.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.463+0000 7f582d5ff700 1 -- 172.21.15.69:0/114776225 wait complete. 2023-07-06T20:22:21.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.465+0000 7f582d5ff700 1 Processor -- start 2023-07-06T20:22:21.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.465+0000 7f582d5ff700 1 -- start start 2023-07-06T20:22:21.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.466+0000 7f582d5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58280feb10 0x7f58281937e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.466+0000 7f582d5ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58280ffd00 0x7f5828193d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.466+0000 7f582d5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5828100690 0x7f582819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.466+0000 7f582d5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582819b360 con 0x7f58280feb10 2023-07-06T20:22:21.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.466+0000 7f582d5ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582819b4a0 con 0x7f58280ffd00 2023-07-06T20:22:21.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.466+0000 7f582d5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582819b630 con 0x7f5828100690 2023-07-06T20:22:21.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.466+0000 7f58267fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58280ffd00 0x7f5828193d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:21.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.466+0000 7f58267fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58280ffd00 0x7f5828193d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:58922/0 (socket says 172.21.15.69:58922) 2023-07-06T20:22:21.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.466+0000 7f58267fc700 1 -- 172.21.15.69:0/1440975988 learned_addr learned my addr 172.21.15.69:0/1440975988 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:22:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.467+0000 7f58267fc700 1 -- 172.21.15.69:0/1440975988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5828100690 msgr2=0x7f582819ad90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T20:22:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.467+0000 7f58267fc700 1 --2- 172.21.15.69:0/1440975988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5828100690 0x7f582819ad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.467+0000 7f58267fc700 1 -- 172.21.15.69:0/1440975988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58280feb10 msgr2=0x7f58281937e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.467+0000 7f58267fc700 1 --2- 172.21.15.69:0/1440975988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58280feb10 0x7f58281937e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.467+0000 7f58267fc700 1 -- 172.21.15.69:0/1440975988 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0095f0 con 0x7f58280ffd00 2023-07-06T20:22:21.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.468+0000 7f58267fc700 1 --2- 172.21.15.69:0/1440975988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58280ffd00 0x7f5828193d20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f581800e9a0 tx=0x7f581800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:22:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.468+0000 7f580ffff700 1 -- 172.21.15.69:0/1440975988 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581800cbd0 con 0x7f58280ffd00 2023-07-06T20:22:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.468+0000 7f580ffff700 1 -- 172.21.15.69:0/1440975988 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f581800cd30 con 0x7f58280ffd00 2023-07-06T20:22:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.468+0000 7f582d5ff700 1 -- 172.21.15.69:0/1440975988 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582819b910 con 0x7f58280ffd00 2023-07-06T20:22:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.468+0000 7f580ffff700 1 -- 172.21.15.69:0/1440975988 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818018710 con 0x7f58280ffd00 2023-07-06T20:22:21.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.468+0000 7f582d5ff700 1 -- 172.21.15.69:0/1440975988 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582819be20 con 0x7f58280ffd00 2023-07-06T20:22:21.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.476+0000 7f5840e71b80 1 Processor -- start 2023-07-06T20:22:21.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.478+0000 7f580ffff700 1 -- 172.21.15.69:0/1440975988 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5818010980 con 0x7f58280ffd00 2023-07-06T20:22:21.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.479+0000 7f5840e71b80 1 -- start start 2023-07-06T20:22:21.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.479+0000 7f5840e71b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2032f20 0x5600a2033330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.479+0000 7f5840e71b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2033870 0x5600a2033e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.479+0000 7f5840e71b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2034390 0x5600a2034830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.479+0000 7f5840e71b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a1f37cb0 con 0x5600a2033870 2023-07-06T20:22:21.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.479+0000 7f5840e71b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a1f8aa40 con 0x5600a2034390 2023-07-06T20:22:21.482 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.479+0000 7f5840e71b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a1f83ed0 con 0x5600a2032f20 2023-07-06T20:22:21.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.480+0000 7f58277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2032f20 0x5600a2033330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:21.483 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.480+0000 7f58277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2032f20 0x5600a2033330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:34990/0 (socket says 172.21.15.69:34990) 2023-07-06T20:22:21.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.480+0000 7f58277fe700 1 -- 172.21.15.69:0/3657687281 learned_addr learned my addr 172.21.15.69:0/3657687281 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:22:21.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.480+0000 7f58277fe700 1 -- 172.21.15.69:0/3657687281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2034390 msgr2=0x5600a2034830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:22:21.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.480+0000 7f58277fe700 1 --2- 172.21.15.69:0/3657687281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2034390 0x5600a2034830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.480+0000 7f58277fe700 1 -- 172.21.15.69:0/3657687281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2033870 msgr2=0x5600a2033e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:21.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.480+0000 7f58277fe700 1 --2- 172.21.15.69:0/3657687281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2033870 0x5600a2033e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.484 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.480+0000 7f58277fe700 1 -- 172.21.15.69:0/3657687281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a1db1de0 con 0x5600a2032f20 2023-07-06T20:22:21.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.480+0000 7f580ffff700 1 --2- 172.21.15.69:0/1440975988 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f58100a11d0 0x7f58100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.480+0000 7f58277fe700 1 --2- 172.21.15.69:0/3657687281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2032f20 0x5600a2033330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5820009be0 tx=0x7f5820009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bd01240637ea31a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:22:21.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.481+0000 7f580ffff700 1 -- 172.21.15.69:0/1440975988 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f5818014070 con 0x7f58280ffd00 2023-07-06T20:22:21.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.481+0000 7f580d7fa700 1 -- 172.21.15.69:0/3657687281 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5820004bb0 con 0x5600a2032f20 2023-07-06T20:22:21.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.481+0000 7f580d7fa700 1 -- 172.21.15.69:0/3657687281 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f582000f040 con 0x5600a2032f20 2023-07-06T20:22:21.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.481+0000 7f580d7fa700 1 -- 172.21.15.69:0/3657687281 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5820013650 con 0x5600a2032f20 2023-07-06T20:22:21.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.482+0000 7f5840e71b80 1 -- 172.21.15.69:0/3657687281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2032f20 msgr2=0x5600a2033330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:21.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.482+0000 7f5840e71b80 1 --2- 172.21.15.69:0/3657687281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2032f20 0x5600a2033330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5820009be0 tx=0x7f5820009ef0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.482+0000 7f5826ffd700 1 --2- 172.21.15.69:0/1440975988 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f58100a11d0 0x7f58100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:21.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.486+0000 7f5826ffd700 1 --2- 172.21.15.69:0/1440975988 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f58100a11d0 0x7f58100a3680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f581c00b620 tx=0x7f581c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:22:21.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.488+0000 7f5840e71b80 1 -- 172.21.15.69:0/3657687281 shutdown_connections 2023-07-06T20:22:21.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.488+0000 7f5840e71b80 1 --2- 172.21.15.69:0/3657687281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2034390 0x5600a2034830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.488+0000 7f5840e71b80 1 --2- 172.21.15.69:0/3657687281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2032f20 0x5600a2033330 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.488+0000 7f5840e71b80 1 --2- 172.21.15.69:0/3657687281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2033870 0x5600a2033e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.488+0000 7f5840e71b80 1 -- 172.21.15.69:0/3657687281 >> 172.21.15.69:0/3657687281 conn(0x5600a2028640 msgr2=0x5600a202d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:22:21.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.488+0000 7f5840e71b80 1 -- 172.21.15.69:0/3657687281 shutdown_connections 2023-07-06T20:22:21.492 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.489+0000 7f5840e71b80 1 -- 172.21.15.69:0/3657687281 wait complete. 2023-07-06T20:22:21.492 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.490+0000 7f5840e71b80 1 -- start start 2023-07-06T20:22:21.493 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.490+0000 7f5840e71b80 1 Processor -- start 2023-07-06T20:22:21.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.495+0000 7f5840e71b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a203c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.495+0000 7f5840e71b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 0x5600a203cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.495+0000 7f5840e71b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2034390 0x5600a2043e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.495+0000 7f5840e71b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a1f23ac0 con 0x5600a2034390 2023-07-06T20:22:21.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.495+0000 7f5840e71b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a1f237c0 con 0x5600a2032f20 2023-07-06T20:22:21.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.495+0000 7f5840e71b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a1e07920 con 0x5600a2033870 2023-07-06T20:22:21.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.495+0000 7f5840e71b80 10 client.0 _ll_register_callbacks cb 0x5600a1f21760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:22:21.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.495+0000 7f5826ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a203c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:21.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.495+0000 7f5826ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a203c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:58946/0 (socket says 172.21.15.69:58946) 2023-07-06T20:22:21.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.495+0000 7f5826ffd700 1 -- 172.21.15.69:0/2489884418 learned_addr learned my addr 172.21.15.69:0/2489884418 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:22:21.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.495+0000 7f58267fc700 1 --2- 172.21.15.69:0/2489884418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 0x5600a203cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:21.500 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.496+0000 7f58267fc700 1 -- 172.21.15.69:0/2489884418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 msgr2=0x5600a203c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:21.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.496+0000 7f58267fc700 1 --2- 172.21.15.69:0/2489884418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a203c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.496+0000 7f58267fc700 1 -- 172.21.15.69:0/2489884418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2034390 msgr2=0x5600a2043e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:21.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.496+0000 7f58267fc700 1 --2- 172.21.15.69:0/2489884418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2034390 0x5600a2043e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.496+0000 7f58267fc700 1 -- 172.21.15.69:0/2489884418 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58200098c0 con 0x5600a2033870 2023-07-06T20:22:21.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.496+0000 7f58267fc700 1 --2- 172.21.15.69:0/2489884418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 0x5600a203cdf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5818000c00 tx=0x7f581809a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1050b5bb8453cd16 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:22:21.501 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.496+0000 7f580aff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:22:21.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.496+0000 7f580aff5700 1 -- 172.21.15.69:0/2489884418 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58180a2290 con 0x5600a2033870 2023-07-06T20:22:21.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.496+0000 7f580aff5700 1 -- 172.21.15.69:0/2489884418 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f581809a0d0 con 0x5600a2033870 2023-07-06T20:22:21.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.496+0000 7f5840e71b80 1 -- 172.21.15.69:0/2489884418 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600a1edd4f0 con 0x5600a2033870 2023-07-06T20:22:21.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.497+0000 7f580aff5700 1 -- 172.21.15.69:0/2489884418 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f581809e100 con 0x5600a2033870 2023-07-06T20:22:21.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.497+0000 7f5840e71b80 10 client.25453 fetch_fsmap learned FSMap version 29 2023-07-06T20:22:21.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.497+0000 7f5840e71b80 1 -- 172.21.15.69:0/2489884418 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5600a1d9ac60 con 0x5600a2033870 2023-07-06T20:22:21.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.497+0000 7f580aff5700 1 -- 172.21.15.69:0/2489884418 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600a1edd4f0 con 0x5600a2033870 2023-07-06T20:22:21.502 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.497+0000 7f580aff5700 1 -- 172.21.15.69:0/2489884418 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f58180105c0 con 0x5600a2033870 2023-07-06T20:22:21.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.497+0000 7f5840e71b80 10 client.25453 fetch_fsmap finished waiting for FSMap version 29 2023-07-06T20:22:21.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.498+0000 7f5840e71b80 10 client.25453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:22:21.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.498+0000 7f5840e71b80 20 client.25453 populate_metadata read hostname 'smithi069' 2023-07-06T20:22:21.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.498+0000 7f5840e71b80 1 --2- 172.21.15.69:0/2489884418 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5600a20486a0 0x5600a204aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.498+0000 7f5840e71b80 4 client.25453 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:22:21.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.498+0000 7f5840e71b80 1 -- 172.21.15.69:0/2489884418 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600a2001d60 con 0x5600a20486a0 2023-07-06T20:22:21.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.498+0000 7f5826ffd700 1 --2- 172.21.15.69:0/2489884418 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5600a20486a0 0x5600a204aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:21.503 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.499+0000 7f5826ffd700 1 --2- 172.21.15.69:0/2489884418 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5600a20486a0 0x5600a204aab0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=236342e785ee7504 server_cookie=c00afa98216b8d6a in_seq=0 out_seq=0 2023-07-06T20:22:21.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.499+0000 7f580aff5700 10 client.25453 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:22:21.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.500+0000 7f580aff5700 1 -- 172.21.15.69:0/2489884418 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5600a2001d60 con 0x5600a20486a0 2023-07-06T20:22:21.504 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.500+0000 7f580aff5700 10 client.25453 handle_command_reply: tid=0 2023-07-06T20:22:21.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.500+0000 7f5840e71b80 1 client.25453 shutdown 2023-07-06T20:22:21.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.500+0000 7f5840e71b80 1 -- 172.21.15.69:0/2489884418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 msgr2=0x5600a203cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:21.505 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.500+0000 7f5840e71b80 1 --2- 172.21.15.69:0/2489884418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 0x5600a203cdf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5818000c00 tx=0x7f581809a610 comp rx=0 tx=0).stop 2023-07-06T20:22:21.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.506+0000 7f5840e71b80 1 -- 172.21.15.69:0/2489884418 shutdown_connections 2023-07-06T20:22:21.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.506+0000 7f5840e71b80 1 --2- 172.21.15.69:0/2489884418 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5600a20486a0 0x5600a204aab0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.506+0000 7f5840e71b80 1 --2- 172.21.15.69:0/2489884418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2034390 0x5600a2043e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.506+0000 7f5840e71b80 1 --2- 172.21.15.69:0/2489884418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a203c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.506+0000 7f5840e71b80 1 --2- 172.21.15.69:0/2489884418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 0x5600a203cdf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.508 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.506+0000 7f5840e71b80 1 -- 172.21.15.69:0/2489884418 >> 172.21.15.69:0/2489884418 conn(0x5600a2028640 msgr2=0x5600a202d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:22:21.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.506+0000 7f580aff5700 0 client.25453 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:22:21.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.506+0000 7f5840e71b80 1 -- 172.21.15.69:0/2489884418 shutdown_connections 2023-07-06T20:22:21.509 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.506+0000 7f5840e71b80 1 -- 172.21.15.69:0/2489884418 wait complete. 2023-07-06T20:22:21.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:21 smithi069 ceph-mon[126399]: pgmap v418: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 MiB/s wr, 4.48k op/s 2023-07-06T20:22:21.510 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.506+0000 7f5840e71b80 20 client.25453 trim_cache size 0 max 16384 2023-07-06T20:22:21.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.538+0000 7f5840e71b80 1 Processor -- start 2023-07-06T20:22:21.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.538+0000 7f5840e71b80 1 -- start start 2023-07-06T20:22:21.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.538+0000 7f5840e71b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a20372d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.538+0000 7f5840e71b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2033870 0x5600a2037810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.538+0000 7f5840e71b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2034390 0x5600a2043df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.538+0000 7f5840e71b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a203bd60 con 0x5600a2033870 2023-07-06T20:22:21.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.538+0000 7f5840e71b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a2048440 con 0x5600a2032f20 2023-07-06T20:22:21.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.538+0000 7f5840e71b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a203da30 con 0x5600a2034390 2023-07-06T20:22:21.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.539+0000 7f58277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2034390 0x5600a2043df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:21.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.539+0000 7f58277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2034390 0x5600a2043df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:34998/0 (socket says 172.21.15.69:34998) 2023-07-06T20:22:21.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.539+0000 7f58277fe700 1 -- 172.21.15.69:0/768049370 learned_addr learned my addr 172.21.15.69:0/768049370 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:22:21.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.539+0000 7f58277fe700 1 -- 172.21.15.69:0/768049370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 msgr2=0x5600a20372d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:22:21.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.539+0000 7f58277fe700 1 --2- 172.21.15.69:0/768049370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a20372d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.539+0000 7f58277fe700 1 -- 172.21.15.69:0/768049370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2033870 msgr2=0x5600a2037810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:21.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.539+0000 7f58277fe700 1 --2- 172.21.15.69:0/768049370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2033870 0x5600a2037810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.539+0000 7f58277fe700 1 -- 172.21.15.69:0/768049370 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a1dd8250 con 0x5600a2034390 2023-07-06T20:22:21.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.540+0000 7f58277fe700 1 --2- 172.21.15.69:0/768049370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2034390 0x5600a2043df0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5820009b00 tx=0x7f5820004a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66512dd59ded3169 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:22:21.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.540+0000 7f580dffb700 1 -- 172.21.15.69:0/768049370 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5820004de0 con 0x5600a2034390 2023-07-06T20:22:21.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.540+0000 7f580dffb700 1 -- 172.21.15.69:0/768049370 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58200054c0 con 0x5600a2034390 2023-07-06T20:22:21.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.540+0000 7f580dffb700 1 -- 172.21.15.69:0/768049370 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582001cc60 con 0x5600a2034390 2023-07-06T20:22:21.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.541+0000 7f5840e71b80 1 -- 172.21.15.69:0/768049370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2034390 msgr2=0x5600a2043df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.541+0000 7f5840e71b80 1 --2- 172.21.15.69:0/768049370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2034390 0x5600a2043df0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5820009b00 tx=0x7f5820004a10 comp rx=0 tx=0).stop 2023-07-06T20:22:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.541+0000 7f5840e71b80 1 -- 172.21.15.69:0/768049370 shutdown_connections 2023-07-06T20:22:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.541+0000 7f5840e71b80 1 --2- 172.21.15.69:0/768049370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2034390 0x5600a2043df0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.541+0000 7f5840e71b80 1 --2- 172.21.15.69:0/768049370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a20372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.541+0000 7f5840e71b80 1 --2- 172.21.15.69:0/768049370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2033870 0x5600a2037810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.541+0000 7f5840e71b80 1 -- 172.21.15.69:0/768049370 >> 172.21.15.69:0/768049370 conn(0x5600a2028640 msgr2=0x5600a203a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:22:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.541+0000 7f5840e71b80 1 -- 172.21.15.69:0/768049370 shutdown_connections 2023-07-06T20:22:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.542+0000 7f5840e71b80 1 -- 172.21.15.69:0/768049370 wait complete. 2023-07-06T20:22:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.542+0000 7f5840e71b80 1 -- start start 2023-07-06T20:22:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.542+0000 7f5840e71b80 1 Processor -- start 2023-07-06T20:22:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.543+0000 7f5840e71b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a2043e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.543+0000 7f5840e71b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 0x5600a20408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.543+0000 7f5840e71b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2034390 0x5600a2040ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.543+0000 7f5840e71b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a209c550 con 0x5600a2034390 2023-07-06T20:22:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.543+0000 7f5840e71b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a209c690 con 0x5600a2032f20 2023-07-06T20:22:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.543+0000 7f5840e71b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a209c800 con 0x5600a2033870 2023-07-06T20:22:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.543+0000 7f5840e71b80 10 client.0 _ll_register_callbacks cb 0x5600a1f21760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:22:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.543+0000 7f58267fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 0x5600a20408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.543+0000 7f58267fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 0x5600a20408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:35002/0 (socket says 172.21.15.69:35002) 2023-07-06T20:22:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.543+0000 7f58267fc700 1 -- 172.21.15.69:0/1938137659 learned_addr learned my addr 172.21.15.69:0/1938137659 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:22:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.543+0000 7f58277fe700 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2034390 0x5600a2040ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.544+0000 7f58267fc700 1 -- 172.21.15.69:0/1938137659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 msgr2=0x5600a2043e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.544+0000 7f5826ffd700 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a2043e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.544+0000 7f58267fc700 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a2043e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.544+0000 7f58267fc700 1 -- 172.21.15.69:0/1938137659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2034390 msgr2=0x5600a2040ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.544+0000 7f58267fc700 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2034390 0x5600a2040ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.544+0000 7f58267fc700 1 -- 172.21.15.69:0/1938137659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600a1db1de0 con 0x5600a2033870 2023-07-06T20:22:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.544+0000 7f58277fe700 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2034390 0x5600a2040ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:22:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.544+0000 7f5826ffd700 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a2043e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:22:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.544+0000 7f58267fc700 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 0x5600a20408b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5818096b40 tx=0x7f5818005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acd8a75cd447332 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:22:21.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.544+0000 7f580aff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:22:21.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.544+0000 7f580aff5700 1 -- 172.21.15.69:0/1938137659 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581809e100 con 0x5600a2033870 2023-07-06T20:22:21.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.545+0000 7f580aff5700 1 -- 172.21.15.69:0/1938137659 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f58180a2520 con 0x5600a2033870 2023-07-06T20:22:21.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.545+0000 7f5840e71b80 1 -- 172.21.15.69:0/1938137659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f582000f040 con 0x5600a2033870 2023-07-06T20:22:21.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.545+0000 7f580aff5700 1 -- 172.21.15.69:0/1938137659 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5818010430 con 0x5600a2033870 2023-07-06T20:22:21.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.545+0000 7f5840e71b80 10 client.25459 fetch_fsmap learned FSMap version 29 2023-07-06T20:22:21.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.545+0000 7f5840e71b80 1 -- 172.21.15.69:0/1938137659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5600a1db02b0 con 0x5600a2033870 2023-07-06T20:22:21.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.546+0000 7f580aff5700 1 -- 172.21.15.69:0/1938137659 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f582000f040 con 0x5600a2033870 2023-07-06T20:22:21.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.546+0000 7f580aff5700 1 -- 172.21.15.69:0/1938137659 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f581809aeb0 con 0x5600a2033870 2023-07-06T20:22:21.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.546+0000 7f5840e71b80 10 client.25459 fetch_fsmap finished waiting for FSMap version 29 2023-07-06T20:22:21.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.546+0000 7f5840e71b80 10 client.25459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:22:21.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.546+0000 7f5840e71b80 20 client.25459 populate_metadata read hostname 'smithi069' 2023-07-06T20:22:21.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.546+0000 7f5840e71b80 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5600a209def0 0x5600a20a0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:21.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.546+0000 7f5840e71b80 4 client.25459 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:22:21.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.546+0000 7f5840e71b80 1 -- 172.21.15.69:0/1938137659 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5600a1f7ceb0 con 0x5600a209def0 2023-07-06T20:22:21.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.546+0000 7f5826ffd700 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5600a209def0 0x5600a20a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:21.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f5826ffd700 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5600a209def0 0x5600a20a0300 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08a6f545f0348a server_cookie=c106ba0c3d2941f4 in_seq=0 out_seq=0 2023-07-06T20:22:21.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f580aff5700 10 client.25459 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:22:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f580aff5700 1 -- 172.21.15.69:0/1938137659 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5600a1f7ceb0 con 0x5600a209def0 2023-07-06T20:22:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f580aff5700 10 client.25459 handle_command_reply: tid=0 2023-07-06T20:22:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f5840e71b80 1 client.25459 shutdown 2023-07-06T20:22:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f5840e71b80 1 -- 172.21.15.69:0/1938137659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 msgr2=0x5600a20408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f5840e71b80 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 0x5600a20408b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5818096b40 tx=0x7f5818005cf0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f5840e71b80 1 -- 172.21.15.69:0/1938137659 shutdown_connections 2023-07-06T20:22:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f5840e71b80 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5600a209def0 0x5600a20a0300 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f5840e71b80 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a2034390 0x5600a2040ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f5840e71b80 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600a2032f20 0x5600a2043e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f5840e71b80 1 --2- 172.21.15.69:0/1938137659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600a2033870 0x5600a20408b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f5840e71b80 1 -- 172.21.15.69:0/1938137659 >> 172.21.15.69:0/1938137659 conn(0x5600a2028640 msgr2=0x5600a20372d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:22:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.547+0000 7f580aff5700 0 client.25459 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:22:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.548+0000 7f5840e71b80 1 -- 172.21.15.69:0/1938137659 shutdown_connections 2023-07-06T20:22:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.548+0000 7f5840e71b80 1 -- 172.21.15.69:0/1938137659 wait complete. 2023-07-06T20:22:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.548+0000 7f5840e71b80 20 client.25459 trim_cache size 0 max 16384 2023-07-06T20:22:21.558 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:22:21.558 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:22:21.559 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:22:21.559 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:22:21.559 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:22:21.559 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:22:21.559 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:22:21.559 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:22:21.559 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:22:21.560 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:22:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.549+0000 7f582d5ff700 1 -- 172.21.15.69:0/1440975988 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f58100a11d0 msgr2=0x7f58100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.549+0000 7f582d5ff700 1 --2- 172.21.15.69:0/1440975988 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f58100a11d0 0x7f58100a3680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f581c00b620 tx=0x7f581c005fb0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.549+0000 7f582d5ff700 1 -- 172.21.15.69:0/1440975988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58280ffd00 msgr2=0x7f5828193d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.549+0000 7f582d5ff700 1 --2- 172.21.15.69:0/1440975988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58280ffd00 0x7f5828193d20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f581800e9a0 tx=0x7f581800ed60 comp rx=0 tx=0).stop 2023-07-06T20:22:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.549+0000 7f582d5ff700 1 -- 172.21.15.69:0/1440975988 shutdown_connections 2023-07-06T20:22:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.549+0000 7f582d5ff700 1 --2- 172.21.15.69:0/1440975988 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f58100a11d0 0x7f58100a3680 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.549+0000 7f582d5ff700 1 --2- 172.21.15.69:0/1440975988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5828100690 0x7f582819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.549+0000 7f582d5ff700 1 --2- 172.21.15.69:0/1440975988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58280feb10 0x7f58281937e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.549+0000 7f582d5ff700 1 --2- 172.21.15.69:0/1440975988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58280ffd00 0x7f5828193d20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.549+0000 7f582d5ff700 1 -- 172.21.15.69:0/1440975988 >> 172.21.15.69:0/1440975988 conn(0x7f58280fa130 msgr2=0x7f58280fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:22:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.549+0000 7f582d5ff700 1 -- 172.21.15.69:0/1440975988 shutdown_connections 2023-07-06T20:22:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:21.555+0000 7f582d5ff700 1 -- 172.21.15.69:0/1440975988 wait complete. 2023-07-06T20:22:21.593 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:22:23.514 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:23 smithi069 ceph-mon[126399]: pgmap v419: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 80 MiB/s wr, 4.28k op/s 2023-07-06T20:22:23.514 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:23 smithi069 ceph-mon[126399]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:22:23.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:23 smithi156 ceph-mon[129280]: pgmap v419: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 80 MiB/s wr, 4.28k op/s 2023-07-06T20:22:23.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:23 smithi156 ceph-mon[129280]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:22:23.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:23 smithi098 ceph-mon[128968]: pgmap v419: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 80 MiB/s wr, 4.28k op/s 2023-07-06T20:22:23.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:23 smithi098 ceph-mon[128968]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-06T20:22:24.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 35.0 s, 2176.1 tps, lat 45.953 ms stddev 73.595 2023-07-06T20:22:25.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:25 smithi069 ceph-mon[126399]: pgmap v420: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 85 MiB/s wr, 4.94k op/s 2023-07-06T20:22:25.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:25 smithi156 ceph-mon[129280]: pgmap v420: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 85 MiB/s wr, 4.94k op/s 2023-07-06T20:22:25.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:25 smithi098 ceph-mon[128968]: pgmap v420: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 85 MiB/s wr, 4.94k op/s 2023-07-06T20:22:27.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:27 smithi069 ceph-mon[126399]: pgmap v421: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 84 MiB/s wr, 5.22k op/s 2023-07-06T20:22:27.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:27 smithi156 ceph-mon[129280]: pgmap v421: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 84 MiB/s wr, 5.22k op/s 2023-07-06T20:22:27.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:27 smithi098 ceph-mon[128968]: pgmap v421: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 84 MiB/s wr, 5.22k op/s 2023-07-06T20:22:29.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 40.0 s, 3001.0 tps, lat 32.474 ms stddev 45.054 2023-07-06T20:22:29.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:29 smithi156 ceph-mon[129280]: pgmap v422: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 90 MiB/s wr, 5.56k op/s 2023-07-06T20:22:29.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:29 smithi098 ceph-mon[128968]: pgmap v422: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 90 MiB/s wr, 5.56k op/s 2023-07-06T20:22:29.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:29 smithi069 ceph-mon[126399]: pgmap v422: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 90 MiB/s wr, 5.56k op/s 2023-07-06T20:22:31.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:31 smithi156 ceph-mon[129280]: pgmap v423: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 82 MiB/s wr, 5.07k op/s 2023-07-06T20:22:31.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:31 smithi098 ceph-mon[128968]: pgmap v423: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 82 MiB/s wr, 5.07k op/s 2023-07-06T20:22:31.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:31 smithi069 ceph-mon[126399]: pgmap v423: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 82 MiB/s wr, 5.07k op/s 2023-07-06T20:22:33.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:33 smithi156 ceph-mon[129280]: pgmap v424: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 88 MiB/s wr, 5.37k op/s 2023-07-06T20:22:33.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:33 smithi098 ceph-mon[128968]: pgmap v424: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 88 MiB/s wr, 5.37k op/s 2023-07-06T20:22:33.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:33 smithi069 ceph-mon[126399]: pgmap v424: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 88 MiB/s wr, 5.37k op/s 2023-07-06T20:22:34.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 45.0 s, 2521.7 tps, lat 40.003 ms stddev 42.445 2023-07-06T20:22:35.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:35 smithi156 ceph-mon[129280]: pgmap v425: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 85 MiB/s wr, 5.34k op/s 2023-07-06T20:22:35.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:35 smithi098 ceph-mon[128968]: pgmap v425: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 85 MiB/s wr, 5.34k op/s 2023-07-06T20:22:35.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:35 smithi069 ceph-mon[126399]: pgmap v425: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 85 MiB/s wr, 5.34k op/s 2023-07-06T20:22:36.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:36 smithi098 ceph-mon[128968]: pgmap v426: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 81 MiB/s wr, 5.28k op/s 2023-07-06T20:22:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:36 smithi069 ceph-mon[126399]: pgmap v426: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 81 MiB/s wr, 5.28k op/s 2023-07-06T20:22:36.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:36 smithi156 ceph-mon[129280]: pgmap v426: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 81 MiB/s wr, 5.28k op/s 2023-07-06T20:22:39.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:38 smithi069 ceph-mon[126399]: pgmap v427: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 84 MiB/s wr, 5.23k op/s 2023-07-06T20:22:39.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:38 smithi156 ceph-mon[129280]: pgmap v427: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 84 MiB/s wr, 5.23k op/s 2023-07-06T20:22:39.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:38 smithi098 ceph-mon[128968]: pgmap v427: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 84 MiB/s wr, 5.23k op/s 2023-07-06T20:22:39.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 50.0 s, 2935.9 tps, lat 34.605 ms stddev 34.445 2023-07-06T20:22:41.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:40 smithi156 ceph-mon[129280]: pgmap v428: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 72 MiB/s wr, 4.27k op/s 2023-07-06T20:22:41.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:40 smithi098 ceph-mon[128968]: pgmap v428: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 72 MiB/s wr, 4.27k op/s 2023-07-06T20:22:41.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:40 smithi069 ceph-mon[126399]: pgmap v428: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 72 MiB/s wr, 4.27k op/s 2023-07-06T20:22:43.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:42 smithi156 ceph-mon[129280]: pgmap v429: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 MiB/s wr, 3.67k op/s 2023-07-06T20:22:43.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:42 smithi098 ceph-mon[128968]: pgmap v429: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 MiB/s wr, 3.67k op/s 2023-07-06T20:22:43.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:42 smithi069 ceph-mon[126399]: pgmap v429: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 MiB/s wr, 3.67k op/s 2023-07-06T20:22:44.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:43 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:22:44.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:43 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:22:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 55.0 s, 2686.8 tps, lat 33.305 ms stddev 31.388 2023-07-06T20:22:44.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:43 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:22:45.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:44 smithi069 ceph-mon[126399]: pgmap v430: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 64 MiB/s wr, 3.19k op/s 2023-07-06T20:22:45.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:44 smithi156 ceph-mon[129280]: pgmap v430: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 64 MiB/s wr, 3.19k op/s 2023-07-06T20:22:45.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:44 smithi098 ceph-mon[128968]: pgmap v430: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 64 MiB/s wr, 3.19k op/s 2023-07-06T20:22:46.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:45 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:22:46.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:45 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:22:46.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:45 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:22:46.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:45 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:22:46.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:45 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:22:46.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:45 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:22:46.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:45 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:22:46.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:45 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:22:46.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:45 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:22:47.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:46 smithi069 ceph-mon[126399]: pgmap v431: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 68 MiB/s wr, 3.72k op/s 2023-07-06T20:22:47.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:46 smithi156 ceph-mon[129280]: pgmap v431: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 68 MiB/s wr, 3.72k op/s 2023-07-06T20:22:47.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:46 smithi098 ceph-mon[128968]: pgmap v431: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 68 MiB/s wr, 3.72k op/s 2023-07-06T20:22:49.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 60.0 s, 2466.2 tps, lat 44.906 ms stddev 71.069 2023-07-06T20:22:49.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:48 smithi069 ceph-mon[126399]: pgmap v432: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 77 MiB/s wr, 4.57k op/s 2023-07-06T20:22:49.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:48 smithi156 ceph-mon[129280]: pgmap v432: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 77 MiB/s wr, 4.57k op/s 2023-07-06T20:22:49.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:48 smithi098 ceph-mon[128968]: pgmap v432: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 77 MiB/s wr, 4.57k op/s 2023-07-06T20:22:51.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:50 smithi069 ceph-mon[126399]: pgmap v433: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 73 MiB/s wr, 4.50k op/s 2023-07-06T20:22:51.360 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:50 smithi156 ceph-mon[129280]: pgmap v433: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 73 MiB/s wr, 4.50k op/s 2023-07-06T20:22:51.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:50 smithi098 ceph-mon[128968]: pgmap v433: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 73 MiB/s wr, 4.50k op/s 2023-07-06T20:22:51.595 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:22:52.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.010+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1401702454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff34107dc0 msgr2=0x7eff3410e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:52.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.010+0000 7eff3a1ff700 1 --2- 172.21.15.69:0/1401702454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff34107dc0 0x7eff3410e680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7eff28009990 tx=0x7eff28009ca0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.014 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.012+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1401702454 shutdown_connections 2023-07-06T20:22:52.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.012+0000 7eff3a1ff700 1 --2- 172.21.15.69:0/1401702454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff34107dc0 0x7eff3410e680 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.012+0000 7eff3a1ff700 1 --2- 172.21.15.69:0/1401702454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff34106240 0x7eff34106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.012+0000 7eff3a1ff700 1 --2- 172.21.15.69:0/1401702454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff34107430 0x7eff34107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.012+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1401702454 >> 172.21.15.69:0/1401702454 conn(0x7eff34075290 msgr2=0x7eff340776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:22:52.016 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.012+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1401702454 shutdown_connections 2023-07-06T20:22:52.016 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.012+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1401702454 wait complete. 2023-07-06T20:22:52.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.014+0000 7eff3a1ff700 1 Processor -- start 2023-07-06T20:22:52.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.014+0000 7eff3a1ff700 1 -- start start 2023-07-06T20:22:52.022 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.014+0000 7eff3a1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff34106240 0x7eff3406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.014+0000 7eff3a1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff34107430 0x7eff3406f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.014+0000 7eff3a1ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff34107dc0 0x7eff341ac100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.014+0000 7eff3a1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff34072440 con 0x7eff34107430 2023-07-06T20:22:52.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.014+0000 7eff3a1ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340725b0 con 0x7eff34107dc0 2023-07-06T20:22:52.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.014+0000 7eff3a1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff341ac640 con 0x7eff34106240 2023-07-06T20:22:52.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.018+0000 7eff32ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff34107430 0x7eff3406f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.023 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.018+0000 7eff32ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff34107430 0x7eff3406f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39888/0 (socket says 172.21.15.69:39888) 2023-07-06T20:22:52.024 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.018+0000 7eff32ffd700 1 -- 172.21.15.69:0/1857633694 learned_addr learned my addr 172.21.15.69:0/1857633694 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:22:52.030 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.021+0000 7eff33fff700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff34107dc0 0x7eff341ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.031 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.022+0000 7eff337fe700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff34106240 0x7eff3406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.034 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.031+0000 7eff33fff700 1 -- 172.21.15.69:0/1857633694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff34106240 msgr2=0x7eff3406ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:52.035 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.032+0000 7eff33fff700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff34106240 0x7eff3406ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.035 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.032+0000 7eff33fff700 1 -- 172.21.15.69:0/1857633694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff34107430 msgr2=0x7eff3406f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:52.035 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.032+0000 7eff33fff700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff34107430 0x7eff3406f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.035 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.032+0000 7eff33fff700 1 -- 172.21.15.69:0/1857633694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280095f0 con 0x7eff34107dc0 2023-07-06T20:22:52.036 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.034+0000 7eff33fff700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff34107dc0 0x7eff341ac100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7eff280059d0 tx=0x7eff28005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:22:52.038 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.036+0000 7eff30ff9700 1 -- 172.21.15.69:0/1857633694 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2801d070 con 0x7eff34107dc0 2023-07-06T20:22:52.038 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.036+0000 7eff30ff9700 1 -- 172.21.15.69:0/1857633694 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff28021410 con 0x7eff34107dc0 2023-07-06T20:22:52.038 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.036+0000 7eff30ff9700 1 -- 172.21.15.69:0/1857633694 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff28017650 con 0x7eff34107dc0 2023-07-06T20:22:52.038 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.036+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1857633694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff341ac780 con 0x7eff34107dc0 2023-07-06T20:22:52.039 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.036+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1857633694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff341acbb0 con 0x7eff34107dc0 2023-07-06T20:22:52.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.042+0000 7eff4da80b80 1 Processor -- start 2023-07-06T20:22:52.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.043+0000 7eff4da80b80 1 -- start start 2023-07-06T20:22:52.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.046+0000 7eff4da80b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aecff20 0x55ba2aed0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.046+0000 7eff4da80b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aed0870 0x55ba2aed0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.046+0000 7eff4da80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aed1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.046+0000 7eff4da80b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2add4cb0 con 0x55ba2aecff20 2023-07-06T20:22:52.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.046+0000 7eff4da80b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2ae27a40 con 0x55ba2aed0870 2023-07-06T20:22:52.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.046+0000 7eff4da80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2ae20ed0 con 0x55ba2aed1390 2023-07-06T20:22:52.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.048+0000 7eff337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aed0870 0x55ba2aed0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.048+0000 7eff337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aed0870 0x55ba2aed0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:60966/0 (socket says 172.21.15.69:60966) 2023-07-06T20:22:52.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.048+0000 7eff337fe700 1 -- 172.21.15.69:0/3063316158 learned_addr learned my addr 172.21.15.69:0/3063316158 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:22:52.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.050+0000 7eff32ffd700 1 --2- 172.21.15.69:0/3063316158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aed1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.050+0000 7eff30ff9700 1 -- 172.21.15.69:0/1857633694 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7eff2800fb10 con 0x7eff34107dc0 2023-07-06T20:22:52.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.051+0000 7eff32ffd700 1 --2- 172.21.15.69:0/3063316158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aecff20 0x55ba2aed0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.051+0000 7eff32ffd700 1 -- 172.21.15.69:0/3063316158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aed0870 msgr2=0x55ba2aed0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:52.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.051+0000 7eff32ffd700 1 --2- 172.21.15.69:0/3063316158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aed0870 0x55ba2aed0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.051+0000 7eff32ffd700 1 -- 172.21.15.69:0/3063316158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aecff20 msgr2=0x55ba2aed0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:52.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.051+0000 7eff32ffd700 1 --2- 172.21.15.69:0/3063316158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aecff20 0x55ba2aed0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.051+0000 7eff32ffd700 1 -- 172.21.15.69:0/3063316158 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba2ac4ede0 con 0x55ba2aed1390 2023-07-06T20:22:52.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.052+0000 7eff32ffd700 1 --2- 172.21.15.69:0/3063316158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aed1830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7eff2400bfa0 tx=0x7eff2400b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ccdc5591f7a44c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:22:52.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.056+0000 7eff30ff9700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7eff1c0a1250 0x7eff1c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.056+0000 7eff30ff9700 1 -- 172.21.15.69:0/1857633694 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7eff280cdbc0 con 0x7eff34107dc0 2023-07-06T20:22:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.056+0000 7eff19ffb700 1 -- 172.21.15.69:0/3063316158 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24013e30 con 0x55ba2aed1390 2023-07-06T20:22:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.056+0000 7eff19ffb700 1 -- 172.21.15.69:0/3063316158 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff24004cf0 con 0x55ba2aed1390 2023-07-06T20:22:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.056+0000 7eff19ffb700 1 -- 172.21.15.69:0/3063316158 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240116b0 con 0x55ba2aed1390 2023-07-06T20:22:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.057+0000 7eff4da80b80 1 -- 172.21.15.69:0/3063316158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 msgr2=0x55ba2aed1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.057+0000 7eff4da80b80 1 --2- 172.21.15.69:0/3063316158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aed1830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7eff2400bfa0 tx=0x7eff2400b720 comp rx=0 tx=0).stop 2023-07-06T20:22:52.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.057+0000 7eff4da80b80 1 -- 172.21.15.69:0/3063316158 shutdown_connections 2023-07-06T20:22:52.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.057+0000 7eff4da80b80 1 --2- 172.21.15.69:0/3063316158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aed1830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.057+0000 7eff4da80b80 1 --2- 172.21.15.69:0/3063316158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aecff20 0x55ba2aed0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.057+0000 7eff4da80b80 1 --2- 172.21.15.69:0/3063316158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aed0870 0x55ba2aed0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.057+0000 7eff4da80b80 1 -- 172.21.15.69:0/3063316158 >> 172.21.15.69:0/3063316158 conn(0x55ba2aec5640 msgr2=0x55ba2aeca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:22:52.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.058+0000 7eff4da80b80 1 -- 172.21.15.69:0/3063316158 shutdown_connections 2023-07-06T20:22:52.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.058+0000 7eff337fe700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7eff1c0a1250 0x7eff1c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.058+0000 7eff4da80b80 1 -- 172.21.15.69:0/3063316158 wait complete. 2023-07-06T20:22:52.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.058+0000 7eff4da80b80 1 -- start start 2023-07-06T20:22:52.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.059+0000 7eff4da80b80 1 Processor -- start 2023-07-06T20:22:52.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.059+0000 7eff4da80b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 0x55ba2aed99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.059+0000 7eff4da80b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aed0870 0x55ba2aed9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.059+0000 7eff4da80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aee0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.059+0000 7eff4da80b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2adc0ac0 con 0x55ba2aed0870 2023-07-06T20:22:52.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.059+0000 7eff4da80b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2adc07c0 con 0x55ba2aecff20 2023-07-06T20:22:52.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.059+0000 7eff4da80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2aca4920 con 0x55ba2aed1390 2023-07-06T20:22:52.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.060+0000 7eff4da80b80 10 client.0 _ll_register_callbacks cb 0x55ba2adbe760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:22:52.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.060+0000 7eff337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 0x55ba2aed99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.060+0000 7eff337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 0x55ba2aed99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:60978/0 (socket says 172.21.15.69:60978) 2023-07-06T20:22:52.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.060+0000 7eff337fe700 1 -- 172.21.15.69:0/2709274087 learned_addr learned my addr 172.21.15.69:0/2709274087 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:22:52.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.061+0000 7eff337fe700 1 -- 172.21.15.69:0/2709274087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 msgr2=0x55ba2aee0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:52.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.061+0000 7eff337fe700 1 --2- 172.21.15.69:0/2709274087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aee0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.061+0000 7eff337fe700 1 -- 172.21.15.69:0/2709274087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aed0870 msgr2=0x55ba2aed9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:52.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.061+0000 7eff337fe700 1 --2- 172.21.15.69:0/2709274087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aed0870 0x55ba2aed9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.061+0000 7eff337fe700 1 -- 172.21.15.69:0/2709274087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2400be10 con 0x55ba2aecff20 2023-07-06T20:22:52.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.061+0000 7eff337fe700 1 --2- 172.21.15.69:0/2709274087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 0x55ba2aed99b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7eff2c01bf30 tx=0x7eff2c0186a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c42cc6489e6bc42c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:22:52.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.062+0000 7eff12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:22:52.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.062+0000 7eff12ffd700 1 -- 172.21.15.69:0/2709274087 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c020070 con 0x55ba2aecff20 2023-07-06T20:22:52.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.062+0000 7eff12ffd700 1 -- 172.21.15.69:0/2709274087 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7eff2c003a00 con 0x55ba2aecff20 2023-07-06T20:22:52.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.062+0000 7eff4da80b80 1 -- 172.21.15.69:0/2709274087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba2ad7a4f0 con 0x55ba2aecff20 2023-07-06T20:22:52.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.062+0000 7eff12ffd700 1 -- 172.21.15.69:0/2709274087 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff2c00e040 con 0x55ba2aecff20 2023-07-06T20:22:52.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.062+0000 7eff4da80b80 10 client.25529 fetch_fsmap learned FSMap version 30 2023-07-06T20:22:52.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.062+0000 7eff4da80b80 1 -- 172.21.15.69:0/2709274087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ba2ac37c60 con 0x55ba2aecff20 2023-07-06T20:22:52.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.063+0000 7eff12ffd700 1 -- 172.21.15.69:0/2709274087 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba2ad7a4f0 con 0x55ba2aecff20 2023-07-06T20:22:52.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.064+0000 7eff12ffd700 1 -- 172.21.15.69:0/2709274087 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7eff2c016430 con 0x55ba2aecff20 2023-07-06T20:22:52.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.064+0000 7eff4da80b80 10 client.25529 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:22:52.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.064+0000 7eff4da80b80 10 client.25529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:22:52.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.064+0000 7eff4da80b80 20 client.25529 populate_metadata read hostname 'smithi069' 2023-07-06T20:22:52.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.064+0000 7eff4da80b80 1 --2- 172.21.15.69:0/2709274087 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2aef6790 0x55ba2aef8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.064+0000 7eff4da80b80 4 client.25529 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:22:52.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.064+0000 7eff4da80b80 1 -- 172.21.15.69:0/2709274087 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba2ae9ed60 con 0x55ba2aef6790 2023-07-06T20:22:52.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.065+0000 7eff32ffd700 1 --2- 172.21.15.69:0/2709274087 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2aef6790 0x55ba2aef8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.066+0000 7eff32ffd700 1 --2- 172.21.15.69:0/2709274087 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2aef6790 0x55ba2aef8ba0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96fd1dc52b691203 server_cookie=d7b82c3699e01904 in_seq=0 out_seq=0 2023-07-06T20:22:52.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.066+0000 7eff12ffd700 10 client.25529 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:22:52.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.066+0000 7eff337fe700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7eff1c0a1250 0x7eff1c0a3700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7eff2c026000 tx=0x7eff2c021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:22:52.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.067+0000 7eff12ffd700 1 -- 172.21.15.69:0/2709274087 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ba2ae9ed60 con 0x55ba2aef6790 2023-07-06T20:22:52.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.067+0000 7eff12ffd700 10 client.25529 handle_command_reply: tid=0 2023-07-06T20:22:52.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.067+0000 7eff4da80b80 1 client.25529 shutdown 2023-07-06T20:22:52.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.067+0000 7eff4da80b80 1 -- 172.21.15.69:0/2709274087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 msgr2=0x55ba2aed99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:52.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.067+0000 7eff4da80b80 1 --2- 172.21.15.69:0/2709274087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 0x55ba2aed99b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7eff2c01bf30 tx=0x7eff2c0186a0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.068+0000 7eff4da80b80 1 -- 172.21.15.69:0/2709274087 shutdown_connections 2023-07-06T20:22:52.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.068+0000 7eff4da80b80 1 --2- 172.21.15.69:0/2709274087 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2aef6790 0x55ba2aef8ba0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.068+0000 7eff4da80b80 1 --2- 172.21.15.69:0/2709274087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aee0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.068+0000 7eff4da80b80 1 --2- 172.21.15.69:0/2709274087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 0x55ba2aed99b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.068+0000 7eff4da80b80 1 --2- 172.21.15.69:0/2709274087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aed0870 0x55ba2aed9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.068+0000 7eff4da80b80 1 -- 172.21.15.69:0/2709274087 >> 172.21.15.69:0/2709274087 conn(0x55ba2aec5640 msgr2=0x55ba2aeca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:22:52.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.068+0000 7eff12ffd700 0 client.25529 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:22:52.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.068+0000 7eff4da80b80 1 -- 172.21.15.69:0/2709274087 shutdown_connections 2023-07-06T20:22:52.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.068+0000 7eff4da80b80 1 -- 172.21.15.69:0/2709274087 wait complete. 2023-07-06T20:22:52.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.068+0000 7eff4da80b80 20 client.25529 trim_cache size 0 max 16384 2023-07-06T20:22:52.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.107+0000 7eff4da80b80 1 Processor -- start 2023-07-06T20:22:52.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.108+0000 7eff4da80b80 1 -- start start 2023-07-06T20:22:52.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.108+0000 7eff4da80b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 0x55ba2aeca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.108+0000 7eff4da80b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aed0870 0x55ba2aee0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.108+0000 7eff4da80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aef2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.108+0000 7eff4da80b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2aecb640 con 0x55ba2aed0870 2023-07-06T20:22:52.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.108+0000 7eff4da80b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2aed1c70 con 0x55ba2aecff20 2023-07-06T20:22:52.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.108+0000 7eff4da80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2aef6300 con 0x55ba2aed1390 2023-07-06T20:22:52.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.108+0000 7eff337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 0x55ba2aeca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.108+0000 7eff337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 0x55ba2aeca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:60988/0 (socket says 172.21.15.69:60988) 2023-07-06T20:22:52.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.108+0000 7eff337fe700 1 -- 172.21.15.69:0/1908377569 learned_addr learned my addr 172.21.15.69:0/1908377569 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:22:52.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.109+0000 7eff33fff700 1 --2- 172.21.15.69:0/1908377569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aef2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.109+0000 7eff33fff700 1 -- 172.21.15.69:0/1908377569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 msgr2=0x55ba2aeca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:52.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.109+0000 7eff33fff700 1 --2- 172.21.15.69:0/1908377569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 0x55ba2aeca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.109+0000 7eff33fff700 1 -- 172.21.15.69:0/1908377569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aed0870 msgr2=0x55ba2aee0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:52.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.109+0000 7eff33fff700 1 --2- 172.21.15.69:0/1908377569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aed0870 0x55ba2aee0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.109+0000 7eff33fff700 1 -- 172.21.15.69:0/1908377569 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba2aecabd0 con 0x55ba2aed1390 2023-07-06T20:22:52.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.110+0000 7eff33fff700 1 --2- 172.21.15.69:0/1908377569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aef2830 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7eff28013ff0 tx=0x7eff2809ab90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf9cd40823d875b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:22:52.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.110+0000 7eff1a7fc700 1 -- 172.21.15.69:0/1908377569 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2809afd0 con 0x55ba2aed1390 2023-07-06T20:22:52.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.110+0000 7eff1a7fc700 1 -- 172.21.15.69:0/1908377569 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff2809f000 con 0x55ba2aed1390 2023-07-06T20:22:52.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.110+0000 7eff1a7fc700 1 -- 172.21.15.69:0/1908377569 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2809b500 con 0x55ba2aed1390 2023-07-06T20:22:52.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.111+0000 7eff4da80b80 1 -- 172.21.15.69:0/1908377569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 msgr2=0x55ba2aef2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:52.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.111+0000 7eff4da80b80 1 --2- 172.21.15.69:0/1908377569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aef2830 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7eff28013ff0 tx=0x7eff2809ab90 comp rx=0 tx=0).stop 2023-07-06T20:22:52.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.112+0000 7eff4da80b80 1 -- 172.21.15.69:0/1908377569 shutdown_connections 2023-07-06T20:22:52.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.112+0000 7eff4da80b80 1 --2- 172.21.15.69:0/1908377569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aef2830 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.112+0000 7eff4da80b80 1 --2- 172.21.15.69:0/1908377569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aecff20 0x55ba2aeca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.112+0000 7eff4da80b80 1 --2- 172.21.15.69:0/1908377569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aed0870 0x55ba2aee0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.112+0000 7eff4da80b80 1 -- 172.21.15.69:0/1908377569 >> 172.21.15.69:0/1908377569 conn(0x55ba2aec5640 msgr2=0x55ba2aed78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:22:52.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.112+0000 7eff4da80b80 1 -- 172.21.15.69:0/1908377569 shutdown_connections 2023-07-06T20:22:52.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.112+0000 7eff4da80b80 1 -- 172.21.15.69:0/1908377569 wait complete. 2023-07-06T20:22:52.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.112+0000 7eff4da80b80 1 -- start start 2023-07-06T20:22:52.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.112+0000 7eff4da80b80 1 Processor -- start 2023-07-06T20:22:52.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.113+0000 7eff4da80b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aecff20 0x55ba2aee06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.113+0000 7eff4da80b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aed0870 0x55ba2aee0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.113+0000 7eff4da80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aedd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.113+0000 7eff4da80b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2aedd6a0 con 0x55ba2aecff20 2023-07-06T20:22:52.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.113+0000 7eff4da80b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2aedd810 con 0x55ba2aed0870 2023-07-06T20:22:52.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.113+0000 7eff4da80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2aedd9d0 con 0x55ba2aed1390 2023-07-06T20:22:52.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.113+0000 7eff4da80b80 10 client.0 _ll_register_callbacks cb 0x7eff2c01b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:22:52.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.114+0000 7eff33fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aedd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.114+0000 7eff33fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aedd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:55282/0 (socket says 172.21.15.69:55282) 2023-07-06T20:22:52.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.114+0000 7eff33fff700 1 -- 172.21.15.69:0/677627186 learned_addr learned my addr 172.21.15.69:0/677627186 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:22:52.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.114+0000 7eff32ffd700 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aed0870 0x55ba2aee0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.114+0000 7eff337fe700 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aecff20 0x55ba2aee06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.114+0000 7eff32ffd700 1 -- 172.21.15.69:0/677627186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 msgr2=0x55ba2aedd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:52.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.114+0000 7eff32ffd700 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aedd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.114+0000 7eff32ffd700 1 -- 172.21.15.69:0/677627186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aecff20 msgr2=0x55ba2aee06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:22:52.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.114+0000 7eff32ffd700 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aecff20 0x55ba2aee06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.114+0000 7eff32ffd700 1 -- 172.21.15.69:0/677627186 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff341acbb0 con 0x55ba2aed0870 2023-07-06T20:22:52.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.115+0000 7eff32ffd700 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aed0870 0x55ba2aee0c00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55ba2aef9480 tx=0x7eff24005f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e1e37bf8b488d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:22:52.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.115+0000 7eff12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:22:52.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.115+0000 7eff12ffd700 1 -- 172.21.15.69:0/677627186 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24024040 con 0x55ba2aed0870 2023-07-06T20:22:52.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.115+0000 7eff4da80b80 1 -- 172.21.15.69:0/677627186 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff24004cf0 con 0x55ba2aed0870 2023-07-06T20:22:52.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.115+0000 7eff12ffd700 1 -- 172.21.15.69:0/677627186 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7eff24023040 con 0x55ba2aed0870 2023-07-06T20:22:52.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.115+0000 7eff12ffd700 1 -- 172.21.15.69:0/677627186 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff240056d0 con 0x55ba2aed0870 2023-07-06T20:22:52.126 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.116+0000 7eff4da80b80 10 client.25535 fetch_fsmap learned FSMap version 30 2023-07-06T20:22:52.126 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.116+0000 7eff4da80b80 1 -- 172.21.15.69:0/677627186 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ba2ac75250 con 0x55ba2aed0870 2023-07-06T20:22:52.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.116+0000 7eff12ffd700 1 -- 172.21.15.69:0/677627186 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff24004cf0 con 0x55ba2aed0870 2023-07-06T20:22:52.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.117+0000 7eff12ffd700 1 -- 172.21.15.69:0/677627186 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7eff24004710 con 0x55ba2aed0870 2023-07-06T20:22:52.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.117+0000 7eff4da80b80 10 client.25535 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:22:52.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.117+0000 7eff4da80b80 10 client.25535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:22:52.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.117+0000 7eff4da80b80 20 client.25535 populate_metadata read hostname 'smithi069' 2023-07-06T20:22:52.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.117+0000 7eff4da80b80 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2af4ba70 0x55ba2af4de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:22:52.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.117+0000 7eff4da80b80 4 client.25535 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:22:52.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.117+0000 7eff4da80b80 1 -- 172.21.15.69:0/677627186 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba2ae19eb0 con 0x55ba2af4ba70 2023-07-06T20:22:52.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.117+0000 7eff337fe700 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2af4ba70 0x55ba2af4de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:22:52.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.117+0000 7eff12ffd700 10 client.25535 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:22:52.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.118+0000 7eff337fe700 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2af4ba70 0x55ba2af4de80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f36b6e905a097e server_cookie=aa5da634ab21a49e in_seq=0 out_seq=0 2023-07-06T20:22:52.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.118+0000 7eff12ffd700 1 -- 172.21.15.69:0/677627186 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ba2ae19eb0 con 0x55ba2af4ba70 2023-07-06T20:22:52.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.118+0000 7eff12ffd700 10 client.25535 handle_command_reply: tid=0 2023-07-06T20:22:52.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.118+0000 7eff4da80b80 1 client.25535 shutdown 2023-07-06T20:22:52.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.119+0000 7eff4da80b80 1 -- 172.21.15.69:0/677627186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aed0870 msgr2=0x55ba2aee0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:52.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.119+0000 7eff4da80b80 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aed0870 0x55ba2aee0c00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55ba2aef9480 tx=0x7eff24005f50 comp rx=0 tx=0).stop 2023-07-06T20:22:52.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.119+0000 7eff4da80b80 1 -- 172.21.15.69:0/677627186 shutdown_connections 2023-07-06T20:22:52.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.119+0000 7eff4da80b80 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2af4ba70 0x55ba2af4de80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.119+0000 7eff4da80b80 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2aed1390 0x55ba2aedd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.119+0000 7eff4da80b80 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2aecff20 0x55ba2aee06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.119+0000 7eff4da80b80 1 --2- 172.21.15.69:0/677627186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2aed0870 0x55ba2aee0c00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.119+0000 7eff4da80b80 1 -- 172.21.15.69:0/677627186 >> 172.21.15.69:0/677627186 conn(0x55ba2aec5640 msgr2=0x55ba2aed3890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:22:52.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.119+0000 7eff12ffd700 0 client.25535 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:22:52.130 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.119+0000 7eff4da80b80 1 -- 172.21.15.69:0/677627186 shutdown_connections 2023-07-06T20:22:52.130 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.119+0000 7eff4da80b80 1 -- 172.21.15.69:0/677627186 wait complete. 2023-07-06T20:22:52.130 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.119+0000 7eff4da80b80 20 client.25535 trim_cache size 0 max 16384 2023-07-06T20:22:52.130 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:22:52.130 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:22:52.130 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:22:52.131 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:22:52.131 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:22:52.131 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:22:52.131 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:22:52.131 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:22:52.131 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:22:52.131 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:22:52.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.120+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1857633694 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7eff1c0a1250 msgr2=0x7eff1c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:52.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.120+0000 7eff3a1ff700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7eff1c0a1250 0x7eff1c0a3700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7eff2c026000 tx=0x7eff2c021000 comp rx=0 tx=0).stop 2023-07-06T20:22:52.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.120+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1857633694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff34107dc0 msgr2=0x7eff341ac100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:22:52.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.120+0000 7eff3a1ff700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff34107dc0 0x7eff341ac100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7eff280059d0 tx=0x7eff28005740 comp rx=0 tx=0).stop 2023-07-06T20:22:52.132 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.120+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1857633694 shutdown_connections 2023-07-06T20:22:52.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.120+0000 7eff3a1ff700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7eff1c0a1250 0x7eff1c0a3700 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.120+0000 7eff3a1ff700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff34107dc0 0x7eff341ac100 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.120+0000 7eff3a1ff700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff34106240 0x7eff3406ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.120+0000 7eff3a1ff700 1 --2- 172.21.15.69:0/1857633694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff34107430 0x7eff3406f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:22:52.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.120+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1857633694 >> 172.21.15.69:0/1857633694 conn(0x7eff34075290 msgr2=0x7eff34077490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:22:52.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.120+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1857633694 shutdown_connections 2023-07-06T20:22:52.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:22:52.120+0000 7eff3a1ff700 1 -- 172.21.15.69:0/1857633694 wait complete. 2023-07-06T20:22:52.155 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:22:53.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:52 smithi069 ceph-mon[126399]: pgmap v434: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 74 MiB/s wr, 4.20k op/s 2023-07-06T20:22:53.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:52 smithi156 ceph-mon[129280]: pgmap v434: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 74 MiB/s wr, 4.20k op/s 2023-07-06T20:22:53.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:52 smithi098 ceph-mon[128968]: pgmap v434: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 74 MiB/s wr, 4.20k op/s 2023-07-06T20:22:54.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 65.0 s, 2991.3 tps, lat 33.440 ms stddev 25.036 2023-07-06T20:22:55.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:54 smithi069 ceph-mon[126399]: pgmap v435: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 74 MiB/s wr, 4.24k op/s 2023-07-06T20:22:55.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:54 smithi156 ceph-mon[129280]: pgmap v435: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 74 MiB/s wr, 4.24k op/s 2023-07-06T20:22:55.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:54 smithi098 ceph-mon[128968]: pgmap v435: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 74 MiB/s wr, 4.24k op/s 2023-07-06T20:22:57.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:57 smithi156 ceph-mon[129280]: pgmap v436: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 75 MiB/s wr, 4.47k op/s 2023-07-06T20:22:57.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:57 smithi098 ceph-mon[128968]: pgmap v436: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 75 MiB/s wr, 4.47k op/s 2023-07-06T20:22:57.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:57 smithi069 ceph-mon[126399]: pgmap v436: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 75 MiB/s wr, 4.47k op/s 2023-07-06T20:22:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 70.0 s, 1677.4 tps, lat 59.576 ms stddev 78.636 2023-07-06T20:22:59.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:22:59 smithi156 ceph-mon[129280]: pgmap v437: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 82 MiB/s wr, 5.16k op/s 2023-07-06T20:22:59.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:22:59 smithi098 ceph-mon[128968]: pgmap v437: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 82 MiB/s wr, 5.16k op/s 2023-07-06T20:22:59.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:22:59 smithi069 ceph-mon[126399]: pgmap v437: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 82 MiB/s wr, 5.16k op/s 2023-07-06T20:23:01.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:01 smithi156 ceph-mon[129280]: pgmap v438: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 77 MiB/s wr, 4.79k op/s 2023-07-06T20:23:01.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:01 smithi098 ceph-mon[128968]: pgmap v438: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 77 MiB/s wr, 4.79k op/s 2023-07-06T20:23:01.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:01 smithi069 ceph-mon[126399]: pgmap v438: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 77 MiB/s wr, 4.79k op/s 2023-07-06T20:23:03.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:03 smithi156 ceph-mon[129280]: pgmap v439: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 77 MiB/s wr, 4.54k op/s 2023-07-06T20:23:03.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:03 smithi098 ceph-mon[128968]: pgmap v439: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 77 MiB/s wr, 4.54k op/s 2023-07-06T20:23:03.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:03 smithi069 ceph-mon[126399]: pgmap v439: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 77 MiB/s wr, 4.54k op/s 2023-07-06T20:23:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 75.0 s, 2882.4 tps, lat 34.370 ms stddev 35.586 2023-07-06T20:23:05.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:05 smithi069 ceph-mon[126399]: pgmap v440: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 73 MiB/s wr, 4.50k op/s 2023-07-06T20:23:05.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:05 smithi156 ceph-mon[129280]: pgmap v440: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 73 MiB/s wr, 4.50k op/s 2023-07-06T20:23:05.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:05 smithi098 ceph-mon[128968]: pgmap v440: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 73 MiB/s wr, 4.50k op/s 2023-07-06T20:23:07.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:07 smithi069 ceph-mon[126399]: pgmap v441: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 68 MiB/s wr, 4.44k op/s 2023-07-06T20:23:07.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:07 smithi156 ceph-mon[129280]: pgmap v441: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 68 MiB/s wr, 4.44k op/s 2023-07-06T20:23:07.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:07 smithi098 ceph-mon[128968]: pgmap v441: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 68 MiB/s wr, 4.44k op/s 2023-07-06T20:23:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 80.0 s, 2123.2 tps, lat 46.000 ms stddev 61.291 2023-07-06T20:23:09.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:09 smithi156 ceph-mon[129280]: pgmap v442: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 75 MiB/s wr, 4.50k op/s 2023-07-06T20:23:09.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:09 smithi098 ceph-mon[128968]: pgmap v442: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 75 MiB/s wr, 4.50k op/s 2023-07-06T20:23:09.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:09 smithi069 ceph-mon[126399]: pgmap v442: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 75 MiB/s wr, 4.50k op/s 2023-07-06T20:23:11.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:11 smithi156 ceph-mon[129280]: pgmap v443: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 68 MiB/s wr, 3.94k op/s 2023-07-06T20:23:11.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:11 smithi098 ceph-mon[128968]: pgmap v443: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 68 MiB/s wr, 3.94k op/s 2023-07-06T20:23:11.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:11 smithi069 ceph-mon[126399]: pgmap v443: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 68 MiB/s wr, 3.94k op/s 2023-07-06T20:23:13.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:13 smithi156 ceph-mon[129280]: pgmap v444: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 70 MiB/s wr, 3.98k op/s 2023-07-06T20:23:13.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:13 smithi098 ceph-mon[128968]: pgmap v444: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 70 MiB/s wr, 3.98k op/s 2023-07-06T20:23:13.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:13 smithi069 ceph-mon[126399]: pgmap v444: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 70 MiB/s wr, 3.98k op/s 2023-07-06T20:23:14.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 85.0 s, 3309.9 tps, lat 31.191 ms stddev 31.852 2023-07-06T20:23:15.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:15 smithi156 ceph-mon[129280]: pgmap v445: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 68 MiB/s wr, 3.95k op/s 2023-07-06T20:23:15.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:15 smithi098 ceph-mon[128968]: pgmap v445: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 68 MiB/s wr, 3.95k op/s 2023-07-06T20:23:15.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:15 smithi069 ceph-mon[126399]: pgmap v445: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 68 MiB/s wr, 3.95k op/s 2023-07-06T20:23:17.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:17 smithi098 ceph-mon[128968]: pgmap v446: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 MiB/s wr, 3.83k op/s 2023-07-06T20:23:17.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:17 smithi069 ceph-mon[126399]: pgmap v446: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 MiB/s wr, 3.83k op/s 2023-07-06T20:23:17.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:17 smithi156 ceph-mon[129280]: pgmap v446: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 MiB/s wr, 3.83k op/s 2023-07-06T20:23:19.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 90.0 s, 2500.4 tps, lat 37.196 ms stddev 41.068 2023-07-06T20:23:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:19 smithi098 ceph-mon[128968]: pgmap v447: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 MiB/s wr, 3.88k op/s 2023-07-06T20:23:19.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:19 smithi069 ceph-mon[126399]: pgmap v447: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 MiB/s wr, 3.88k op/s 2023-07-06T20:23:19.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:19 smithi156 ceph-mon[129280]: pgmap v447: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 72 MiB/s wr, 3.88k op/s 2023-07-06T20:23:20.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:20 smithi069 ceph-mon[126399]: pgmap v448: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 65 MiB/s wr, 3.73k op/s 2023-07-06T20:23:20.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:20 smithi156 ceph-mon[129280]: pgmap v448: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 65 MiB/s wr, 3.73k op/s 2023-07-06T20:23:20.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:20 smithi098 ceph-mon[128968]: pgmap v448: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 65 MiB/s wr, 3.73k op/s 2023-07-06T20:23:22.156 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:23:22.520 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.518+0000 7fb950dff700 1 -- 172.21.15.69:0/1450392402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb94c102fb0 msgr2=0x7fb94c105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:22.521 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.518+0000 7fb950dff700 1 --2- 172.21.15.69:0/1450392402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb94c102fb0 0x7fb94c105390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9340098f0 tx=0x7fb934009c00 comp rx=0 tx=0).stop 2023-07-06T20:23:22.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.520+0000 7fb950dff700 1 -- 172.21.15.69:0/1450392402 shutdown_connections 2023-07-06T20:23:22.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.520+0000 7fb950dff700 1 --2- 172.21.15.69:0/1450392402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c1058d0 0x7fb94c107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.520+0000 7fb950dff700 1 --2- 172.21.15.69:0/1450392402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb94c100690 0x7fb94c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.520+0000 7fb950dff700 1 --2- 172.21.15.69:0/1450392402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb94c102fb0 0x7fb94c105390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.520+0000 7fb950dff700 1 -- 172.21.15.69:0/1450392402 >> 172.21.15.69:0/1450392402 conn(0x7fb94c0fa130 msgr2=0x7fb94c0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:23:22.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.520+0000 7fb950dff700 1 -- 172.21.15.69:0/1450392402 shutdown_connections 2023-07-06T20:23:22.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.520+0000 7fb950dff700 1 -- 172.21.15.69:0/1450392402 wait complete. 2023-07-06T20:23:22.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.522+0000 7fb950dff700 1 Processor -- start 2023-07-06T20:23:22.524 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.522+0000 7fb950dff700 1 -- start start 2023-07-06T20:23:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.522+0000 7fb950dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb94c100690 0x7fb94c06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.522+0000 7fb950dff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb94c102fb0 0x7fb94c06f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.525 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.523+0000 7fb941d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb94c102fb0 0x7fb94c06f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.523+0000 7fb94a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb94c100690 0x7fb94c06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.523+0000 7fb941d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb94c102fb0 0x7fb94c06f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:49402/0 (socket says 172.21.15.69:49402) 2023-07-06T20:23:22.526 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.523+0000 7fb94a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb94c100690 0x7fb94c06eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:56444/0 (socket says 172.21.15.69:56444) 2023-07-06T20:23:22.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.523+0000 7fb950dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c1058d0 0x7fb94c1a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.523+0000 7fb941d9b700 1 -- 172.21.15.69:0/2052183036 learned_addr learned my addr 172.21.15.69:0/2052183036 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:23:22.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.523+0000 7fb950dff700 1 -- 172.21.15.69:0/2052183036 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c072500 con 0x7fb94c1058d0 2023-07-06T20:23:22.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.523+0000 7fb950dff700 1 -- 172.21.15.69:0/2052183036 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c072670 con 0x7fb94c102fb0 2023-07-06T20:23:22.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.523+0000 7fb950dff700 1 -- 172.21.15.69:0/2052183036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1a60b0 con 0x7fb94c100690 2023-07-06T20:23:22.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.524+0000 7fb94a59c700 1 -- 172.21.15.69:0/2052183036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb94c102fb0 msgr2=0x7fb94c06f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:22.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.524+0000 7fb94a59c700 1 --2- 172.21.15.69:0/2052183036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb94c102fb0 0x7fb94c06f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.524+0000 7fb94a59c700 1 -- 172.21.15.69:0/2052183036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c1058d0 msgr2=0x7fb94c1a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:22.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.524+0000 7fb94a59c700 1 --2- 172.21.15.69:0/2052183036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c1058d0 0x7fb94c1a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.524+0000 7fb94a59c700 1 -- 172.21.15.69:0/2052183036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c009c40 con 0x7fb94c100690 2023-07-06T20:23:22.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.524+0000 7fb94a59c700 1 --2- 172.21.15.69:0/2052183036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb94c100690 0x7fb94c06eee0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00efc0 tx=0x7fb93c00c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:23:22.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.524+0000 7fb943fff700 1 -- 172.21.15.69:0/2052183036 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c018070 con 0x7fb94c100690 2023-07-06T20:23:22.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.525+0000 7fb943fff700 1 -- 172.21.15.69:0/2052183036 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb93c00f040 con 0x7fb94c100690 2023-07-06T20:23:22.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.525+0000 7fb943fff700 1 -- 172.21.15.69:0/2052183036 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0043e0 con 0x7fb94c100690 2023-07-06T20:23:22.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.525+0000 7fb950dff700 1 -- 172.21.15.69:0/2052183036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb934009520 con 0x7fb94c100690 2023-07-06T20:23:22.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.525+0000 7fb950dff700 1 -- 172.21.15.69:0/2052183036 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c0fe810 con 0x7fb94c100690 2023-07-06T20:23:22.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.531+0000 7fb964748b80 1 Processor -- start 2023-07-06T20:23:22.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.535+0000 7fb964748b80 1 -- start start 2023-07-06T20:23:22.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.535+0000 7fb964748b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 0x55cb25961b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.535+0000 7fb964748b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962080 0x55cb25962610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.538 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.535+0000 7fb964748b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962ba0 0x55cb25963040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.538 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.535+0000 7fb964748b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2585dcb0 con 0x55cb25962ba0 2023-07-06T20:23:22.538 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.535+0000 7fb964748b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb258b0a40 con 0x55cb25962080 2023-07-06T20:23:22.538 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.535+0000 7fb964748b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb258a9ed0 con 0x55cb25961730 2023-07-06T20:23:22.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.535+0000 7fb94ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 0x55cb25961b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.535+0000 7fb94ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 0x55cb25961b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:56456/0 (socket says 172.21.15.69:56456) 2023-07-06T20:23:22.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.535+0000 7fb94ad9d700 1 -- 172.21.15.69:0/1479935871 learned_addr learned my addr 172.21.15.69:0/1479935871 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:23:22.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.536+0000 7fb941d9b700 1 --2- 172.21.15.69:0/1479935871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962ba0 0x55cb25963040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.536+0000 7fb94ad9d700 1 -- 172.21.15.69:0/1479935871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962080 msgr2=0x55cb25962610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:23:22.539 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.536+0000 7fb94ad9d700 1 --2- 172.21.15.69:0/1479935871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962080 0x55cb25962610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.536+0000 7fb94ad9d700 1 -- 172.21.15.69:0/1479935871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962ba0 msgr2=0x55cb25963040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:22.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.536+0000 7fb94ad9d700 1 --2- 172.21.15.69:0/1479935871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962ba0 0x55cb25963040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.536+0000 7fb94ad9d700 1 -- 172.21.15.69:0/1479935871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb256d7de0 con 0x55cb25961730 2023-07-06T20:23:22.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.536+0000 7fb94ad9d700 1 --2- 172.21.15.69:0/1479935871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 0x55cb25961b40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb944009bc0 tx=0x7fb944009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c1e4f6863673544 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:23:22.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.536+0000 7fb943fff700 1 -- 172.21.15.69:0/2052183036 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb93c004540 con 0x7fb94c100690 2023-07-06T20:23:22.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.536+0000 7fb940d99700 1 -- 172.21.15.69:0/1479935871 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94400eae0 con 0x55cb25961730 2023-07-06T20:23:22.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.536+0000 7fb940d99700 1 -- 172.21.15.69:0/1479935871 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb944004cf0 con 0x55cb25961730 2023-07-06T20:23:22.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.536+0000 7fb940d99700 1 -- 172.21.15.69:0/1479935871 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944005680 con 0x55cb25961730 2023-07-06T20:23:22.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.537+0000 7fb964748b80 1 -- 172.21.15.69:0/1479935871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 msgr2=0x55cb25961b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:22.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.537+0000 7fb964748b80 1 --2- 172.21.15.69:0/1479935871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 0x55cb25961b40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb944009bc0 tx=0x7fb944009ed0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.537+0000 7fb964748b80 1 -- 172.21.15.69:0/1479935871 shutdown_connections 2023-07-06T20:23:22.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.537+0000 7fb964748b80 1 --2- 172.21.15.69:0/1479935871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962ba0 0x55cb25963040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.537+0000 7fb964748b80 1 --2- 172.21.15.69:0/1479935871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 0x55cb25961b40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.537+0000 7fb964748b80 1 --2- 172.21.15.69:0/1479935871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962080 0x55cb25962610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.537+0000 7fb964748b80 1 -- 172.21.15.69:0/1479935871 >> 172.21.15.69:0/1479935871 conn(0x55cb2594e640 msgr2=0x55cb25953940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:23:22.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.537+0000 7fb964748b80 1 -- 172.21.15.69:0/1479935871 shutdown_connections 2023-07-06T20:23:22.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.537+0000 7fb964748b80 1 -- 172.21.15.69:0/1479935871 wait complete. 2023-07-06T20:23:22.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.538+0000 7fb943fff700 1 --2- 172.21.15.69:0/2052183036 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb9380a1160 0x7fb9380a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.538+0000 7fb943fff700 1 -- 172.21.15.69:0/2052183036 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fb93c0cc5c0 con 0x7fb94c100690 2023-07-06T20:23:22.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.538+0000 7fb964748b80 1 -- start start 2023-07-06T20:23:22.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.538+0000 7fb941d9b700 1 --2- 172.21.15.69:0/2052183036 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb9380a1160 0x7fb9380a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.538+0000 7fb964748b80 1 Processor -- start 2023-07-06T20:23:22.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.539+0000 7fb964748b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25961730 0x55cb2596b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.539+0000 7fb964748b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962080 0x55cb2596b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.539+0000 7fb941d9b700 1 --2- 172.21.15.69:0/2052183036 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb9380a1160 0x7fb9380a3610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c010 tx=0x7fb934019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:23:22.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.539+0000 7fb964748b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25962ba0 0x55cb25972610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.539+0000 7fb964748b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb25849ac0 con 0x55cb25961730 2023-07-06T20:23:22.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.539+0000 7fb964748b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb258497c0 con 0x55cb25962080 2023-07-06T20:23:22.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.539+0000 7fb964748b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2572d920 con 0x55cb25962ba0 2023-07-06T20:23:22.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.539+0000 7fb964748b80 10 client.0 _ll_register_callbacks cb 0x55cb25847760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:23:22.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.539+0000 7fb941d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962080 0x55cb2596b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.539+0000 7fb941d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962080 0x55cb2596b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:49414/0 (socket says 172.21.15.69:49414) 2023-07-06T20:23:22.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.539+0000 7fb941d9b700 1 -- 172.21.15.69:0/1177538047 learned_addr learned my addr 172.21.15.69:0/1177538047 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:23:22.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.540+0000 7fb94ad9d700 1 --2- 172.21.15.69:0/1177538047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25962ba0 0x55cb25972610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.540+0000 7fb941d9b700 1 -- 172.21.15.69:0/1177538047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25962ba0 msgr2=0x55cb25972610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:22.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.540+0000 7fb941d9b700 1 --2- 172.21.15.69:0/1177538047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25962ba0 0x55cb25972610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.540+0000 7fb941d9b700 1 -- 172.21.15.69:0/1177538047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25961730 msgr2=0x55cb2596b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:22.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.540+0000 7fb941d9b700 1 --2- 172.21.15.69:0/1177538047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25961730 0x55cb2596b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.540+0000 7fb941d9b700 1 -- 172.21.15.69:0/1177538047 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9440098a0 con 0x55cb25962080 2023-07-06T20:23:22.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.541+0000 7fb941d9b700 1 --2- 172.21.15.69:0/1177538047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962080 0x55cb2596b5d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb93400b8e0 tx=0x7fb934009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccafd06ba27ca405 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:23:22.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.541+0000 7fb931ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:23:22.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.541+0000 7fb931ffb700 1 -- 172.21.15.69:0/1177538047 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934028070 con 0x55cb25962080 2023-07-06T20:23:22.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.541+0000 7fb964748b80 1 -- 172.21.15.69:0/1177538047 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb258034f0 con 0x55cb25962080 2023-07-06T20:23:22.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.541+0000 7fb931ffb700 1 -- 172.21.15.69:0/1177538047 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fb934005250 con 0x55cb25962080 2023-07-06T20:23:22.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.541+0000 7fb931ffb700 1 -- 172.21.15.69:0/1177538047 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb93401c040 con 0x55cb25962080 2023-07-06T20:23:22.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.542+0000 7fb964748b80 10 client.25544 fetch_fsmap learned FSMap version 30 2023-07-06T20:23:22.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.542+0000 7fb964748b80 1 -- 172.21.15.69:0/1177538047 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cb256c0c60 con 0x55cb25962080 2023-07-06T20:23:22.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.542+0000 7fb931ffb700 1 -- 172.21.15.69:0/1177538047 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb258034f0 con 0x55cb25962080 2023-07-06T20:23:22.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.542+0000 7fb931ffb700 1 -- 172.21.15.69:0/1177538047 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb934023480 con 0x55cb25962080 2023-07-06T20:23:22.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.542+0000 7fb964748b80 10 client.25544 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:23:22.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.542+0000 7fb964748b80 10 client.25544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:23:22.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.542+0000 7fb964748b80 20 client.25544 populate_metadata read hostname 'smithi069' 2023-07-06T20:23:22.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.542+0000 7fb964748b80 1 --2- 172.21.15.69:0/1177538047 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55cb25976e90 0x55cb25979340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.542+0000 7fb964748b80 4 client.25544 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:23:22.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.542+0000 7fb964748b80 1 -- 172.21.15.69:0/1177538047 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb25927d60 con 0x55cb25976e90 2023-07-06T20:23:22.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.545+0000 7fb94a59c700 1 --2- 172.21.15.69:0/1177538047 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55cb25976e90 0x55cb25979340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.546+0000 7fb931ffb700 10 client.25544 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:23:22.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.546+0000 7fb94a59c700 1 --2- 172.21.15.69:0/1177538047 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55cb25976e90 0x55cb25979340 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aefec376f3afb6a server_cookie=d82e107b8fc3646f in_seq=0 out_seq=0 2023-07-06T20:23:22.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.549+0000 7fb931ffb700 1 -- 172.21.15.69:0/1177538047 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cb25927d60 con 0x55cb25976e90 2023-07-06T20:23:22.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.549+0000 7fb931ffb700 10 client.25544 handle_command_reply: tid=0 2023-07-06T20:23:22.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.549+0000 7fb964748b80 1 client.25544 shutdown 2023-07-06T20:23:22.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.549+0000 7fb964748b80 1 -- 172.21.15.69:0/1177538047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962080 msgr2=0x55cb2596b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:22.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.549+0000 7fb964748b80 1 --2- 172.21.15.69:0/1177538047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962080 0x55cb2596b5d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb93400b8e0 tx=0x7fb934009c00 comp rx=0 tx=0).stop 2023-07-06T20:23:22.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.550+0000 7fb964748b80 1 -- 172.21.15.69:0/1177538047 shutdown_connections 2023-07-06T20:23:22.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.550+0000 7fb964748b80 1 --2- 172.21.15.69:0/1177538047 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55cb25976e90 0x55cb25979340 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.550+0000 7fb964748b80 1 --2- 172.21.15.69:0/1177538047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25962ba0 0x55cb25972610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.550+0000 7fb964748b80 1 --2- 172.21.15.69:0/1177538047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25961730 0x55cb2596b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.550+0000 7fb964748b80 1 --2- 172.21.15.69:0/1177538047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962080 0x55cb2596b5d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.550+0000 7fb964748b80 1 -- 172.21.15.69:0/1177538047 >> 172.21.15.69:0/1177538047 conn(0x55cb2594e640 msgr2=0x55cb259530b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:23:22.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.550+0000 7fb931ffb700 0 client.25544 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:23:22.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.550+0000 7fb964748b80 1 -- 172.21.15.69:0/1177538047 shutdown_connections 2023-07-06T20:23:22.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.550+0000 7fb964748b80 1 -- 172.21.15.69:0/1177538047 wait complete. 2023-07-06T20:23:22.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.550+0000 7fb964748b80 20 client.25544 trim_cache size 0 max 16384 2023-07-06T20:23:22.584 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.582+0000 7fb964748b80 1 Processor -- start 2023-07-06T20:23:22.586 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.584+0000 7fb964748b80 1 -- start start 2023-07-06T20:23:22.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.584+0000 7fb964748b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 0x55cb2597dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.584+0000 7fb964748b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962080 0x55cb259725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.584+0000 7fb964748b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962ba0 0x55cb259530b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.584+0000 7fb964748b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb259554c0 con 0x55cb25962080 2023-07-06T20:23:22.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.584+0000 7fb964748b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb25968d30 con 0x55cb25962ba0 2023-07-06T20:23:22.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.584+0000 7fb964748b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2597e220 con 0x55cb25961730 2023-07-06T20:23:22.587 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.585+0000 7fb94a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 0x55cb2597dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.585+0000 7fb94ad9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962ba0 0x55cb259530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.585+0000 7fb94a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 0x55cb2597dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:56466/0 (socket says 172.21.15.69:56466) 2023-07-06T20:23:22.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.585+0000 7fb94a59c700 1 -- 172.21.15.69:0/2257665997 learned_addr learned my addr 172.21.15.69:0/2257665997 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:23:22.588 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.585+0000 7fb94ad9d700 1 --2- 172.21.15.69:0/2257665997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962ba0 0x55cb259530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:49424/0 (socket says 172.21.15.69:49424) 2023-07-06T20:23:22.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.585+0000 7fb941d9b700 1 --2- 172.21.15.69:0/2257665997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962080 0x55cb259725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.585+0000 7fb94a59c700 1 -- 172.21.15.69:0/2257665997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962ba0 msgr2=0x55cb259530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:22.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.585+0000 7fb94a59c700 1 --2- 172.21.15.69:0/2257665997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962ba0 0x55cb259530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.589 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.585+0000 7fb94a59c700 1 -- 172.21.15.69:0/2257665997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962080 msgr2=0x55cb259725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:22.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.585+0000 7fb94a59c700 1 --2- 172.21.15.69:0/2257665997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962080 0x55cb259725d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.585+0000 7fb94a59c700 1 -- 172.21.15.69:0/2257665997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb256fe250 con 0x55cb25961730 2023-07-06T20:23:22.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.585+0000 7fb94a59c700 1 --2- 172.21.15.69:0/2257665997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 0x55cb2597dce0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00c850 tx=0x7fb93c0b9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c64d4d56e0eefa66 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:23:22.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.586+0000 7fb94159a700 1 -- 172.21.15.69:0/2257665997 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0b1070 con 0x55cb25961730 2023-07-06T20:23:22.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.586+0000 7fb94159a700 1 -- 172.21.15.69:0/2257665997 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb93c099510 con 0x55cb25961730 2023-07-06T20:23:22.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.586+0000 7fb94159a700 1 -- 172.21.15.69:0/2257665997 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0a1d90 con 0x55cb25961730 2023-07-06T20:23:22.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.587+0000 7fb964748b80 1 -- 172.21.15.69:0/2257665997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 msgr2=0x55cb2597dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:22.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.587+0000 7fb964748b80 1 --2- 172.21.15.69:0/2257665997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 0x55cb2597dce0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00c850 tx=0x7fb93c0b9040 comp rx=0 tx=0).stop 2023-07-06T20:23:22.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.588+0000 7fb964748b80 1 -- 172.21.15.69:0/2257665997 shutdown_connections 2023-07-06T20:23:22.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.588+0000 7fb964748b80 1 --2- 172.21.15.69:0/2257665997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25962ba0 0x55cb259530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.588+0000 7fb964748b80 1 --2- 172.21.15.69:0/2257665997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25961730 0x55cb2597dce0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.588+0000 7fb964748b80 1 --2- 172.21.15.69:0/2257665997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962080 0x55cb259725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.588+0000 7fb964748b80 1 -- 172.21.15.69:0/2257665997 >> 172.21.15.69:0/2257665997 conn(0x55cb2594e640 msgr2=0x55cb25953910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:23:22.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.603+0000 7fb964748b80 1 -- 172.21.15.69:0/2257665997 shutdown_connections 2023-07-06T20:23:22.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.606+0000 7fb964748b80 1 -- 172.21.15.69:0/2257665997 wait complete. 2023-07-06T20:23:22.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.606+0000 7fb964748b80 1 -- start start 2023-07-06T20:23:22.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.607+0000 7fb964748b80 1 Processor -- start 2023-07-06T20:23:22.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.607+0000 7fb964748b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25961730 0x55cb25971e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.607+0000 7fb964748b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962080 0x55cb25972380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.607+0000 7fb964748b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25962ba0 0x55cb2596e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.607+0000 7fb964748b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2596ee60 con 0x55cb25962080 2023-07-06T20:23:22.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.607+0000 7fb964748b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2596efd0 con 0x55cb25961730 2023-07-06T20:23:22.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.607+0000 7fb964748b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2596f190 con 0x55cb25962ba0 2023-07-06T20:23:22.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.608+0000 7fb964748b80 10 client.0 _ll_register_callbacks cb 0x7fb93400f890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:23:22.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.608+0000 7fb94ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25962ba0 0x55cb2596e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.608+0000 7fb94ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25962ba0 0x55cb2596e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:56472/0 (socket says 172.21.15.69:56472) 2023-07-06T20:23:22.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.608+0000 7fb94ad9d700 1 -- 172.21.15.69:0/1729024889 learned_addr learned my addr 172.21.15.69:0/1729024889 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:23:22.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.609+0000 7fb94ad9d700 1 -- 172.21.15.69:0/1729024889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25961730 msgr2=0x55cb25971e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:22.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.609+0000 7fb94ad9d700 1 --2- 172.21.15.69:0/1729024889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25961730 0x55cb25971e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.609+0000 7fb94ad9d700 1 -- 172.21.15.69:0/1729024889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962080 msgr2=0x55cb25972380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:22.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.609+0000 7fb94ad9d700 1 --2- 172.21.15.69:0/1729024889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962080 0x55cb25972380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.609+0000 7fb94ad9d700 1 -- 172.21.15.69:0/1729024889 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c0fe810 con 0x55cb25962ba0 2023-07-06T20:23:22.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.609+0000 7fb94ad9d700 1 --2- 172.21.15.69:0/1729024889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25962ba0 0x55cb2596e8e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb9440095a0 tx=0x7fb94400ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35649abaea43251f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:23:22.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.609+0000 7fb931ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:23:22.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.609+0000 7fb931ffb700 1 -- 172.21.15.69:0/1729024889 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944004460 con 0x55cb25962ba0 2023-07-06T20:23:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.609+0000 7fb931ffb700 1 -- 172.21.15.69:0/1729024889 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fb944014070 con 0x55cb25962ba0 2023-07-06T20:23:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.610+0000 7fb964748b80 1 -- 172.21.15.69:0/1729024889 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb944004cf0 con 0x55cb25962ba0 2023-07-06T20:23:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.610+0000 7fb931ffb700 1 -- 172.21.15.69:0/1729024889 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb94400eed0 con 0x55cb25962ba0 2023-07-06T20:23:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.610+0000 7fb964748b80 10 client.25492 fetch_fsmap learned FSMap version 30 2023-07-06T20:23:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.610+0000 7fb964748b80 1 -- 172.21.15.69:0/1729024889 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cb256d62b0 con 0x55cb25962ba0 2023-07-06T20:23:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.611+0000 7fb931ffb700 1 -- 172.21.15.69:0/1729024889 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb944004cf0 con 0x55cb25962ba0 2023-07-06T20:23:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.611+0000 7fb931ffb700 1 -- 172.21.15.69:0/1729024889 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb944005200 con 0x55cb25962ba0 2023-07-06T20:23:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.611+0000 7fb964748b80 10 client.25492 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:23:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.611+0000 7fb964748b80 10 client.25492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:23:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.611+0000 7fb964748b80 20 client.25492 populate_metadata read hostname 'smithi069' 2023-07-06T20:23:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.611+0000 7fb964748b80 1 --2- 172.21.15.69:0/1729024889 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55cb259cc330 0x55cb259ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.611+0000 7fb964748b80 4 client.25492 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:23:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.611+0000 7fb964748b80 1 -- 172.21.15.69:0/1729024889 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cb258a2eb0 con 0x55cb259cc330 2023-07-06T20:23:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.612+0000 7fb94a59c700 1 --2- 172.21.15.69:0/1729024889 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55cb259cc330 0x55cb259ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:22.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.613+0000 7fb94a59c700 1 --2- 172.21.15.69:0/1729024889 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55cb259cc330 0x55cb259ce740 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9bf14da78eba8f9 server_cookie=9156951151516c2f in_seq=0 out_seq=0 2023-07-06T20:23:22.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.613+0000 7fb931ffb700 10 client.25492 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:23:22.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.613+0000 7fb931ffb700 1 -- 172.21.15.69:0/1729024889 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55cb258a2eb0 con 0x55cb259cc330 2023-07-06T20:23:22.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.613+0000 7fb931ffb700 10 client.25492 handle_command_reply: tid=0 2023-07-06T20:23:22.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.613+0000 7fb964748b80 1 client.25492 shutdown 2023-07-06T20:23:22.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.614+0000 7fb964748b80 1 -- 172.21.15.69:0/1729024889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25962ba0 msgr2=0x55cb2596e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:22.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.614+0000 7fb964748b80 1 --2- 172.21.15.69:0/1729024889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25962ba0 0x55cb2596e8e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb9440095a0 tx=0x7fb94400ec30 comp rx=0 tx=0).stop 2023-07-06T20:23:22.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.614+0000 7fb964748b80 1 -- 172.21.15.69:0/1729024889 shutdown_connections 2023-07-06T20:23:22.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.614+0000 7fb964748b80 1 --2- 172.21.15.69:0/1729024889 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55cb259cc330 0x55cb259ce740 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.614+0000 7fb964748b80 1 --2- 172.21.15.69:0/1729024889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb25962ba0 0x55cb2596e8e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.614+0000 7fb964748b80 1 --2- 172.21.15.69:0/1729024889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb25961730 0x55cb25971e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.614+0000 7fb964748b80 1 --2- 172.21.15.69:0/1729024889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cb25962080 0x55cb25972380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.614+0000 7fb964748b80 1 -- 172.21.15.69:0/1729024889 >> 172.21.15.69:0/1729024889 conn(0x55cb2594e640 msgr2=0x55cb25972f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:23:22.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.614+0000 7fb931ffb700 0 client.25492 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:23:22.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.614+0000 7fb964748b80 1 -- 172.21.15.69:0/1729024889 shutdown_connections 2023-07-06T20:23:22.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.614+0000 7fb964748b80 1 -- 172.21.15.69:0/1729024889 wait complete. 2023-07-06T20:23:22.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.614+0000 7fb964748b80 20 client.25492 trim_cache size 0 max 16384 2023-07-06T20:23:22.620 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:23:22.620 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:23:22.620 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:23:22.620 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:23:22.621 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:23:22.621 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:23:22.621 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:23:22.621 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:23:22.621 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:23:22.621 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:23:22.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.615+0000 7fb950dff700 1 -- 172.21.15.69:0/2052183036 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb9380a1160 msgr2=0x7fb9380a3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:22.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.615+0000 7fb950dff700 1 --2- 172.21.15.69:0/2052183036 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb9380a1160 0x7fb9380a3610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c010 tx=0x7fb934019040 comp rx=0 tx=0).stop 2023-07-06T20:23:22.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.615+0000 7fb950dff700 1 -- 172.21.15.69:0/2052183036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb94c100690 msgr2=0x7fb94c06eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:22.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.615+0000 7fb950dff700 1 --2- 172.21.15.69:0/2052183036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb94c100690 0x7fb94c06eee0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00efc0 tx=0x7fb93c00c450 comp rx=0 tx=0).stop 2023-07-06T20:23:22.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.615+0000 7fb950dff700 1 -- 172.21.15.69:0/2052183036 shutdown_connections 2023-07-06T20:23:22.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.616+0000 7fb950dff700 1 --2- 172.21.15.69:0/2052183036 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb9380a1160 0x7fb9380a3610 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.616+0000 7fb950dff700 1 --2- 172.21.15.69:0/2052183036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb94c1058d0 0x7fb94c1a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.616+0000 7fb950dff700 1 --2- 172.21.15.69:0/2052183036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb94c100690 0x7fb94c06eee0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.616+0000 7fb950dff700 1 --2- 172.21.15.69:0/2052183036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb94c102fb0 0x7fb94c06f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:22.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.616+0000 7fb950dff700 1 -- 172.21.15.69:0/2052183036 >> 172.21.15.69:0/2052183036 conn(0x7fb94c0fa130 msgr2=0x7fb94c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:23:22.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.616+0000 7fb950dff700 1 -- 172.21.15.69:0/2052183036 shutdown_connections 2023-07-06T20:23:22.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:22.616+0000 7fb950dff700 1 -- 172.21.15.69:0/2052183036 wait complete. 2023-07-06T20:23:22.648 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:23:23.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:22 smithi069 ceph-mon[126399]: pgmap v449: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 71 MiB/s wr, 4.22k op/s 2023-07-06T20:23:23.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:22 smithi156 ceph-mon[129280]: pgmap v449: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 71 MiB/s wr, 4.22k op/s 2023-07-06T20:23:23.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:22 smithi098 ceph-mon[128968]: pgmap v449: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 71 MiB/s wr, 4.22k op/s 2023-07-06T20:23:24.233 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 95.0 s, 2280.5 tps, lat 46.766 ms stddev 76.703 2023-07-06T20:23:25.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:24 smithi156 ceph-mon[129280]: pgmap v450: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 72 MiB/s wr, 4.37k op/s 2023-07-06T20:23:25.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:24 smithi098 ceph-mon[128968]: pgmap v450: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 72 MiB/s wr, 4.37k op/s 2023-07-06T20:23:25.263 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:24 smithi069 ceph-mon[126399]: pgmap v450: 97 pgs: 97 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 72 MiB/s wr, 4.37k op/s 2023-07-06T20:23:27.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:26 smithi156 ceph-mon[129280]: pgmap v451: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 71 MiB/s wr, 4.21k op/s 2023-07-06T20:23:27.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:26 smithi098 ceph-mon[128968]: pgmap v451: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 71 MiB/s wr, 4.21k op/s 2023-07-06T20:23:27.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:26 smithi069 ceph-mon[126399]: pgmap v451: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 71 MiB/s wr, 4.21k op/s 2023-07-06T20:23:29.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:28 smithi156 ceph-mon[129280]: pgmap v452: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 73 MiB/s wr, 4.26k op/s 2023-07-06T20:23:29.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:28 smithi098 ceph-mon[128968]: pgmap v452: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 73 MiB/s wr, 4.26k op/s 2023-07-06T20:23:29.233 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 100.0 s, 2934.7 tps, lat 34.198 ms stddev 31.024 2023-07-06T20:23:29.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:28 smithi069 ceph-mon[126399]: pgmap v452: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 73 MiB/s wr, 4.26k op/s 2023-07-06T20:23:31.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:30 smithi156 ceph-mon[129280]: pgmap v453: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 70 MiB/s wr, 4.18k op/s 2023-07-06T20:23:31.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:30 smithi098 ceph-mon[128968]: pgmap v453: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 70 MiB/s wr, 4.18k op/s 2023-07-06T20:23:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:30 smithi069 ceph-mon[126399]: pgmap v453: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 70 MiB/s wr, 4.18k op/s 2023-07-06T20:23:33.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:32 smithi069 ceph-mon[126399]: pgmap v454: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 MiB/s wr, 4.85k op/s 2023-07-06T20:23:33.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:32 smithi156 ceph-mon[129280]: pgmap v454: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 MiB/s wr, 4.85k op/s 2023-07-06T20:23:33.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:32 smithi098 ceph-mon[128968]: pgmap v454: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 MiB/s wr, 4.85k op/s 2023-07-06T20:23:34.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 105.0 s, 1748.3 tps, lat 57.253 ms stddev 98.644 2023-07-06T20:23:35.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:34 smithi069 ceph-mon[126399]: pgmap v455: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 73 MiB/s wr, 4.83k op/s 2023-07-06T20:23:35.360 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:34 smithi156 ceph-mon[129280]: pgmap v455: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 73 MiB/s wr, 4.83k op/s 2023-07-06T20:23:35.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:34 smithi098 ceph-mon[128968]: pgmap v455: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 73 MiB/s wr, 4.83k op/s 2023-07-06T20:23:37.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:36 smithi069 ceph-mon[126399]: pgmap v456: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 66 MiB/s wr, 4.18k op/s 2023-07-06T20:23:37.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:36 smithi156 ceph-mon[129280]: pgmap v456: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 66 MiB/s wr, 4.18k op/s 2023-07-06T20:23:37.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:36 smithi098 ceph-mon[128968]: pgmap v456: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 66 MiB/s wr, 4.18k op/s 2023-07-06T20:23:39.232 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 110.0 s, 3075.7 tps, lat 32.439 ms stddev 29.553 2023-07-06T20:23:39.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:38 smithi069 ceph-mon[126399]: pgmap v457: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 69 MiB/s wr, 4.06k op/s 2023-07-06T20:23:39.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:38 smithi156 ceph-mon[129280]: pgmap v457: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 69 MiB/s wr, 4.06k op/s 2023-07-06T20:23:39.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:38 smithi098 ceph-mon[128968]: pgmap v457: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 69 MiB/s wr, 4.06k op/s 2023-07-06T20:23:41.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:40 smithi069 ceph-mon[126399]: pgmap v458: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 63 MiB/s wr, 3.95k op/s 2023-07-06T20:23:41.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:40 smithi156 ceph-mon[129280]: pgmap v458: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 63 MiB/s wr, 3.95k op/s 2023-07-06T20:23:41.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:40 smithi098 ceph-mon[128968]: pgmap v458: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 63 MiB/s wr, 3.95k op/s 2023-07-06T20:23:43.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:42 smithi069 ceph-mon[126399]: pgmap v459: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 70 MiB/s wr, 4.04k op/s 2023-07-06T20:23:43.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:42 smithi156 ceph-mon[129280]: pgmap v459: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 70 MiB/s wr, 4.04k op/s 2023-07-06T20:23:43.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:42 smithi098 ceph-mon[128968]: pgmap v459: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 70 MiB/s wr, 4.04k op/s 2023-07-06T20:23:44.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 115.0 s, 2455.2 tps, lat 39.671 ms stddev 58.894 2023-07-06T20:23:45.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:45 smithi156 ceph-mon[129280]: pgmap v460: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 63 MiB/s wr, 3.37k op/s 2023-07-06T20:23:45.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:45 smithi098 ceph-mon[128968]: pgmap v460: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 63 MiB/s wr, 3.37k op/s 2023-07-06T20:23:45.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:45 smithi069 ceph-mon[126399]: pgmap v460: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 63 MiB/s wr, 3.37k op/s 2023-07-06T20:23:47.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:47 smithi156 ceph-mon[129280]: pgmap v461: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 MiB/s wr, 3.13k op/s 2023-07-06T20:23:47.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:47 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:23:47.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:47 smithi098 ceph-mon[128968]: pgmap v461: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 MiB/s wr, 3.13k op/s 2023-07-06T20:23:47.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:47 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:23:47.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:47 smithi069 ceph-mon[126399]: pgmap v461: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 MiB/s wr, 3.13k op/s 2023-07-06T20:23:47.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:47 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:23:48.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:23:48.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:23:48.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:23:48.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:23:48.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:23:48.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:23:48.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:23:48.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:23:48.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:23:49.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 120.0 s, 2381.0 tps, lat 43.118 ms stddev 53.532 2023-07-06T20:23:49.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:49 smithi156 ceph-mon[129280]: pgmap v462: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 69 MiB/s wr, 3.96k op/s 2023-07-06T20:23:49.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:49 smithi098 ceph-mon[128968]: pgmap v462: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 69 MiB/s wr, 3.96k op/s 2023-07-06T20:23:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:49 smithi069 ceph-mon[126399]: pgmap v462: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 69 MiB/s wr, 3.96k op/s 2023-07-06T20:23:51.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:51 smithi156 ceph-mon[129280]: pgmap v463: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 MiB/s wr, 4.07k op/s 2023-07-06T20:23:51.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:51 smithi098 ceph-mon[128968]: pgmap v463: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 MiB/s wr, 4.07k op/s 2023-07-06T20:23:51.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:51 smithi069 ceph-mon[126399]: pgmap v463: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 MiB/s wr, 4.07k op/s 2023-07-06T20:23:52.649 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:23:53.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.045+0000 7f4476bff700 1 -- 172.21.15.69:0/524001766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44701037a0 msgr2=0x7f4470103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:53.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.045+0000 7f4476bff700 1 --2- 172.21.15.69:0/524001766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44701037a0 0x7f4470103c10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4468009f90 tx=0x7f446800b330 comp rx=0 tx=0).stop 2023-07-06T20:23:53.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.046+0000 7f4476bff700 1 -- 172.21.15.69:0/524001766 shutdown_connections 2023-07-06T20:23:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.046+0000 7f4476bff700 1 --2- 172.21.15.69:0/524001766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44701037a0 0x7f4470103c10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.046+0000 7f4476bff700 1 --2- 172.21.15.69:0/524001766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44700fe480 0x7f44701008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.046+0000 7f4476bff700 1 --2- 172.21.15.69:0/524001766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4470100de0 0x7f4470103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.046+0000 7f4476bff700 1 -- 172.21.15.69:0/524001766 >> 172.21.15.69:0/524001766 conn(0x7f44700fa130 msgr2=0x7f44700fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:23:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.048+0000 7f4476bff700 1 -- 172.21.15.69:0/524001766 shutdown_connections 2023-07-06T20:23:53.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.048+0000 7f4476bff700 1 -- 172.21.15.69:0/524001766 wait complete. 2023-07-06T20:23:53.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.049+0000 7f4476bff700 1 Processor -- start 2023-07-06T20:23:53.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.050+0000 7f4476bff700 1 -- start start 2023-07-06T20:23:53.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.050+0000 7f4476bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44700fe480 0x7f4470109850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.050+0000 7f4476bff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4470100de0 0x7f4470109d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.050+0000 7f4476bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44701037a0 0x7f44701ac210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.050+0000 7f4476bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447010ce70 con 0x7f44701037a0 2023-07-06T20:23:53.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.050+0000 7f4476bff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447010cfe0 con 0x7f4470100de0 2023-07-06T20:23:53.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.050+0000 7f4476bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44701ac750 con 0x7f44700fe480 2023-07-06T20:23:53.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f447499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44700fe480 0x7f4470109850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f447499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44700fe480 0x7f4470109850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:49106/0 (socket says 172.21.15.69:49106) 2023-07-06T20:23:53.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f447499b700 1 -- 172.21.15.69:0/1445563989 learned_addr learned my addr 172.21.15.69:0/1445563989 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:23:53.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f447519c700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44701037a0 0x7f44701ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f446ffff700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4470100de0 0x7f4470109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f447499b700 1 -- 172.21.15.69:0/1445563989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4470100de0 msgr2=0x7f4470109d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:53.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f447499b700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4470100de0 0x7f4470109d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f447499b700 1 -- 172.21.15.69:0/1445563989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44701037a0 msgr2=0x7f44701ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:53.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f447499b700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44701037a0 0x7f44701ac210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f447499b700 1 -- 172.21.15.69:0/1445563989 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4468009c40 con 0x7f44700fe480 2023-07-06T20:23:53.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f446ffff700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4470100de0 0x7f4470109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:23:53.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f447519c700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44701037a0 0x7f44701ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:23:53.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.051+0000 7f447499b700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44700fe480 0x7f4470109850 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f446400b530 tx=0x7f446400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:23:53.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.052+0000 7f446dffb700 1 -- 172.21.15.69:0/1445563989 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464011830 con 0x7f44700fe480 2023-07-06T20:23:53.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.052+0000 7f446dffb700 1 -- 172.21.15.69:0/1445563989 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4464011e50 con 0x7f44700fe480 2023-07-06T20:23:53.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.052+0000 7f446dffb700 1 -- 172.21.15.69:0/1445563989 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446400f2e0 con 0x7f44700fe480 2023-07-06T20:23:53.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.052+0000 7f4476bff700 1 -- 172.21.15.69:0/1445563989 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44701ac890 con 0x7f44700fe480 2023-07-06T20:23:53.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.052+0000 7f4476bff700 1 -- 172.21.15.69:0/1445563989 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44701ace20 con 0x7f44700fe480 2023-07-06T20:23:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.065+0000 7f448a4eeb80 1 Processor -- start 2023-07-06T20:23:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.068+0000 7f448a4eeb80 1 -- start start 2023-07-06T20:23:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.069+0000 7f448a4eeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc4307330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.069+0000 7f448a4eeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4307870 0x55acc4307e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.069+0000 7f448a4eeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4308390 0x55acc4308830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.069+0000 7f448a4eeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc420bcb0 con 0x55acc4308390 2023-07-06T20:23:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.069+0000 7f448a4eeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc425ea40 con 0x55acc4306f20 2023-07-06T20:23:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.069+0000 7f448a4eeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc4257ed0 con 0x55acc4307870 2023-07-06T20:23:53.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.069+0000 7f446dffb700 1 -- 172.21.15.69:0/1445563989 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4464010580 con 0x7f44700fe480 2023-07-06T20:23:53.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.069+0000 7f447519c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc4307330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.069+0000 7f447519c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc4307330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:60278/0 (socket says 172.21.15.69:60278) 2023-07-06T20:23:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.069+0000 7f447519c700 1 -- 172.21.15.69:0/2008939950 learned_addr learned my addr 172.21.15.69:0/2008939950 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:23:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.069+0000 7f447499b700 1 --2- 172.21.15.69:0/2008939950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4307870 0x55acc4307e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.069+0000 7f446ffff700 1 --2- 172.21.15.69:0/2008939950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4308390 0x55acc4308830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.070+0000 7f447499b700 1 -- 172.21.15.69:0/2008939950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 msgr2=0x55acc4307330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.070+0000 7f447499b700 1 --2- 172.21.15.69:0/2008939950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc4307330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.070+0000 7f447499b700 1 -- 172.21.15.69:0/2008939950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4308390 msgr2=0x55acc4308830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.070+0000 7f447499b700 1 --2- 172.21.15.69:0/2008939950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4308390 0x55acc4308830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.070+0000 7f447499b700 1 -- 172.21.15.69:0/2008939950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acc4085de0 con 0x55acc4307870 2023-07-06T20:23:53.074 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.071+0000 7f446dffb700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f44600a1220 0x7f44600a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.071+0000 7f446dffb700 1 -- 172.21.15.69:0/1445563989 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f44640cc120 con 0x7f44700fe480 2023-07-06T20:23:53.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.072+0000 7f447499b700 1 --2- 172.21.15.69:0/2008939950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4307870 0x55acc4307e00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f44640c9bc0 tx=0x7f44640d4160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f8207a5c581e29c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:23:53.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.072+0000 7f445affd700 1 -- 172.21.15.69:0/2008939950 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44640cf6d0 con 0x55acc4307870 2023-07-06T20:23:53.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.072+0000 7f445affd700 1 -- 172.21.15.69:0/2008939950 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4464010ee0 con 0x55acc4307870 2023-07-06T20:23:53.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.072+0000 7f445affd700 1 -- 172.21.15.69:0/2008939950 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464099040 con 0x55acc4307870 2023-07-06T20:23:53.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.073+0000 7f448a4eeb80 1 -- 172.21.15.69:0/2008939950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4307870 msgr2=0x55acc4307e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:53.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.073+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/2008939950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4307870 0x55acc4307e00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f44640c9bc0 tx=0x7f44640d4160 comp rx=0 tx=0).stop 2023-07-06T20:23:53.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.073+0000 7f448a4eeb80 1 -- 172.21.15.69:0/2008939950 shutdown_connections 2023-07-06T20:23:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.073+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/2008939950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4308390 0x55acc4308830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.073+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/2008939950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc4307330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.073+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/2008939950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4307870 0x55acc4307e00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.073+0000 7f448a4eeb80 1 -- 172.21.15.69:0/2008939950 >> 172.21.15.69:0/2008939950 conn(0x55acc42fc640 msgr2=0x55acc43019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:23:53.077 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.074+0000 7f447519c700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f44600a1220 0x7f44600a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.074+0000 7f448a4eeb80 1 -- 172.21.15.69:0/2008939950 shutdown_connections 2023-07-06T20:23:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.074+0000 7f448a4eeb80 1 -- 172.21.15.69:0/2008939950 wait complete. 2023-07-06T20:23:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.074+0000 7f447519c700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f44600a1220 0x7f44600a36d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f446800b6c0 tx=0x7f4468009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:23:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.075+0000 7f448a4eeb80 1 -- start start 2023-07-06T20:23:53.078 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.075+0000 7f448a4eeb80 1 Processor -- start 2023-07-06T20:23:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.076+0000 7f448a4eeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc43108f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.076+0000 7f448a4eeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4307870 0x55acc4310e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.076+0000 7f448a4eeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4308390 0x55acc4317e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.076+0000 7f448a4eeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc41f7ac0 con 0x55acc4308390 2023-07-06T20:23:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.076+0000 7f448a4eeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc41f77c0 con 0x55acc4306f20 2023-07-06T20:23:53.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.076+0000 7f448a4eeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc40db920 con 0x55acc4307870 2023-07-06T20:23:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.076+0000 7f448a4eeb80 10 client.0 _ll_register_callbacks cb 0x55acc41f5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:23:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.076+0000 7f447499b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc43108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.076+0000 7f447499b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc43108f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:60290/0 (socket says 172.21.15.69:60290) 2023-07-06T20:23:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.076+0000 7f447499b700 1 -- 172.21.15.69:0/4034188487 learned_addr learned my addr 172.21.15.69:0/4034188487 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:23:53.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.076+0000 7f446ffff700 1 --2- 172.21.15.69:0/4034188487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4307870 0x55acc4310e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.081 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.077+0000 7f447499b700 1 -- 172.21.15.69:0/4034188487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4307870 msgr2=0x55acc4310e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.077+0000 7f447499b700 1 --2- 172.21.15.69:0/4034188487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4307870 0x55acc4310e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.077+0000 7f447499b700 1 -- 172.21.15.69:0/4034188487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4308390 msgr2=0x55acc4317e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.077+0000 7f447499b700 1 --2- 172.21.15.69:0/4034188487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4308390 0x55acc4317e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.077+0000 7f447499b700 1 -- 172.21.15.69:0/4034188487 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44701ace20 con 0x55acc4306f20 2023-07-06T20:23:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.077+0000 7f447499b700 1 --2- 172.21.15.69:0/4034188487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc43108f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f44640c9b90 tx=0x7f446400bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c78e300771b2e17 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:23:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.077+0000 7f444ffff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:23:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.078+0000 7f444ffff700 1 -- 172.21.15.69:0/4034188487 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44640cf6d0 con 0x55acc4306f20 2023-07-06T20:23:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.078+0000 7f444ffff700 1 -- 172.21.15.69:0/4034188487 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f4464010380 con 0x55acc4306f20 2023-07-06T20:23:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.078+0000 7f448a4eeb80 1 -- 172.21.15.69:0/4034188487 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acc41b14f0 con 0x55acc4306f20 2023-07-06T20:23:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.078+0000 7f444ffff700 1 -- 172.21.15.69:0/4034188487 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4464098e60 con 0x55acc4306f20 2023-07-06T20:23:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.078+0000 7f448a4eeb80 10 client.25559 fetch_fsmap learned FSMap version 30 2023-07-06T20:23:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.078+0000 7f448a4eeb80 1 -- 172.21.15.69:0/4034188487 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55acc406ec60 con 0x55acc4306f20 2023-07-06T20:23:53.083 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.079+0000 7f444ffff700 1 -- 172.21.15.69:0/4034188487 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55acc41b14f0 con 0x55acc4306f20 2023-07-06T20:23:53.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.079+0000 7f444ffff700 1 -- 172.21.15.69:0/4034188487 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4464095c10 con 0x55acc4306f20 2023-07-06T20:23:53.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.079+0000 7f448a4eeb80 10 client.25559 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:23:53.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.079+0000 7f448a4eeb80 10 client.25559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:23:53.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.079+0000 7f448a4eeb80 20 client.25559 populate_metadata read hostname 'smithi069' 2023-07-06T20:23:53.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.079+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/4034188487 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55acc432d0b0 0x55acc432f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.079+0000 7f448a4eeb80 4 client.25559 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:23:53.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.079+0000 7f448a4eeb80 1 -- 172.21.15.69:0/4034188487 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acc42d5d60 con 0x55acc432d0b0 2023-07-06T20:23:53.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.079+0000 7f446ffff700 1 --2- 172.21.15.69:0/4034188487 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55acc432d0b0 0x55acc432f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.080+0000 7f446ffff700 1 --2- 172.21.15.69:0/4034188487 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55acc432d0b0 0x55acc432f490 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=251529570c46217c server_cookie=fe62f8a1adda9be7 in_seq=0 out_seq=0 2023-07-06T20:23:53.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.080+0000 7f444ffff700 10 client.25559 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:23:53.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.081+0000 7f444ffff700 1 -- 172.21.15.69:0/4034188487 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55acc42d5d60 con 0x55acc432d0b0 2023-07-06T20:23:53.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.081+0000 7f444ffff700 10 client.25559 handle_command_reply: tid=0 2023-07-06T20:23:53.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.081+0000 7f448a4eeb80 1 client.25559 shutdown 2023-07-06T20:23:53.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.081+0000 7f448a4eeb80 1 -- 172.21.15.69:0/4034188487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 msgr2=0x55acc43108f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.081+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/4034188487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc43108f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f44640c9b90 tx=0x7f446400bf20 comp rx=0 tx=0).stop 2023-07-06T20:23:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.081+0000 7f448a4eeb80 1 -- 172.21.15.69:0/4034188487 shutdown_connections 2023-07-06T20:23:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.081+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/4034188487 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55acc432d0b0 0x55acc432f490 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.081+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/4034188487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4308390 0x55acc4317e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.081+0000 7f444ffff700 0 client.25559 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:23:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.081+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/4034188487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc43108f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.081+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/4034188487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4307870 0x55acc4310e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.081+0000 7f448a4eeb80 1 -- 172.21.15.69:0/4034188487 >> 172.21.15.69:0/4034188487 conn(0x55acc42fc640 msgr2=0x55acc43010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:23:53.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.082+0000 7f448a4eeb80 1 -- 172.21.15.69:0/4034188487 shutdown_connections 2023-07-06T20:23:53.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.082+0000 7f448a4eeb80 1 -- 172.21.15.69:0/4034188487 wait complete. 2023-07-06T20:23:53.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.082+0000 7f448a4eeb80 20 client.25559 trim_cache size 0 max 16384 2023-07-06T20:23:53.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.109+0000 7f448a4eeb80 1 Processor -- start 2023-07-06T20:23:53.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.109+0000 7f448a4eeb80 1 -- start start 2023-07-06T20:23:53.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.110+0000 7f448a4eeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4306f20 0x55acc430b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.110+0000 7f448a4eeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4307870 0x55acc4317e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.110+0000 7f448a4eeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 0x55acc430b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.110+0000 7f448a4eeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc42d5d60 con 0x55acc4306f20 2023-07-06T20:23:53.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.110+0000 7f448a4eeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc41b14f0 con 0x55acc4307870 2023-07-06T20:23:53.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.110+0000 7f448a4eeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc42fc210 con 0x55acc4308390 2023-07-06T20:23:53.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.110+0000 7f447519c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 0x55acc430b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.111+0000 7f447519c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 0x55acc430b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:49126/0 (socket says 172.21.15.69:49126) 2023-07-06T20:23:53.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.111+0000 7f447519c700 1 -- 172.21.15.69:0/3061116523 learned_addr learned my addr 172.21.15.69:0/3061116523 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:23:53.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.111+0000 7f446ffff700 1 --2- 172.21.15.69:0/3061116523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4307870 0x55acc4317e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.112+0000 7f447519c700 1 -- 172.21.15.69:0/3061116523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4307870 msgr2=0x55acc4317e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:53.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.112+0000 7f447519c700 1 --2- 172.21.15.69:0/3061116523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4307870 0x55acc4317e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.112+0000 7f447519c700 1 -- 172.21.15.69:0/3061116523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4306f20 msgr2=0x55acc430b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:53.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.112+0000 7f447519c700 1 --2- 172.21.15.69:0/3061116523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4306f20 0x55acc430b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.112+0000 7f447519c700 1 -- 172.21.15.69:0/3061116523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acc40ac250 con 0x55acc4308390 2023-07-06T20:23:53.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.112+0000 7f447519c700 1 --2- 172.21.15.69:0/3061116523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 0x55acc430b790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f44680075f0 tx=0x7f446801e070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0a83daf28362a93 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:23:53.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.116+0000 7f445b7fe700 1 -- 172.21.15.69:0/3061116523 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446800e040 con 0x55acc4308390 2023-07-06T20:23:53.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.116+0000 7f445b7fe700 1 -- 172.21.15.69:0/3061116523 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f446804e070 con 0x55acc4308390 2023-07-06T20:23:53.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.116+0000 7f445b7fe700 1 -- 172.21.15.69:0/3061116523 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468053ea0 con 0x55acc4308390 2023-07-06T20:23:53.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.117+0000 7f448a4eeb80 1 -- 172.21.15.69:0/3061116523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 msgr2=0x55acc430b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:53.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.117+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/3061116523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 0x55acc430b790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f44680075f0 tx=0x7f446801e070 comp rx=0 tx=0).stop 2023-07-06T20:23:53.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.118+0000 7f448a4eeb80 1 -- 172.21.15.69:0/3061116523 shutdown_connections 2023-07-06T20:23:53.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.118+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/3061116523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 0x55acc430b790 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.118+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/3061116523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4306f20 0x55acc430b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.118+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/3061116523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4307870 0x55acc4317e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.118+0000 7f448a4eeb80 1 -- 172.21.15.69:0/3061116523 >> 172.21.15.69:0/3061116523 conn(0x55acc42fc640 msgr2=0x55acc430e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:23:53.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.118+0000 7f448a4eeb80 1 -- 172.21.15.69:0/3061116523 shutdown_connections 2023-07-06T20:23:53.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.118+0000 7f448a4eeb80 1 -- 172.21.15.69:0/3061116523 wait complete. 2023-07-06T20:23:53.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.118+0000 7f448a4eeb80 1 -- start start 2023-07-06T20:23:53.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.119+0000 7f448a4eeb80 1 Processor -- start 2023-07-06T20:23:53.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.119+0000 7f448a4eeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc4314fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.119+0000 7f448a4eeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4307870 0x55acc43154f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.119+0000 7f448a4eeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 0x55acc4316500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.119+0000 7f448a4eeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc4316a40 con 0x55acc4307870 2023-07-06T20:23:53.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.119+0000 7f448a4eeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc4316bb0 con 0x55acc4306f20 2023-07-06T20:23:53.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.119+0000 7f448a4eeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acc4316d70 con 0x55acc4308390 2023-07-06T20:23:53.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.119+0000 7f448a4eeb80 10 client.0 _ll_register_callbacks cb 0x7f4468009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:23:53.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f447499b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc4314fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f447499b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc4314fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:60306/0 (socket says 172.21.15.69:60306) 2023-07-06T20:23:53.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f447499b700 1 -- 172.21.15.69:0/224031738 learned_addr learned my addr 172.21.15.69:0/224031738 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:23:53.126 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f446ffff700 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4307870 0x55acc43154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.126 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f447519c700 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 0x55acc4316500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f447499b700 1 -- 172.21.15.69:0/224031738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 msgr2=0x55acc4316500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:53.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f447499b700 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 0x55acc4316500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f447499b700 1 -- 172.21.15.69:0/224031738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4307870 msgr2=0x55acc43154f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:23:53.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f447499b700 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4307870 0x55acc43154f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f447499b700 1 -- 172.21.15.69:0/224031738 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4470104740 con 0x55acc4306f20 2023-07-06T20:23:53.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f447519c700 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 0x55acc4316500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:23:53.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f446ffff700 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4307870 0x55acc43154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:23:53.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f447499b700 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc4314fb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4464097850 tx=0x7f4464098cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5334a9a380090fb5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:23:53.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.120+0000 7f444ffff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:23:53.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.121+0000 7f444ffff700 1 -- 172.21.15.69:0/224031738 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44640a16c0 con 0x55acc4306f20 2023-07-06T20:23:53.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.121+0000 7f444ffff700 1 -- 172.21.15.69:0/224031738 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f4464010380 con 0x55acc4306f20 2023-07-06T20:23:53.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.121+0000 7f448a4eeb80 1 -- 172.21.15.69:0/224031738 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4464010ee0 con 0x55acc4306f20 2023-07-06T20:23:53.128 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.122+0000 7f444ffff700 1 -- 172.21.15.69:0/224031738 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4464099a20 con 0x55acc4306f20 2023-07-06T20:23:53.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.125+0000 7f448a4eeb80 10 client.25565 fetch_fsmap learned FSMap version 30 2023-07-06T20:23:53.129 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.125+0000 7f448a4eeb80 1 -- 172.21.15.69:0/224031738 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55acc40842b0 con 0x55acc4306f20 2023-07-06T20:23:53.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.127+0000 7f444ffff700 1 -- 172.21.15.69:0/224031738 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44640c7080 con 0x55acc4306f20 2023-07-06T20:23:53.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.127+0000 7f444ffff700 1 -- 172.21.15.69:0/224031738 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f446409d070 con 0x55acc4306f20 2023-07-06T20:23:53.134 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.131+0000 7f448a4eeb80 10 client.25565 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:23:53.134 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.131+0000 7f448a4eeb80 10 client.25565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:23:53.134 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.131+0000 7f448a4eeb80 20 client.25565 populate_metadata read hostname 'smithi069' 2023-07-06T20:23:53.134 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.131+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55acc431e3e0 0x55acc43207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:23:53.134 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.131+0000 7f448a4eeb80 4 client.25565 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:23:53.134 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.131+0000 7f448a4eeb80 1 -- 172.21.15.69:0/224031738 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55acc4250eb0 con 0x55acc431e3e0 2023-07-06T20:23:53.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.152+0000 7f446ffff700 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55acc431e3e0 0x55acc43207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:23:53.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.166+0000 7f446ffff700 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55acc431e3e0 0x55acc43207f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5debda2856d13025 server_cookie=b0e91378e8594ea2 in_seq=0 out_seq=0 2023-07-06T20:23:53.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.166+0000 7f444ffff700 10 client.25565 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:23:53.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.173+0000 7f444ffff700 1 -- 172.21.15.69:0/224031738 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55acc4250eb0 con 0x55acc431e3e0 2023-07-06T20:23:53.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.173+0000 7f444ffff700 10 client.25565 handle_command_reply: tid=0 2023-07-06T20:23:53.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.173+0000 7f448a4eeb80 1 client.25565 shutdown 2023-07-06T20:23:53.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.174+0000 7f448a4eeb80 1 -- 172.21.15.69:0/224031738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 msgr2=0x55acc4314fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:53.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.174+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc4314fb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4464097850 tx=0x7f4464098cd0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.174+0000 7f448a4eeb80 1 -- 172.21.15.69:0/224031738 shutdown_connections 2023-07-06T20:23:53.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.174+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55acc431e3e0 0x55acc43207f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.174+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acc4308390 0x55acc4316500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.174+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55acc4306f20 0x55acc4314fb0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4464097850 tx=0x7f4464098cd0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.174+0000 7f448a4eeb80 1 --2- 172.21.15.69:0/224031738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55acc4307870 0x55acc43154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.174+0000 7f448a4eeb80 1 -- 172.21.15.69:0/224031738 >> 172.21.15.69:0/224031738 conn(0x55acc42fc640 msgr2=0x55acc430e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:23:53.181 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.179+0000 7f444ffff700 0 client.25565 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:23:53.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.180+0000 7f448a4eeb80 1 -- 172.21.15.69:0/224031738 shutdown_connections 2023-07-06T20:23:53.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.181+0000 7f448a4eeb80 1 -- 172.21.15.69:0/224031738 wait complete. 2023-07-06T20:23:53.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.183+0000 7f448a4eeb80 20 client.25565 trim_cache size 0 max 16384 2023-07-06T20:23:53.190 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:23:53.190 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:23:53.191 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:23:53.191 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:23:53.191 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:23:53.191 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:23:53.191 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:23:53.191 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:23:53.191 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:23:53.191 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:23:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.190+0000 7f4476bff700 1 -- 172.21.15.69:0/1445563989 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f44600a1220 msgr2=0x7f44600a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.190+0000 7f4476bff700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f44600a1220 0x7f44600a36d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f446800b6c0 tx=0x7f4468009510 comp rx=0 tx=0).stop 2023-07-06T20:23:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.190+0000 7f4476bff700 1 -- 172.21.15.69:0/1445563989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44700fe480 msgr2=0x7f4470109850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:23:53.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.190+0000 7f4476bff700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44700fe480 0x7f4470109850 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f446400b530 tx=0x7f446400b8f0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.197+0000 7f4476bff700 1 -- 172.21.15.69:0/1445563989 shutdown_connections 2023-07-06T20:23:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.197+0000 7f4476bff700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f44600a1220 0x7f44600a36d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.197+0000 7f4476bff700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44701037a0 0x7f44701ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.197+0000 7f4476bff700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44700fe480 0x7f4470109850 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.197+0000 7f4476bff700 1 --2- 172.21.15.69:0/1445563989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4470100de0 0x7f4470109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:23:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.197+0000 7f4476bff700 1 -- 172.21.15.69:0/1445563989 >> 172.21.15.69:0/1445563989 conn(0x7f44700fa130 msgr2=0x7f44700ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:23:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.197+0000 7f4476bff700 1 -- 172.21.15.69:0/1445563989 shutdown_connections 2023-07-06T20:23:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:23:53.197+0000 7f4476bff700 1 -- 172.21.15.69:0/1445563989 wait complete. 2023-07-06T20:23:53.227 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:23:53.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:53 smithi069 ceph-mon[126399]: pgmap v464: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 72 MiB/s wr, 4.21k op/s 2023-07-06T20:23:53.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:53 smithi098 ceph-mon[128968]: pgmap v464: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 72 MiB/s wr, 4.21k op/s 2023-07-06T20:23:53.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:53 smithi156 ceph-mon[129280]: pgmap v464: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 72 MiB/s wr, 4.21k op/s 2023-07-06T20:23:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 125.0 s, 2933.0 tps, lat 33.888 ms stddev 36.410 2023-07-06T20:23:55.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:55 smithi069 ceph-mon[126399]: pgmap v465: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 69 MiB/s wr, 4.18k op/s 2023-07-06T20:23:55.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:55 smithi156 ceph-mon[129280]: pgmap v465: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 69 MiB/s wr, 4.18k op/s 2023-07-06T20:23:55.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:55 smithi098 ceph-mon[128968]: pgmap v465: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 69 MiB/s wr, 4.18k op/s 2023-07-06T20:23:57.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:57 smithi069 ceph-mon[126399]: pgmap v466: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 65 MiB/s wr, 3.96k op/s 2023-07-06T20:23:57.864 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:57 smithi156 ceph-mon[129280]: pgmap v466: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 65 MiB/s wr, 3.96k op/s 2023-07-06T20:23:57.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:57 smithi098 ceph-mon[128968]: pgmap v466: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 65 MiB/s wr, 3.96k op/s 2023-07-06T20:23:59.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:23:58 smithi156 ceph-mon[129280]: pgmap v467: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 70 MiB/s wr, 4.09k op/s 2023-07-06T20:23:59.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:23:58 smithi098 ceph-mon[128968]: pgmap v467: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 70 MiB/s wr, 4.09k op/s 2023-07-06T20:23:59.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 130.0 s, 1709.7 tps, lat 58.886 ms stddev 94.002 2023-07-06T20:23:59.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:23:58 smithi069 ceph-mon[126399]: pgmap v467: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 70 MiB/s wr, 4.09k op/s 2023-07-06T20:24:01.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:00 smithi156 ceph-mon[129280]: pgmap v468: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 62 MiB/s wr, 3.34k op/s 2023-07-06T20:24:01.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:00 smithi098 ceph-mon[128968]: pgmap v468: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 62 MiB/s wr, 3.34k op/s 2023-07-06T20:24:01.263 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:00 smithi069 ceph-mon[126399]: pgmap v468: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 62 MiB/s wr, 3.34k op/s 2023-07-06T20:24:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:02 smithi156 ceph-mon[129280]: pgmap v469: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 73 MiB/s wr, 4.35k op/s 2023-07-06T20:24:03.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:02 smithi098 ceph-mon[128968]: pgmap v469: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 73 MiB/s wr, 4.35k op/s 2023-07-06T20:24:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:02 smithi069 ceph-mon[126399]: pgmap v469: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 73 MiB/s wr, 4.35k op/s 2023-07-06T20:24:04.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 135.0 s, 2771.3 tps, lat 36.079 ms stddev 54.909 2023-07-06T20:24:05.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:04 smithi098 ceph-mon[128968]: pgmap v470: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 71 MiB/s wr, 4.75k op/s 2023-07-06T20:24:05.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:04 smithi069 ceph-mon[126399]: pgmap v470: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 71 MiB/s wr, 4.75k op/s 2023-07-06T20:24:05.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:04 smithi156 ceph-mon[129280]: pgmap v470: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 71 MiB/s wr, 4.75k op/s 2023-07-06T20:24:07.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:06 smithi069 ceph-mon[126399]: pgmap v471: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 68 MiB/s wr, 4.84k op/s 2023-07-06T20:24:07.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:06 smithi156 ceph-mon[129280]: pgmap v471: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 68 MiB/s wr, 4.84k op/s 2023-07-06T20:24:07.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:06 smithi098 ceph-mon[128968]: pgmap v471: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 68 MiB/s wr, 4.84k op/s 2023-07-06T20:24:09.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 140.0 s, 2682.1 tps, lat 34.946 ms stddev 33.954 2023-07-06T20:24:09.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:08 smithi069 ceph-mon[126399]: pgmap v472: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 MiB/s wr, 4.99k op/s 2023-07-06T20:24:09.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:08 smithi156 ceph-mon[129280]: pgmap v472: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 MiB/s wr, 4.99k op/s 2023-07-06T20:24:09.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:08 smithi098 ceph-mon[128968]: pgmap v472: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 MiB/s wr, 4.99k op/s 2023-07-06T20:24:11.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:10 smithi069 ceph-mon[126399]: pgmap v473: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 62 MiB/s wr, 3.90k op/s 2023-07-06T20:24:11.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:10 smithi156 ceph-mon[129280]: pgmap v473: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 62 MiB/s wr, 3.90k op/s 2023-07-06T20:24:11.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:10 smithi098 ceph-mon[128968]: pgmap v473: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 62 MiB/s wr, 3.90k op/s 2023-07-06T20:24:13.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:12 smithi069 ceph-mon[126399]: pgmap v474: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 MiB/s wr, 4.19k op/s 2023-07-06T20:24:13.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:12 smithi156 ceph-mon[129280]: pgmap v474: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 MiB/s wr, 4.19k op/s 2023-07-06T20:24:13.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:12 smithi098 ceph-mon[128968]: pgmap v474: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 MiB/s wr, 4.19k op/s 2023-07-06T20:24:14.233 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 145.0 s, 2255.2 tps, lat 47.158 ms stddev 65.800 2023-07-06T20:24:15.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:15 smithi156 ceph-mon[129280]: pgmap v475: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 61 MiB/s wr, 3.72k op/s 2023-07-06T20:24:15.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:15 smithi098 ceph-mon[128968]: pgmap v475: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 61 MiB/s wr, 3.72k op/s 2023-07-06T20:24:15.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:15 smithi069 ceph-mon[126399]: pgmap v475: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 61 MiB/s wr, 3.72k op/s 2023-07-06T20:24:17.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:17 smithi156 ceph-mon[129280]: pgmap v476: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 59 MiB/s wr, 3.50k op/s 2023-07-06T20:24:17.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:17 smithi098 ceph-mon[128968]: pgmap v476: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 59 MiB/s wr, 3.50k op/s 2023-07-06T20:24:17.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:17 smithi069 ceph-mon[126399]: pgmap v476: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 59 MiB/s wr, 3.50k op/s 2023-07-06T20:24:19.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 150.0 s, 2807.9 tps, lat 35.601 ms stddev 38.875 2023-07-06T20:24:19.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:19 smithi156 ceph-mon[129280]: pgmap v477: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 64 MiB/s wr, 3.40k op/s 2023-07-06T20:24:19.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:19 smithi098 ceph-mon[128968]: pgmap v477: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 64 MiB/s wr, 3.40k op/s 2023-07-06T20:24:19.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:19 smithi069 ceph-mon[126399]: pgmap v477: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 64 MiB/s wr, 3.40k op/s 2023-07-06T20:24:21.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:21 smithi098 ceph-mon[128968]: pgmap v478: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 57 MiB/s wr, 3.24k op/s 2023-07-06T20:24:21.514 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:21 smithi069 ceph-mon[126399]: pgmap v478: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 57 MiB/s wr, 3.24k op/s 2023-07-06T20:24:21.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:21 smithi156 ceph-mon[129280]: pgmap v478: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 57 MiB/s wr, 3.24k op/s 2023-07-06T20:24:23.229 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:24:23.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:23 smithi069 ceph-mon[126399]: pgmap v479: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 MiB/s wr, 3.37k op/s 2023-07-06T20:24:23.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:23 smithi156 ceph-mon[129280]: pgmap v479: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 MiB/s wr, 3.37k op/s 2023-07-06T20:24:23.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:23 smithi098 ceph-mon[128968]: pgmap v479: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 MiB/s wr, 3.37k op/s 2023-07-06T20:24:23.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.718+0000 7fd67a9ff700 1 -- 172.21.15.69:0/2003584672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd674107b50 msgr2=0x7fd674107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:23.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.718+0000 7fd67a9ff700 1 --2- 172.21.15.69:0/2003584672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd674107b50 0x7fd674107f60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd664009990 tx=0x7fd664009ca0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.718+0000 7fd67a9ff700 1 -- 172.21.15.69:0/2003584672 shutdown_connections 2023-07-06T20:24:23.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.718+0000 7fd67a9ff700 1 --2- 172.21.15.69:0/2003584672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd674071750 0x7fd674110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.718+0000 7fd67a9ff700 1 --2- 172.21.15.69:0/2003584672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd674107b50 0x7fd674107f60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.718+0000 7fd67a9ff700 1 --2- 172.21.15.69:0/2003584672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd674070da0 0x7fd674071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.718+0000 7fd67a9ff700 1 -- 172.21.15.69:0/2003584672 >> 172.21.15.69:0/2003584672 conn(0x7fd67406cbc0 msgr2=0x7fd67406eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:24:23.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.719+0000 7fd67a9ff700 1 -- 172.21.15.69:0/2003584672 shutdown_connections 2023-07-06T20:24:23.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.719+0000 7fd67a9ff700 1 -- 172.21.15.69:0/2003584672 wait complete. 2023-07-06T20:24:23.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.720+0000 7fd67a9ff700 1 Processor -- start 2023-07-06T20:24:23.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.720+0000 7fd67a9ff700 1 -- start start 2023-07-06T20:24:23.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.720+0000 7fd67a9ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd674070da0 0x7fd67410e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.720+0000 7fd67a9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd674071750 0x7fd67410e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.721+0000 7fd67a9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd674107b50 0x7fd67410ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.721+0000 7fd67a9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67410b600 con 0x7fd674107b50 2023-07-06T20:24:23.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.721+0000 7fd67a9ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67410b770 con 0x7fd674070da0 2023-07-06T20:24:23.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.721+0000 7fd67a9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67410b930 con 0x7fd674071750 2023-07-06T20:24:23.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.721+0000 7fd673fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd674070da0 0x7fd67410e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:23.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.721+0000 7fd673fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd674070da0 0x7fd67410e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:57972/0 (socket says 172.21.15.69:57972) 2023-07-06T20:24:23.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.721+0000 7fd673fff700 1 -- 172.21.15.69:0/735196855 learned_addr learned my addr 172.21.15.69:0/735196855 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:24:23.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.721+0000 7fd6737fe700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd674071750 0x7fd67410e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:23.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.721+0000 7fd673fff700 1 -- 172.21.15.69:0/735196855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd674071750 msgr2=0x7fd67410e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:23.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.721+0000 7fd673fff700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd674071750 0x7fd67410e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.722+0000 7fd673fff700 1 -- 172.21.15.69:0/735196855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd674107b50 msgr2=0x7fd67410ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:23.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.722+0000 7fd673fff700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd674107b50 0x7fd67410ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.722+0000 7fd673fff700 1 -- 172.21.15.69:0/735196855 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6640095f0 con 0x7fd674070da0 2023-07-06T20:24:23.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.722+0000 7fd673fff700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd674070da0 0x7fd67410e2d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd664005910 tx=0x7fd66400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:24:23.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.722+0000 7fd6717fa700 1 -- 172.21.15.69:0/735196855 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66401d070 con 0x7fd674070da0 2023-07-06T20:24:23.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.722+0000 7fd6717fa700 1 -- 172.21.15.69:0/735196855 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd66400fb80 con 0x7fd674070da0 2023-07-06T20:24:23.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.722+0000 7fd6717fa700 1 -- 172.21.15.69:0/735196855 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664017650 con 0x7fd674070da0 2023-07-06T20:24:23.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.722+0000 7fd67a9ff700 1 -- 172.21.15.69:0/735196855 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6741ac910 con 0x7fd674070da0 2023-07-06T20:24:23.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.723+0000 7fd67a9ff700 1 -- 172.21.15.69:0/735196855 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6741acce0 con 0x7fd674070da0 2023-07-06T20:24:23.731 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.729+0000 7fd6717fa700 1 -- 172.21.15.69:0/735196855 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd664022470 con 0x7fd674070da0 2023-07-06T20:24:23.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.731+0000 7fd6717fa700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd6600a1250 0x7fd6600a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.732+0000 7fd6717fa700 1 -- 172.21.15.69:0/735196855 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd6640cdd90 con 0x7fd674070da0 2023-07-06T20:24:23.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.733+0000 7fd68e2e1b80 1 Processor -- start 2023-07-06T20:24:23.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.733+0000 7fd68e2e1b80 1 -- start start 2023-07-06T20:24:23.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.734+0000 7fd6737fe700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd6600a1250 0x7fd6600a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:23.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.734+0000 7fd68e2e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab660113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.734+0000 7fd68e2e1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 0x55ab66012240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.734+0000 7fd68e2e1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66012780 0x55ab66019020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.734+0000 7fd68e2e1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65f04cb0 con 0x55ab66012780 2023-07-06T20:24:23.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.734+0000 7fd68e2e1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65f57a40 con 0x55ab66011dd0 2023-07-06T20:24:23.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.734+0000 7fd68e2e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65f50ed0 con 0x55ab66010f80 2023-07-06T20:24:23.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.735+0000 7fd6737fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 0x55ab66012240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:23.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.735+0000 7fd6737fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 0x55ab66012240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:57982/0 (socket says 172.21.15.69:57982) 2023-07-06T20:24:23.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.735+0000 7fd6737fe700 1 -- 172.21.15.69:0/3503387902 learned_addr learned my addr 172.21.15.69:0/3503387902 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:24:23.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.735+0000 7fd673fff700 1 --2- 172.21.15.69:0/3503387902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab660113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:23.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.735+0000 7fd6737fe700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd6600a1250 0x7fd6600a3700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009f80 tx=0x7fd65c015690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:24:23.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.735+0000 7fd6737fe700 1 -- 172.21.15.69:0/3503387902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 msgr2=0x55ab660113b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:23.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.735+0000 7fd6737fe700 1 --2- 172.21.15.69:0/3503387902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab660113b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.736+0000 7fd6737fe700 1 -- 172.21.15.69:0/3503387902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66012780 msgr2=0x55ab66019020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:24:23.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.736+0000 7fd6737fe700 1 --2- 172.21.15.69:0/3503387902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66012780 0x55ab66019020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.736+0000 7fd6737fe700 1 -- 172.21.15.69:0/3503387902 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab65d7ede0 con 0x55ab66011dd0 2023-07-06T20:24:23.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.736+0000 7fd6737fe700 1 --2- 172.21.15.69:0/3503387902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 0x55ab66012240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd65c015a90 tx=0x7fd65c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca0183fc2afd7a8a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:24:23.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.736+0000 7fd66a7fc700 1 -- 172.21.15.69:0/3503387902 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c018d90 con 0x55ab66011dd0 2023-07-06T20:24:23.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.736+0000 7fd66a7fc700 1 -- 172.21.15.69:0/3503387902 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd65c0128e0 con 0x55ab66011dd0 2023-07-06T20:24:23.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.737+0000 7fd66a7fc700 1 -- 172.21.15.69:0/3503387902 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c023790 con 0x55ab66011dd0 2023-07-06T20:24:23.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.737+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3503387902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 msgr2=0x55ab66012240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:23.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.737+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/3503387902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 0x55ab66012240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd65c015a90 tx=0x7fd65c01d000 comp rx=0 tx=0).stop 2023-07-06T20:24:23.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.737+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3503387902 shutdown_connections 2023-07-06T20:24:23.741 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.737+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/3503387902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66012780 0x55ab66019020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.741 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.737+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/3503387902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab660113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.741 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.737+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/3503387902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 0x55ab66012240 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.741 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.737+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3503387902 >> 172.21.15.69:0/3503387902 conn(0x55ab65ff5640 msgr2=0x55ab65ffa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:24:23.741 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.738+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3503387902 shutdown_connections 2023-07-06T20:24:23.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.738+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3503387902 wait complete. 2023-07-06T20:24:23.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.739+0000 7fd68e2e1b80 1 -- start start 2023-07-06T20:24:23.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.739+0000 7fd68e2e1b80 1 Processor -- start 2023-07-06T20:24:23.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.740+0000 7fd68e2e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab6601a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.740+0000 7fd68e2e1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66011dd0 0x55ab6601ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.740+0000 7fd68e2e1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66012780 0x55ab66021dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.740+0000 7fd68e2e1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65ef0ac0 con 0x55ab66011dd0 2023-07-06T20:24:23.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.740+0000 7fd68e2e1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65ef07c0 con 0x55ab66012780 2023-07-06T20:24:23.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.740+0000 7fd68e2e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65dd4920 con 0x55ab66010f80 2023-07-06T20:24:23.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.740+0000 7fd68e2e1b80 10 client.0 _ll_register_callbacks cb 0x55ab65eee760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:24:23.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.740+0000 7fd673fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab6601a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:23.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.740+0000 7fd673fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab6601a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:57526/0 (socket says 172.21.15.69:57526) 2023-07-06T20:24:23.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.740+0000 7fd673fff700 1 -- 172.21.15.69:0/1145583152 learned_addr learned my addr 172.21.15.69:0/1145583152 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:24:23.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.741+0000 7fd673fff700 1 -- 172.21.15.69:0/1145583152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66012780 msgr2=0x55ab66021dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:24:23.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.741+0000 7fd673fff700 1 --2- 172.21.15.69:0/1145583152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66012780 0x55ab66021dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.741+0000 7fd673fff700 1 -- 172.21.15.69:0/1145583152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66011dd0 msgr2=0x55ab6601ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:23.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.741+0000 7fd673fff700 1 --2- 172.21.15.69:0/1145583152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66011dd0 0x55ab6601ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.741+0000 7fd673fff700 1 -- 172.21.15.69:0/1145583152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c015d40 con 0x55ab66010f80 2023-07-06T20:24:23.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.744+0000 7fd673fff700 1 --2- 172.21.15.69:0/1145583152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab6601a840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd6640cb800 tx=0x7fd6640a11b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e58258270acd2dd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:24:23.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.744+0000 7fd6537fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:24:23.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.744+0000 7fd6537fe700 1 -- 172.21.15.69:0/1145583152 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66409f1d0 con 0x55ab66010f80 2023-07-06T20:24:23.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.745+0000 7fd6537fe700 1 -- 172.21.15.69:0/1145583152 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd6640a3560 con 0x55ab66010f80 2023-07-06T20:24:23.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.745+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/1145583152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab65eaa4f0 con 0x55ab66010f80 2023-07-06T20:24:23.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.745+0000 7fd6537fe700 1 -- 172.21.15.69:0/1145583152 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd66409a690 con 0x55ab66010f80 2023-07-06T20:24:23.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.745+0000 7fd68e2e1b80 10 client.25522 fetch_fsmap learned FSMap version 30 2023-07-06T20:24:23.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.745+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/1145583152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ab65d67c60 con 0x55ab66010f80 2023-07-06T20:24:23.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.745+0000 7fd6537fe700 1 -- 172.21.15.69:0/1145583152 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab65eaa4f0 con 0x55ab66010f80 2023-07-06T20:24:23.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.746+0000 7fd6537fe700 1 -- 172.21.15.69:0/1145583152 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd6640b9240 con 0x55ab66010f80 2023-07-06T20:24:23.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.746+0000 7fd68e2e1b80 10 client.25522 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:24:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.746+0000 7fd68e2e1b80 10 client.25522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:24:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.746+0000 7fd68e2e1b80 20 client.25522 populate_metadata read hostname 'smithi069' 2023-07-06T20:24:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.746+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/1145583152 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ab66026200 0x55ab660285e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.746+0000 7fd68e2e1b80 4 client.25522 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:24:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.746+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/1145583152 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab65fced60 con 0x55ab66026200 2023-07-06T20:24:23.749 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.746+0000 7fd6737fe700 1 --2- 172.21.15.69:0/1145583152 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ab66026200 0x55ab660285e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.747+0000 7fd6737fe700 1 --2- 172.21.15.69:0/1145583152 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ab66026200 0x55ab660285e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=155392cd6e47ef6e server_cookie=22b12bb1b0931cb1 in_seq=0 out_seq=0 2023-07-06T20:24:23.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.747+0000 7fd6537fe700 10 client.25522 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:24:23.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.748+0000 7fd6537fe700 1 -- 172.21.15.69:0/1145583152 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ab65fced60 con 0x55ab66026200 2023-07-06T20:24:23.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.748+0000 7fd6537fe700 10 client.25522 handle_command_reply: tid=0 2023-07-06T20:24:23.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.748+0000 7fd68e2e1b80 1 client.25522 shutdown 2023-07-06T20:24:23.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.749+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/1145583152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 msgr2=0x55ab6601a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:23.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.749+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/1145583152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab6601a840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd6640cb800 tx=0x7fd6640a11b0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.753+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/1145583152 shutdown_connections 2023-07-06T20:24:23.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.753+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/1145583152 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ab66026200 0x55ab660285e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.753+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/1145583152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66012780 0x55ab66021dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.753+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/1145583152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab6601a840 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.753+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/1145583152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66011dd0 0x55ab6601ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.753+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/1145583152 >> 172.21.15.69:0/1145583152 conn(0x55ab65ff5640 msgr2=0x55ab65ffa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:24:23.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.753+0000 7fd6537fe700 0 client.25522 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:24:23.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.753+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/1145583152 shutdown_connections 2023-07-06T20:24:23.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.754+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/1145583152 wait complete. 2023-07-06T20:24:23.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.755+0000 7fd68e2e1b80 20 client.25522 trim_cache size 0 max 16384 2023-07-06T20:24:23.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.790+0000 7fd68e2e1b80 1 Processor -- start 2023-07-06T20:24:23.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.790+0000 7fd68e2e1b80 1 -- start start 2023-07-06T20:24:23.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.791+0000 7fd68e2e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab66028210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.791+0000 7fd68e2e1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66011dd0 0x55ab66021d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.791+0000 7fd68e2e1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66012780 0x55ab660156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.791+0000 7fd68e2e1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab66015250 con 0x55ab66011dd0 2023-07-06T20:24:23.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.791+0000 7fd68e2e1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab66019cf0 con 0x55ab66012780 2023-07-06T20:24:23.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.791+0000 7fd68e2e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6601a3b0 con 0x55ab66010f80 2023-07-06T20:24:23.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.793+0000 7fd673fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab66028210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:23.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.793+0000 7fd678f9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66012780 0x55ab660156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:23.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.793+0000 7fd678f9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66012780 0x55ab660156d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:57992/0 (socket says 172.21.15.69:57992) 2023-07-06T20:24:23.795 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.793+0000 7fd678f9c700 1 -- 172.21.15.69:0/288900525 learned_addr learned my addr 172.21.15.69:0/288900525 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:24:23.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.794+0000 7fd678f9c700 1 -- 172.21.15.69:0/288900525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 msgr2=0x55ab66028210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:23.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.794+0000 7fd678f9c700 1 --2- 172.21.15.69:0/288900525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab66028210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.794+0000 7fd678f9c700 1 -- 172.21.15.69:0/288900525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66011dd0 msgr2=0x55ab66021d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:23.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.794+0000 7fd678f9c700 1 --2- 172.21.15.69:0/288900525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66011dd0 0x55ab66021d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.794+0000 7fd678f9c700 1 -- 172.21.15.69:0/288900525 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab65da5250 con 0x55ab66012780 2023-07-06T20:24:23.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.794+0000 7fd678f9c700 1 --2- 172.21.15.69:0/288900525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66012780 0x55ab660156d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0094f0 tx=0x7fd66c009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=785c8fb3284c2818 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:24:23.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.794+0000 7fd66affd700 1 -- 172.21.15.69:0/288900525 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c017070 con 0x55ab66012780 2023-07-06T20:24:23.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.795+0000 7fd66affd700 1 -- 172.21.15.69:0/288900525 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd66c00f040 con 0x55ab66012780 2023-07-06T20:24:23.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.795+0000 7fd66affd700 1 -- 172.21.15.69:0/288900525 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0057d0 con 0x55ab66012780 2023-07-06T20:24:23.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.796+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/288900525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66012780 msgr2=0x55ab660156d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:23.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.796+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/288900525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66012780 0x55ab660156d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0094f0 tx=0x7fd66c009ec0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.797+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/288900525 shutdown_connections 2023-07-06T20:24:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.797+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/288900525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66012780 0x55ab660156d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.797+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/288900525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66010f80 0x55ab66028210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.797+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/288900525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66011dd0 0x55ab66021d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.797+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/288900525 >> 172.21.15.69:0/288900525 conn(0x55ab65ff5640 msgr2=0x55ab65ffa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:24:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.798+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/288900525 shutdown_connections 2023-07-06T20:24:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.798+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/288900525 wait complete. 2023-07-06T20:24:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.798+0000 7fd68e2e1b80 1 -- start start 2023-07-06T20:24:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.799+0000 7fd68e2e1b80 1 Processor -- start 2023-07-06T20:24:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.800+0000 7fd68e2e1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66010f80 0x55ab66021ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.800+0000 7fd68e2e1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 0x55ab660202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.800+0000 7fd68e2e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66012780 0x55ab660208b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.800+0000 7fd68e2e1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab66020fc0 con 0x55ab66010f80 2023-07-06T20:24:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.800+0000 7fd68e2e1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab66021130 con 0x55ab66011dd0 2023-07-06T20:24:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.800+0000 7fd68e2e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6607a0c0 con 0x55ab66012780 2023-07-06T20:24:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.800+0000 7fd68e2e1b80 10 client.0 _ll_register_callbacks cb 0x55ab65eee760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:24:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.801+0000 7fd6737fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 0x55ab660202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.801+0000 7fd6737fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 0x55ab660202f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:57994/0 (socket says 172.21.15.69:57994) 2023-07-06T20:24:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.801+0000 7fd6737fe700 1 -- 172.21.15.69:0/3262744770 learned_addr learned my addr 172.21.15.69:0/3262744770 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:24:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.801+0000 7fd678f9c700 1 --2- 172.21.15.69:0/3262744770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66012780 0x55ab660208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.801+0000 7fd6737fe700 1 -- 172.21.15.69:0/3262744770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66012780 msgr2=0x55ab660208b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.801+0000 7fd6737fe700 1 --2- 172.21.15.69:0/3262744770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66012780 0x55ab660208b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.801+0000 7fd6737fe700 1 -- 172.21.15.69:0/3262744770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66010f80 msgr2=0x55ab66021ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.801+0000 7fd6737fe700 1 --2- 172.21.15.69:0/3262744770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66010f80 0x55ab66021ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.801+0000 7fd6737fe700 1 -- 172.21.15.69:0/3262744770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66c0098c0 con 0x55ab66011dd0 2023-07-06T20:24:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.802+0000 7fd6737fe700 1 --2- 172.21.15.69:0/3262744770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 0x55ab660202f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd65c006e40 tx=0x7fd65c009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1cd8964d2bbcde4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:24:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.802+0000 7fd6537fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:24:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.802+0000 7fd6537fe700 1 -- 172.21.15.69:0/3262744770 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0126b0 con 0x55ab66011dd0 2023-07-06T20:24:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.802+0000 7fd6537fe700 1 -- 172.21.15.69:0/3262744770 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd65c032ae0 con 0x55ab66011dd0 2023-07-06T20:24:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.803+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3262744770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd65c0128e0 con 0x55ab66011dd0 2023-07-06T20:24:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.803+0000 7fd6537fe700 1 -- 172.21.15.69:0/3262744770 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd65c009200 con 0x55ab66011dd0 2023-07-06T20:24:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.803+0000 7fd68e2e1b80 10 client.25580 fetch_fsmap learned FSMap version 30 2023-07-06T20:24:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.803+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3262744770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ab65d7d2b0 con 0x55ab66011dd0 2023-07-06T20:24:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.804+0000 7fd6537fe700 1 -- 172.21.15.69:0/3262744770 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd65c0128e0 con 0x55ab66011dd0 2023-07-06T20:24:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.804+0000 7fd6537fe700 1 -- 172.21.15.69:0/3262744770 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd65c018480 con 0x55ab66011dd0 2023-07-06T20:24:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.804+0000 7fd68e2e1b80 10 client.25580 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:24:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.804+0000 7fd68e2e1b80 10 client.25580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:24:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.804+0000 7fd68e2e1b80 20 client.25580 populate_metadata read hostname 'smithi069' 2023-07-06T20:24:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.804+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/3262744770 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ab6607ba40 0x55ab6607de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.804+0000 7fd68e2e1b80 4 client.25580 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:24:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.804+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3262744770 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ab65f49eb0 con 0x55ab6607ba40 2023-07-06T20:24:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.804+0000 7fd673fff700 1 --2- 172.21.15.69:0/3262744770 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ab6607ba40 0x55ab6607de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.805+0000 7fd673fff700 1 --2- 172.21.15.69:0/3262744770 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ab6607ba40 0x55ab6607de50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f53c1b22a88a4c server_cookie=385a2138690e40ef in_seq=0 out_seq=0 2023-07-06T20:24:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.805+0000 7fd6537fe700 10 client.25580 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:24:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.805+0000 7fd6537fe700 1 -- 172.21.15.69:0/3262744770 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ab65f49eb0 con 0x55ab6607ba40 2023-07-06T20:24:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.805+0000 7fd6537fe700 10 client.25580 handle_command_reply: tid=0 2023-07-06T20:24:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.806+0000 7fd68e2e1b80 1 client.25580 shutdown 2023-07-06T20:24:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.806+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3262744770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 msgr2=0x55ab660202f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.806+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/3262744770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 0x55ab660202f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd65c006e40 tx=0x7fd65c009a10 comp rx=0 tx=0).stop 2023-07-06T20:24:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.807+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3262744770 shutdown_connections 2023-07-06T20:24:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.807+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/3262744770 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ab6607ba40 0x55ab6607de50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.807+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/3262744770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab66012780 0x55ab660208b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.807+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/3262744770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ab66010f80 0x55ab66021ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.807+0000 7fd68e2e1b80 1 --2- 172.21.15.69:0/3262744770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab66011dd0 0x55ab660202f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.807+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3262744770 >> 172.21.15.69:0/3262744770 conn(0x55ab65ff5640 msgr2=0x55ab66028210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:24:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.807+0000 7fd6537fe700 0 client.25580 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:24:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.807+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3262744770 shutdown_connections 2023-07-06T20:24:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.807+0000 7fd68e2e1b80 1 -- 172.21.15.69:0/3262744770 wait complete. 2023-07-06T20:24:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.807+0000 7fd68e2e1b80 20 client.25580 trim_cache size 0 max 16384 2023-07-06T20:24:23.812 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:24:23.812 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:24:23.812 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:24:23.813 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:24:23.813 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:24:23.813 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:24:23.813 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:24:23.813 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:24:23.813 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:24:23.814 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:24:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.813+0000 7fd67a9ff700 1 -- 172.21.15.69:0/735196855 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd6600a1250 msgr2=0x7fd6600a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.813+0000 7fd67a9ff700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd6600a1250 0x7fd6600a3700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009f80 tx=0x7fd65c015690 comp rx=0 tx=0).stop 2023-07-06T20:24:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.813+0000 7fd67a9ff700 1 -- 172.21.15.69:0/735196855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd674070da0 msgr2=0x7fd67410e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.813+0000 7fd67a9ff700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd674070da0 0x7fd67410e2d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd664005910 tx=0x7fd66400f690 comp rx=0 tx=0).stop 2023-07-06T20:24:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.818+0000 7fd67a9ff700 1 -- 172.21.15.69:0/735196855 shutdown_connections 2023-07-06T20:24:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.819+0000 7fd67a9ff700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd6600a1250 0x7fd6600a3700 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.819+0000 7fd67a9ff700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd674107b50 0x7fd67410ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.819+0000 7fd67a9ff700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd674070da0 0x7fd67410e2d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.819+0000 7fd67a9ff700 1 --2- 172.21.15.69:0/735196855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd674071750 0x7fd67410e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.819+0000 7fd67a9ff700 1 -- 172.21.15.69:0/735196855 >> 172.21.15.69:0/735196855 conn(0x7fd67406cbc0 msgr2=0x7fd67406fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:24:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.819+0000 7fd67a9ff700 1 -- 172.21.15.69:0/735196855 shutdown_connections 2023-07-06T20:24:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:23.819+0000 7fd67a9ff700 1 -- 172.21.15.69:0/735196855 wait complete. 2023-07-06T20:24:23.846 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:24:24.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 155.0 s, 1729.2 tps, lat 56.858 ms stddev 91.500 2023-07-06T20:24:25.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:25 smithi069 ceph-mon[126399]: pgmap v480: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 62 MiB/s wr, 3.25k op/s 2023-07-06T20:24:25.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:25 smithi156 ceph-mon[129280]: pgmap v480: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 62 MiB/s wr, 3.25k op/s 2023-07-06T20:24:25.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:25 smithi098 ceph-mon[128968]: pgmap v480: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 62 MiB/s wr, 3.25k op/s 2023-07-06T20:24:27.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:27 smithi069 ceph-mon[126399]: pgmap v481: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 57 MiB/s wr, 2.91k op/s 2023-07-06T20:24:27.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:27 smithi156 ceph-mon[129280]: pgmap v481: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 57 MiB/s wr, 2.91k op/s 2023-07-06T20:24:27.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:27 smithi098 ceph-mon[128968]: pgmap v481: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 57 MiB/s wr, 2.91k op/s 2023-07-06T20:24:29.232 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 160.0 s, 3254.2 tps, lat 31.240 ms stddev 21.057 2023-07-06T20:24:29.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:29 smithi069 ceph-mon[126399]: pgmap v482: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 MiB/s wr, 3.91k op/s 2023-07-06T20:24:29.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:29 smithi156 ceph-mon[129280]: pgmap v482: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 MiB/s wr, 3.91k op/s 2023-07-06T20:24:29.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:29 smithi098 ceph-mon[128968]: pgmap v482: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 MiB/s wr, 3.91k op/s 2023-07-06T20:24:31.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:31 smithi069 ceph-mon[126399]: pgmap v483: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 64 MiB/s wr, 4.02k op/s 2023-07-06T20:24:31.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:31 smithi156 ceph-mon[129280]: pgmap v483: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 64 MiB/s wr, 4.02k op/s 2023-07-06T20:24:31.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:31 smithi098 ceph-mon[128968]: pgmap v483: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 64 MiB/s wr, 4.02k op/s 2023-07-06T20:24:33.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:33 smithi069 ceph-mon[126399]: pgmap v484: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 65 MiB/s wr, 4.07k op/s 2023-07-06T20:24:33.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:33 smithi156 ceph-mon[129280]: pgmap v484: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 65 MiB/s wr, 4.07k op/s 2023-07-06T20:24:33.628 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:33 smithi098 ceph-mon[128968]: pgmap v484: 97 pgs: 97 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 65 MiB/s wr, 4.07k op/s 2023-07-06T20:24:34.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 165.0 s, 1933.9 tps, lat 51.732 ms stddev 99.963 2023-07-06T20:24:35.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:35 smithi156 ceph-mon[129280]: pgmap v485: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 71 MiB/s wr, 4.75k op/s 2023-07-06T20:24:35.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:35 smithi098 ceph-mon[128968]: pgmap v485: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 71 MiB/s wr, 4.75k op/s 2023-07-06T20:24:35.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:35 smithi069 ceph-mon[126399]: pgmap v485: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 71 MiB/s wr, 4.75k op/s 2023-07-06T20:24:37.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:37 smithi156 ceph-mon[129280]: pgmap v486: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 60 MiB/s wr, 4.12k op/s 2023-07-06T20:24:37.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:37 smithi098 ceph-mon[128968]: pgmap v486: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 60 MiB/s wr, 4.12k op/s 2023-07-06T20:24:37.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:37 smithi069 ceph-mon[126399]: pgmap v486: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 60 MiB/s wr, 4.12k op/s 2023-07-06T20:24:39.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 170.0 s, 2190.5 tps, lat 45.356 ms stddev 57.312 2023-07-06T20:24:39.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:39 smithi156 ceph-mon[129280]: pgmap v487: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 71 MiB/s wr, 4.70k op/s 2023-07-06T20:24:39.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:39 smithi098 ceph-mon[128968]: pgmap v487: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 71 MiB/s wr, 4.70k op/s 2023-07-06T20:24:39.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:39 smithi069 ceph-mon[126399]: pgmap v487: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 71 MiB/s wr, 4.70k op/s 2023-07-06T20:24:41.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:41 smithi156 ceph-mon[129280]: pgmap v488: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 62 MiB/s wr, 3.79k op/s 2023-07-06T20:24:41.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:41 smithi098 ceph-mon[128968]: pgmap v488: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 62 MiB/s wr, 3.79k op/s 2023-07-06T20:24:41.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:41 smithi069 ceph-mon[126399]: pgmap v488: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 62 MiB/s wr, 3.79k op/s 2023-07-06T20:24:43.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:43 smithi156 ceph-mon[129280]: pgmap v489: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 62 MiB/s wr, 4.06k op/s 2023-07-06T20:24:43.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:43 smithi098 ceph-mon[128968]: pgmap v489: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 62 MiB/s wr, 4.06k op/s 2023-07-06T20:24:43.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:43 smithi069 ceph-mon[126399]: pgmap v489: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 62 MiB/s wr, 4.06k op/s 2023-07-06T20:24:44.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 175.0 s, 2965.4 tps, lat 33.917 ms stddev 33.665 2023-07-06T20:24:45.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:45 smithi156 ceph-mon[129280]: pgmap v490: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 69 MiB/s wr, 4.11k op/s 2023-07-06T20:24:45.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:45 smithi098 ceph-mon[128968]: pgmap v490: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 69 MiB/s wr, 4.11k op/s 2023-07-06T20:24:45.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:45 smithi069 ceph-mon[126399]: pgmap v490: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 69 MiB/s wr, 4.11k op/s 2023-07-06T20:24:47.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:47 smithi156 ceph-mon[129280]: pgmap v491: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 59 MiB/s wr, 3.23k op/s 2023-07-06T20:24:47.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:47 smithi098 ceph-mon[128968]: pgmap v491: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 59 MiB/s wr, 3.23k op/s 2023-07-06T20:24:47.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:47 smithi069 ceph-mon[126399]: pgmap v491: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 59 MiB/s wr, 3.23k op/s 2023-07-06T20:24:48.771 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:48 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:24:48.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:48 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:24:48.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:48 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:24:49.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 180.0 s, 2496.8 tps, lat 39.014 ms stddev 48.786 2023-07-06T20:24:49.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:49 smithi069 ceph-mon[126399]: pgmap v492: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 MiB/s wr, 3.00k op/s 2023-07-06T20:24:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:24:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:24:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:49 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:24:49.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:49 smithi156 ceph-mon[129280]: pgmap v492: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 MiB/s wr, 3.00k op/s 2023-07-06T20:24:49.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:24:49.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:24:49.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:49 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:24:49.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:49 smithi098 ceph-mon[128968]: pgmap v492: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 MiB/s wr, 3.00k op/s 2023-07-06T20:24:49.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:24:49.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:24:49.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:49 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:24:50.863 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:50 smithi156 ceph-mon[129280]: pgmap v493: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 48 MiB/s wr, 1.87k op/s 2023-07-06T20:24:50.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:50 smithi098 ceph-mon[128968]: pgmap v493: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 48 MiB/s wr, 1.87k op/s 2023-07-06T20:24:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:50 smithi069 ceph-mon[126399]: pgmap v493: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 48 MiB/s wr, 1.87k op/s 2023-07-06T20:24:53.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:52 smithi156 ceph-mon[129280]: pgmap v494: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 MiB/s wr, 2.56k op/s 2023-07-06T20:24:53.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:52 smithi098 ceph-mon[128968]: pgmap v494: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 MiB/s wr, 2.56k op/s 2023-07-06T20:24:53.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:52 smithi069 ceph-mon[126399]: pgmap v494: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 MiB/s wr, 2.56k op/s 2023-07-06T20:24:53.847 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:24:54.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.189+0000 7f2445d9d700 1 -- 172.21.15.69:0/1735361752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2440102f00 msgr2=0x7f24401052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:54.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.189+0000 7f2445d9d700 1 --2- 172.21.15.69:0/1735361752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2440102f00 0x7f24401052e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f242c009990 tx=0x7f242c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.189+0000 7f2445d9d700 1 -- 172.21.15.69:0/1735361752 shutdown_connections 2023-07-06T20:24:54.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.189+0000 7f2445d9d700 1 --2- 172.21.15.69:0/1735361752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2440105820 0x7f2440107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.189+0000 7f2445d9d700 1 --2- 172.21.15.69:0/1735361752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24401005e0 0x7f24401029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.189+0000 7f2445d9d700 1 --2- 172.21.15.69:0/1735361752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2440102f00 0x7f24401052e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.189+0000 7f2445d9d700 1 -- 172.21.15.69:0/1735361752 >> 172.21.15.69:0/1735361752 conn(0x7f24400fa000 msgr2=0x7f24400fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:24:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.190+0000 7f2445d9d700 1 -- 172.21.15.69:0/1735361752 shutdown_connections 2023-07-06T20:24:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.190+0000 7f2445d9d700 1 -- 172.21.15.69:0/1735361752 wait complete. 2023-07-06T20:24:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f2445d9d700 1 Processor -- start 2023-07-06T20:24:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f2445d9d700 1 -- start start 2023-07-06T20:24:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f2445d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24401005e0 0x7f24401978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f2445d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2440102f00 0x7f2440197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f2445d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2440105820 0x7f244019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f2445d9d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019f420 con 0x7f24401005e0 2023-07-06T20:24:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f2445d9d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019f560 con 0x7f2440102f00 2023-07-06T20:24:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f2445d9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019f6f0 con 0x7f2440105820 2023-07-06T20:24:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f243ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2440102f00 0x7f2440197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f2444d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24401005e0 0x7f24401978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f243ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2440102f00 0x7f2440197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:36284/0 (socket says 172.21.15.69:36284) 2023-07-06T20:24:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f243ffff700 1 -- 172.21.15.69:0/702610482 learned_addr learned my addr 172.21.15.69:0/702610482 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:24:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.191+0000 7f244559c700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2440105820 0x7f244019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.192+0000 7f243ffff700 1 -- 172.21.15.69:0/702610482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2440105820 msgr2=0x7f244019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.192+0000 7f243ffff700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2440105820 0x7f244019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.192+0000 7f243ffff700 1 -- 172.21.15.69:0/702610482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24401005e0 msgr2=0x7f24401978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.192+0000 7f243ffff700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24401005e0 0x7f24401978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.192+0000 7f243ffff700 1 -- 172.21.15.69:0/702610482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2434009520 con 0x7f2440102f00 2023-07-06T20:24:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.192+0000 7f243ffff700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2440102f00 0x7f2440197de0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f242c000c00 tx=0x7f242c0057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:24:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.192+0000 7f243dffb700 1 -- 172.21.15.69:0/702610482 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c01d070 con 0x7f2440102f00 2023-07-06T20:24:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.192+0000 7f2445d9d700 1 -- 172.21.15.69:0/702610482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242c0095f0 con 0x7f2440102f00 2023-07-06T20:24:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.192+0000 7f243dffb700 1 -- 172.21.15.69:0/702610482 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f242c00fc40 con 0x7f2440102f00 2023-07-06T20:24:54.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.192+0000 7f243dffb700 1 -- 172.21.15.69:0/702610482 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0173f0 con 0x7f2440102f00 2023-07-06T20:24:54.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.193+0000 7f2445d9d700 1 -- 172.21.15.69:0/702610482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244019fce0 con 0x7f2440102f00 2023-07-06T20:24:54.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.197+0000 7f243dffb700 1 -- 172.21.15.69:0/702610482 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f242c021410 con 0x7f2440102f00 2023-07-06T20:24:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.200+0000 7f243dffb700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f24300a12f0 0x7f24300a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.200+0000 7f243dffb700 1 -- 172.21.15.69:0/702610482 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f242c0cdb40 con 0x7f2440102f00 2023-07-06T20:24:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.201+0000 7f2444d9b700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f24300a12f0 0x7f24300a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.202+0000 7f2444d9b700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f24300a12f0 0x7f24300a37a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2434009df0 tx=0x7f2434009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:24:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.205+0000 7f245a9f9b80 1 Processor -- start 2023-07-06T20:24:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.205+0000 7f245a9f9b80 1 -- start start 2023-07-06T20:24:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.205+0000 7f245a9f9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfa70e0 0x56291cfa74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.205+0000 7f245a9f9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa7a30 0x56291cfa8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.205+0000 7f245a9f9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291cfa8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.205+0000 7f245a9f9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291ceabcb0 con 0x56291cfa7a30 2023-07-06T20:24:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.205+0000 7f245a9f9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291cefea40 con 0x56291cfa8620 2023-07-06T20:24:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.205+0000 7f245a9f9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291cef7ed0 con 0x56291cfa70e0 2023-07-06T20:24:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.206+0000 7f244559c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291cfa8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.206+0000 7f244559c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291cfa8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:36294/0 (socket says 172.21.15.69:36294) 2023-07-06T20:24:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.206+0000 7f244559c700 1 -- 172.21.15.69:0/303725883 learned_addr learned my addr 172.21.15.69:0/303725883 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:24:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.206+0000 7f2444d9b700 1 --2- 172.21.15.69:0/303725883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfa70e0 0x56291cfa74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.206+0000 7f243ffff700 1 --2- 172.21.15.69:0/303725883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa7a30 0x56291cfa8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.207+0000 7f2444d9b700 1 -- 172.21.15.69:0/303725883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 msgr2=0x56291cfa8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.207+0000 7f2444d9b700 1 --2- 172.21.15.69:0/303725883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291cfa8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.207+0000 7f2444d9b700 1 -- 172.21.15.69:0/303725883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa7a30 msgr2=0x56291cfa8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:54.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.207+0000 7f2444d9b700 1 --2- 172.21.15.69:0/303725883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa7a30 0x56291cfa8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.207+0000 7f2444d9b700 1 -- 172.21.15.69:0/303725883 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56291cd25de0 con 0x56291cfa70e0 2023-07-06T20:24:54.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.209+0000 7f2444d9b700 1 --2- 172.21.15.69:0/303725883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfa70e0 0x56291cfa74f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2434011ab0 tx=0x7f2434014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7453d3000535bd91 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:24:54.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.210+0000 7f242affd700 1 -- 172.21.15.69:0/303725883 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243402e430 con 0x56291cfa70e0 2023-07-06T20:24:54.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.210+0000 7f242affd700 1 -- 172.21.15.69:0/303725883 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2434029070 con 0x56291cfa70e0 2023-07-06T20:24:54.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.210+0000 7f242affd700 1 -- 172.21.15.69:0/303725883 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243402d780 con 0x56291cfa70e0 2023-07-06T20:24:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.211+0000 7f245a9f9b80 1 -- 172.21.15.69:0/303725883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfa70e0 msgr2=0x56291cfa74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.211+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/303725883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfa70e0 0x56291cfa74f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2434011ab0 tx=0x7f2434014000 comp rx=0 tx=0).stop 2023-07-06T20:24:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.212+0000 7f245a9f9b80 1 -- 172.21.15.69:0/303725883 shutdown_connections 2023-07-06T20:24:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.212+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/303725883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291cfa8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.212+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/303725883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfa70e0 0x56291cfa74f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.212+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/303725883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa7a30 0x56291cfa8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.212+0000 7f245a9f9b80 1 -- 172.21.15.69:0/303725883 >> 172.21.15.69:0/303725883 conn(0x56291cf9c7a0 msgr2=0x56291cfa1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:24:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.213+0000 7f245a9f9b80 1 -- 172.21.15.69:0/303725883 shutdown_connections 2023-07-06T20:24:54.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.213+0000 7f245a9f9b80 1 -- 172.21.15.69:0/303725883 wait complete. 2023-07-06T20:24:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.214+0000 7f245a9f9b80 1 -- start start 2023-07-06T20:24:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.214+0000 7f245a9f9b80 1 Processor -- start 2023-07-06T20:24:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.215+0000 7f245a9f9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 0x56291cfb0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.215+0000 7f245a9f9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291cfb1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.215+0000 7f245a9f9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 0x56291cfb8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.215+0000 7f245a9f9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291ce97ac0 con 0x56291cfa70e0 2023-07-06T20:24:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.215+0000 7f245a9f9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291ce977c0 con 0x56291cfa8620 2023-07-06T20:24:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.215+0000 7f245a9f9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291cd7b920 con 0x56291cfb60a0 2023-07-06T20:24:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.215+0000 7f245a9f9b80 10 client.0 _ll_register_callbacks cb 0x56291ce95760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:24:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.216+0000 7f243ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291cfb1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.216+0000 7f243ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291cfb1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:36306/0 (socket says 172.21.15.69:36306) 2023-07-06T20:24:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.216+0000 7f243ffff700 1 -- 172.21.15.69:0/2993482061 learned_addr learned my addr 172.21.15.69:0/2993482061 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:24:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.216+0000 7f244559c700 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 0x56291cfb8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.216+0000 7f2444d9b700 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 0x56291cfb0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.216+0000 7f244559c700 1 -- 172.21.15.69:0/2993482061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 msgr2=0x56291cfb1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.216+0000 7f244559c700 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291cfb1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.216+0000 7f244559c700 1 -- 172.21.15.69:0/2993482061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 msgr2=0x56291cfb0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.216+0000 7f244559c700 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 0x56291cfb0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.216+0000 7f244559c700 1 -- 172.21.15.69:0/2993482061 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2434017070 con 0x56291cfb60a0 2023-07-06T20:24:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.217+0000 7f244559c700 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 0x56291cfb8480 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2438007bc0 tx=0x7f2438009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64cbfdcf85da3c1e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:24:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.217+0000 7f243ffff700 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291cfb1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:24:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.217+0000 7f241ffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:24:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.217+0000 7f241ffff700 1 -- 172.21.15.69:0/2993482061 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800e070 con 0x56291cfb60a0 2023-07-06T20:24:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.217+0000 7f241ffff700 1 -- 172.21.15.69:0/2993482061 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f2438003a00 con 0x56291cfb60a0 2023-07-06T20:24:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.217+0000 7f245a9f9b80 1 -- 172.21.15.69:0/2993482061 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56291ce514f0 con 0x56291cfb60a0 2023-07-06T20:24:54.226 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.219+0000 7f241ffff700 1 -- 172.21.15.69:0/2993482061 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2438003bc0 con 0x56291cfb60a0 2023-07-06T20:24:54.226 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.223+0000 7f245a9f9b80 10 client.25540 fetch_fsmap learned FSMap version 30 2023-07-06T20:24:54.226 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.223+0000 7f245a9f9b80 1 -- 172.21.15.69:0/2993482061 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56291cd0ec60 con 0x56291cfb60a0 2023-07-06T20:24:54.241 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 185.0 s, 2050.9 tps, lat 49.890 ms stddev 67.610 2023-07-06T20:24:54.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.240+0000 7f241ffff700 1 -- 172.21.15.69:0/2993482061 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56291ce514f0 con 0x56291cfb60a0 2023-07-06T20:24:54.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.240+0000 7f241ffff700 1 -- 172.21.15.69:0/2993482061 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2438004230 con 0x56291cfb60a0 2023-07-06T20:24:54.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.242+0000 7f245a9f9b80 10 client.25540 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:24:54.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.242+0000 7f245a9f9b80 10 client.25540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:24:54.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.242+0000 7f245a9f9b80 20 client.25540 populate_metadata read hostname 'smithi069' 2023-07-06T20:24:54.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.242+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56291cfbcd10 0x56291cfbf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.242+0000 7f245a9f9b80 4 client.25540 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:24:54.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.242+0000 7f245a9f9b80 1 -- 172.21.15.69:0/2993482061 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56291cf75d60 con 0x56291cfbcd10 2023-07-06T20:24:54.253 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.245+0000 7f2444d9b700 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56291cfbcd10 0x56291cfbf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.253 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.247+0000 7f2444d9b700 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56291cfbcd10 0x56291cfbf120 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b48bde5fc3a0b5 server_cookie=8d67c19878115c2c in_seq=0 out_seq=0 2023-07-06T20:24:54.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.250+0000 7f241ffff700 10 client.25540 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:24:54.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.250+0000 7f241ffff700 1 -- 172.21.15.69:0/2993482061 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56291cf75d60 con 0x56291cfbcd10 2023-07-06T20:24:54.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.250+0000 7f241ffff700 10 client.25540 handle_command_reply: tid=0 2023-07-06T20:24:54.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.251+0000 7f245a9f9b80 1 client.25540 shutdown 2023-07-06T20:24:54.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.254+0000 7f245a9f9b80 1 -- 172.21.15.69:0/2993482061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 msgr2=0x56291cfb8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:54.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.254+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 0x56291cfb8480 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2438007bc0 tx=0x7f2438009510 comp rx=0 tx=0).stop 2023-07-06T20:24:54.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.254+0000 7f245a9f9b80 1 -- 172.21.15.69:0/2993482061 shutdown_connections 2023-07-06T20:24:54.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.254+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56291cfbcd10 0x56291cfbf120 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.254+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 0x56291cfb8480 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.254+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 0x56291cfb0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.254+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/2993482061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291cfb1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.254+0000 7f245a9f9b80 1 -- 172.21.15.69:0/2993482061 >> 172.21.15.69:0/2993482061 conn(0x56291cf9c7a0 msgr2=0x56291cfa1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:24:54.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.255+0000 7f241ffff700 0 client.25540 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:24:54.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.255+0000 7f245a9f9b80 1 -- 172.21.15.69:0/2993482061 shutdown_connections 2023-07-06T20:24:54.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.255+0000 7f245a9f9b80 1 -- 172.21.15.69:0/2993482061 wait complete. 2023-07-06T20:24:54.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.260+0000 7f245a9f9b80 20 client.25540 trim_cache size 0 max 16384 2023-07-06T20:24:54.291 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.289+0000 7f245a9f9b80 1 Processor -- start 2023-07-06T20:24:54.292 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.289+0000 7f245a9f9b80 1 -- start start 2023-07-06T20:24:54.292 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.289+0000 7f245a9f9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 0x56291cfcc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.292 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.289+0000 7f245a9f9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfa8620 0x56291cfb8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.292 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.289+0000 7f245a9f9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfb60a0 0x56291cfaea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.292 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.289+0000 7f245a9f9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291cfba180 con 0x56291cfa70e0 2023-07-06T20:24:54.292 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.289+0000 7f245a9f9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291cfaffe0 con 0x56291cfb60a0 2023-07-06T20:24:54.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.289+0000 7f245a9f9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291cfb06a0 con 0x56291cfa8620 2023-07-06T20:24:54.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.290+0000 7f244559c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfb60a0 0x56291cfaea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.290+0000 7f244559c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfb60a0 0x56291cfaea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:36322/0 (socket says 172.21.15.69:36322) 2023-07-06T20:24:54.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.290+0000 7f244559c700 1 -- 172.21.15.69:0/1451154495 learned_addr learned my addr 172.21.15.69:0/1451154495 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:24:54.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.290+0000 7f243ffff700 1 --2- 172.21.15.69:0/1451154495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfa8620 0x56291cfb8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.290+0000 7f244559c700 1 -- 172.21.15.69:0/1451154495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfa8620 msgr2=0x56291cfb8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:54.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.290+0000 7f244559c700 1 --2- 172.21.15.69:0/1451154495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfa8620 0x56291cfb8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.290+0000 7f244559c700 1 -- 172.21.15.69:0/1451154495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 msgr2=0x56291cfcc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:54.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.290+0000 7f244559c700 1 --2- 172.21.15.69:0/1451154495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 0x56291cfcc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.290+0000 7f244559c700 1 -- 172.21.15.69:0/1451154495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56291cd4c250 con 0x56291cfb60a0 2023-07-06T20:24:54.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.291+0000 7f244559c700 1 --2- 172.21.15.69:0/1451154495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfb60a0 0x56291cfaea70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2438013310 tx=0x7f2438013340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b99a3b57930871c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:24:54.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.291+0000 7f242b7fe700 1 -- 172.21.15.69:0/1451154495 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800e070 con 0x56291cfb60a0 2023-07-06T20:24:54.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.291+0000 7f242b7fe700 1 -- 172.21.15.69:0/1451154495 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f243800f040 con 0x56291cfb60a0 2023-07-06T20:24:54.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.291+0000 7f242b7fe700 1 -- 172.21.15.69:0/1451154495 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438003a00 con 0x56291cfb60a0 2023-07-06T20:24:54.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.293+0000 7f245a9f9b80 1 -- 172.21.15.69:0/1451154495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfb60a0 msgr2=0x56291cfaea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:54.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.293+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/1451154495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfb60a0 0x56291cfaea70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2438013310 tx=0x7f2438013340 comp rx=0 tx=0).stop 2023-07-06T20:24:54.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.294+0000 7f245a9f9b80 1 -- 172.21.15.69:0/1451154495 shutdown_connections 2023-07-06T20:24:54.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.294+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/1451154495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfb60a0 0x56291cfaea70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.294+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/1451154495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 0x56291cfcc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.294+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/1451154495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfa8620 0x56291cfb8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.294+0000 7f245a9f9b80 1 -- 172.21.15.69:0/1451154495 >> 172.21.15.69:0/1451154495 conn(0x56291cf9c7a0 msgr2=0x56291cfaf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:24:54.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.294+0000 7f245a9f9b80 1 -- 172.21.15.69:0/1451154495 shutdown_connections 2023-07-06T20:24:54.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.294+0000 7f245a9f9b80 1 -- 172.21.15.69:0/1451154495 wait complete. 2023-07-06T20:24:54.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.295+0000 7f245a9f9b80 1 -- start start 2023-07-06T20:24:54.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.295+0000 7f245a9f9b80 1 Processor -- start 2023-07-06T20:24:54.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.296+0000 7f245a9f9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 0x56291cfcabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.296+0000 7f245a9f9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291d01e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.296+0000 7f245a9f9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 0x56291d0208f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.296+0000 7f245a9f9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291d021000 con 0x56291cfa70e0 2023-07-06T20:24:54.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.296+0000 7f245a9f9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291d021170 con 0x56291cfa8620 2023-07-06T20:24:54.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.296+0000 7f245a9f9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291d021330 con 0x56291cfb60a0 2023-07-06T20:24:54.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.296+0000 7f245a9f9b80 10 client.0 _ll_register_callbacks cb 0x7f2438007590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:24:54.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.296+0000 7f244559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 0x56291d0208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.296+0000 7f244559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 0x56291d0208f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:51204/0 (socket says 172.21.15.69:51204) 2023-07-06T20:24:54.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.296+0000 7f244559c700 1 -- 172.21.15.69:0/4136862134 learned_addr learned my addr 172.21.15.69:0/4136862134 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:24:54.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.297+0000 7f2444d9b700 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 0x56291cfcabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.297+0000 7f243ffff700 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291d01e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.297+0000 7f244559c700 1 -- 172.21.15.69:0/4136862134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 msgr2=0x56291d01e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:54.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.297+0000 7f244559c700 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291d01e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.297+0000 7f244559c700 1 -- 172.21.15.69:0/4136862134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 msgr2=0x56291cfcabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:24:54.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.297+0000 7f244559c700 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 0x56291cfcabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.297+0000 7f244559c700 1 -- 172.21.15.69:0/4136862134 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56291cd0ec60 con 0x56291cfb60a0 2023-07-06T20:24:54.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.297+0000 7f244559c700 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 0x56291d0208f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f243800af20 tx=0x7f2438009a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34a9e45528f438e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:24:54.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.297+0000 7f241ffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:24:54.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.298+0000 7f241ffff700 1 -- 172.21.15.69:0/4136862134 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800e040 con 0x56291cfb60a0 2023-07-06T20:24:54.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.298+0000 7f245a9f9b80 1 -- 172.21.15.69:0/4136862134 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2434029070 con 0x56291cfb60a0 2023-07-06T20:24:54.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.298+0000 7f241ffff700 1 -- 172.21.15.69:0/4136862134 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f2438004ca0 con 0x56291cfb60a0 2023-07-06T20:24:54.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.298+0000 7f241ffff700 1 -- 172.21.15.69:0/4136862134 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2438013ee0 con 0x56291cfb60a0 2023-07-06T20:24:54.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.298+0000 7f245a9f9b80 10 client.25546 fetch_fsmap learned FSMap version 30 2023-07-06T20:24:54.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.298+0000 7f245a9f9b80 1 -- 172.21.15.69:0/4136862134 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56291cd242b0 con 0x56291cfb60a0 2023-07-06T20:24:54.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.298+0000 7f241ffff700 1 -- 172.21.15.69:0/4136862134 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2434029070 con 0x56291cfb60a0 2023-07-06T20:24:54.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.299+0000 7f241ffff700 1 -- 172.21.15.69:0/4136862134 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2438033070 con 0x56291cfb60a0 2023-07-06T20:24:54.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.299+0000 7f245a9f9b80 10 client.25546 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:24:54.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.299+0000 7f245a9f9b80 10 client.25546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:24:54.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.299+0000 7f245a9f9b80 20 client.25546 populate_metadata read hostname 'smithi069' 2023-07-06T20:24:54.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.299+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56291d022950 0x56291d024d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:24:54.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.299+0000 7f245a9f9b80 4 client.25546 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:24:54.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.299+0000 7f245a9f9b80 1 -- 172.21.15.69:0/4136862134 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56291cef0eb0 con 0x56291d022950 2023-07-06T20:24:54.304 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.299+0000 7f2444d9b700 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56291d022950 0x56291d024d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:24:54.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.300+0000 7f241ffff700 10 client.25546 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:24:54.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.300+0000 7f2444d9b700 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56291d022950 0x56291d024d60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71283038464f26b server_cookie=2464a7ba86ed940a in_seq=0 out_seq=0 2023-07-06T20:24:54.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.300+0000 7f241ffff700 1 -- 172.21.15.69:0/4136862134 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56291cef0eb0 con 0x56291d022950 2023-07-06T20:24:54.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.300+0000 7f241ffff700 10 client.25546 handle_command_reply: tid=0 2023-07-06T20:24:54.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.300+0000 7f245a9f9b80 1 client.25546 shutdown 2023-07-06T20:24:54.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.301+0000 7f245a9f9b80 1 -- 172.21.15.69:0/4136862134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 msgr2=0x56291d0208f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:54.305 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.301+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 0x56291d0208f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f243800af20 tx=0x7f2438009a40 comp rx=0 tx=0).stop 2023-07-06T20:24:54.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.301+0000 7f245a9f9b80 1 -- 172.21.15.69:0/4136862134 shutdown_connections 2023-07-06T20:24:54.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.301+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56291d022950 0x56291d024d60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.301+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56291cfb60a0 0x56291d0208f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.301+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56291cfa70e0 0x56291cfcabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.301+0000 7f241ffff700 0 client.25546 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:24:54.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.301+0000 7f245a9f9b80 1 --2- 172.21.15.69:0/4136862134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56291cfa8620 0x56291d01e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.301+0000 7f245a9f9b80 1 -- 172.21.15.69:0/4136862134 >> 172.21.15.69:0/4136862134 conn(0x56291cf9c7a0 msgr2=0x56291cfb8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:24:54.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.301+0000 7f245a9f9b80 1 -- 172.21.15.69:0/4136862134 shutdown_connections 2023-07-06T20:24:54.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.301+0000 7f245a9f9b80 1 -- 172.21.15.69:0/4136862134 wait complete. 2023-07-06T20:24:54.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.301+0000 7f245a9f9b80 20 client.25546 trim_cache size 0 max 16384 2023-07-06T20:24:54.307 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:24:54.307 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:24:54.307 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:24:54.307 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:24:54.308 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:24:54.308 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:24:54.308 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:24:54.308 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:24:54.308 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:24:54.308 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:24:54.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.302+0000 7f2445d9d700 1 -- 172.21.15.69:0/702610482 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f24300a12f0 msgr2=0x7f24300a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:54.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.302+0000 7f2445d9d700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f24300a12f0 0x7f24300a37a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2434009df0 tx=0x7f2434009260 comp rx=0 tx=0).stop 2023-07-06T20:24:54.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.302+0000 7f2445d9d700 1 -- 172.21.15.69:0/702610482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2440102f00 msgr2=0x7f2440197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:24:54.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.302+0000 7f2445d9d700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2440102f00 0x7f2440197de0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f242c000c00 tx=0x7f242c0057a0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.302+0000 7f2445d9d700 1 -- 172.21.15.69:0/702610482 shutdown_connections 2023-07-06T20:24:54.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.302+0000 7f2445d9d700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f24300a12f0 0x7f24300a37a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.302+0000 7f2445d9d700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2440105820 0x7f244019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.302+0000 7f2445d9d700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24401005e0 0x7f24401978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.302+0000 7f2445d9d700 1 --2- 172.21.15.69:0/702610482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2440102f00 0x7f2440197de0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:24:54.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.302+0000 7f2445d9d700 1 -- 172.21.15.69:0/702610482 >> 172.21.15.69:0/702610482 conn(0x7f24400fa000 msgr2=0x7f24400fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:24:54.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.302+0000 7f2445d9d700 1 -- 172.21.15.69:0/702610482 shutdown_connections 2023-07-06T20:24:54.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:24:54.302+0000 7f2445d9d700 1 -- 172.21.15.69:0/702610482 wait complete. 2023-07-06T20:24:54.331 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:24:55.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:54 smithi098 ceph-mon[128968]: pgmap v495: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 66 MiB/s wr, 3.82k op/s 2023-07-06T20:24:55.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:54 smithi069 ceph-mon[126399]: pgmap v495: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 66 MiB/s wr, 3.82k op/s 2023-07-06T20:24:55.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:54 smithi156 ceph-mon[129280]: pgmap v495: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 66 MiB/s wr, 3.82k op/s 2023-07-06T20:24:57.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:56 smithi069 ceph-mon[126399]: pgmap v496: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 58 MiB/s wr, 4.00k op/s 2023-07-06T20:24:57.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:56 smithi156 ceph-mon[129280]: pgmap v496: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 58 MiB/s wr, 4.00k op/s 2023-07-06T20:24:57.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:56 smithi098 ceph-mon[128968]: pgmap v496: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 58 MiB/s wr, 4.00k op/s 2023-07-06T20:24:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 190.0 s, 2710.3 tps, lat 37.015 ms stddev 45.231 2023-07-06T20:24:59.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:24:58 smithi069 ceph-mon[126399]: pgmap v497: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 MiB/s wr, 3.97k op/s 2023-07-06T20:24:59.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:24:58 smithi156 ceph-mon[129280]: pgmap v497: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 MiB/s wr, 3.97k op/s 2023-07-06T20:24:59.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:24:58 smithi098 ceph-mon[128968]: pgmap v497: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 MiB/s wr, 3.97k op/s 2023-07-06T20:25:01.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:00 smithi069 ceph-mon[126399]: pgmap v498: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 61 MiB/s wr, 3.97k op/s 2023-07-06T20:25:01.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:00 smithi156 ceph-mon[129280]: pgmap v498: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 61 MiB/s wr, 3.97k op/s 2023-07-06T20:25:01.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:00 smithi098 ceph-mon[128968]: pgmap v498: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 61 MiB/s wr, 3.97k op/s 2023-07-06T20:25:03.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:03 smithi156 ceph-mon[129280]: pgmap v499: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 61 MiB/s wr, 3.88k op/s 2023-07-06T20:25:03.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:03 smithi098 ceph-mon[128968]: pgmap v499: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 61 MiB/s wr, 3.88k op/s 2023-07-06T20:25:03.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:03 smithi069 ceph-mon[126399]: pgmap v499: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 61 MiB/s wr, 3.88k op/s 2023-07-06T20:25:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 195.0 s, 2078.3 tps, lat 47.626 ms stddev 105.845 2023-07-06T20:25:05.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:05 smithi156 ceph-mon[129280]: pgmap v500: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 60 MiB/s wr, 3.33k op/s 2023-07-06T20:25:05.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:05 smithi098 ceph-mon[128968]: pgmap v500: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 60 MiB/s wr, 3.33k op/s 2023-07-06T20:25:05.517 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:05 smithi069 ceph-mon[126399]: pgmap v500: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 60 MiB/s wr, 3.33k op/s 2023-07-06T20:25:07.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:07 smithi156 ceph-mon[129280]: pgmap v501: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 52 MiB/s wr, 2.54k op/s 2023-07-06T20:25:07.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:07 smithi098 ceph-mon[128968]: pgmap v501: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 52 MiB/s wr, 2.54k op/s 2023-07-06T20:25:07.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:07 smithi069 ceph-mon[126399]: pgmap v501: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 52 MiB/s wr, 2.54k op/s 2023-07-06T20:25:09.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 200.0 s, 2648.5 tps, lat 37.787 ms stddev 41.509 2023-07-06T20:25:09.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:09 smithi156 ceph-mon[129280]: pgmap v502: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 55 MiB/s wr, 3.00k op/s 2023-07-06T20:25:09.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:09 smithi098 ceph-mon[128968]: pgmap v502: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 55 MiB/s wr, 3.00k op/s 2023-07-06T20:25:09.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:09 smithi069 ceph-mon[126399]: pgmap v502: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 55 MiB/s wr, 3.00k op/s 2023-07-06T20:25:11.378 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:11 smithi098 ceph-mon[128968]: pgmap v503: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 66 MiB/s wr, 4.17k op/s 2023-07-06T20:25:11.514 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:11 smithi069 ceph-mon[126399]: pgmap v503: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 66 MiB/s wr, 4.17k op/s 2023-07-06T20:25:11.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:11 smithi156 ceph-mon[129280]: pgmap v503: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 66 MiB/s wr, 4.17k op/s 2023-07-06T20:25:13.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:13 smithi069 ceph-mon[126399]: pgmap v504: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 63 MiB/s wr, 4.07k op/s 2023-07-06T20:25:13.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:13 smithi156 ceph-mon[129280]: pgmap v504: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 63 MiB/s wr, 4.07k op/s 2023-07-06T20:25:13.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:13 smithi098 ceph-mon[128968]: pgmap v504: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 63 MiB/s wr, 4.07k op/s 2023-07-06T20:25:14.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 205.0 s, 2597.2 tps, lat 38.626 ms stddev 42.779 2023-07-06T20:25:15.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:15 smithi069 ceph-mon[126399]: pgmap v505: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 65 MiB/s wr, 4.07k op/s 2023-07-06T20:25:15.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:15 smithi156 ceph-mon[129280]: pgmap v505: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 65 MiB/s wr, 4.07k op/s 2023-07-06T20:25:15.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:15 smithi098 ceph-mon[128968]: pgmap v505: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 65 MiB/s wr, 4.07k op/s 2023-07-06T20:25:17.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:17 smithi069 ceph-mon[126399]: pgmap v506: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 57 MiB/s wr, 3.55k op/s 2023-07-06T20:25:17.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:17 smithi156 ceph-mon[129280]: pgmap v506: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 57 MiB/s wr, 3.55k op/s 2023-07-06T20:25:17.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:17 smithi098 ceph-mon[128968]: pgmap v506: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 57 MiB/s wr, 3.55k op/s 2023-07-06T20:25:19.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 210.0 s, 1645.0 tps, lat 59.757 ms stddev 88.960 2023-07-06T20:25:19.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:19 smithi069 ceph-mon[126399]: pgmap v507: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 51 MiB/s wr, 2.72k op/s 2023-07-06T20:25:19.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:19 smithi156 ceph-mon[129280]: pgmap v507: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 51 MiB/s wr, 2.72k op/s 2023-07-06T20:25:19.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:19 smithi098 ceph-mon[128968]: pgmap v507: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 51 MiB/s wr, 2.72k op/s 2023-07-06T20:25:21.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:21 smithi069 ceph-mon[126399]: pgmap v508: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 59 MiB/s wr, 3.23k op/s 2023-07-06T20:25:21.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:21 smithi156 ceph-mon[129280]: pgmap v508: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 59 MiB/s wr, 3.23k op/s 2023-07-06T20:25:21.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:21 smithi098 ceph-mon[128968]: pgmap v508: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 59 MiB/s wr, 3.23k op/s 2023-07-06T20:25:23.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:23 smithi069 ceph-mon[126399]: pgmap v509: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 54 MiB/s wr, 2.95k op/s 2023-07-06T20:25:23.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:23 smithi156 ceph-mon[129280]: pgmap v509: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 54 MiB/s wr, 2.95k op/s 2023-07-06T20:25:23.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:23 smithi098 ceph-mon[128968]: pgmap v509: 97 pgs: 97 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 54 MiB/s wr, 2.95k op/s 2023-07-06T20:25:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 215.0 s, 2980.2 tps, lat 34.237 ms stddev 32.714 2023-07-06T20:25:24.332 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:25:24.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.692+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/3816061823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d1410be90 msgr2=0x7f3d1410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:24.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.692+0000 7f3d1b3ff700 1 --2- 172.21.15.69:0/3816061823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d1410be90 0x7f3d1410e320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009f90 tx=0x7f3d0c00b330 comp rx=0 tx=0).stop 2023-07-06T20:25:24.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.692+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/3816061823 shutdown_connections 2023-07-06T20:25:24.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.692+0000 7f3d1b3ff700 1 --2- 172.21.15.69:0/3816061823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d1410be90 0x7f3d1410e320 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.692+0000 7f3d1b3ff700 1 --2- 172.21.15.69:0/3816061823 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d14071010 0x7f3d14071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.692+0000 7f3d1b3ff700 1 --2- 172.21.15.69:0/3816061823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d14071960 0x7f3d1410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.692+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/3816061823 >> 172.21.15.69:0/3816061823 conn(0x7f3d1406cbc0 msgr2=0x7f3d1406eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:25:24.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.692+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/3816061823 shutdown_connections 2023-07-06T20:25:24.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.693+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/3816061823 wait complete. 2023-07-06T20:25:24.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.694+0000 7f3d1b3ff700 1 Processor -- start 2023-07-06T20:25:24.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.695+0000 7f3d1b3ff700 1 -- start start 2023-07-06T20:25:24.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.695+0000 7f3d1b3ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d14071010 0x7f3d14113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.695+0000 7f3d1b3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d14071960 0x7f3d14114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.696+0000 7f3d1899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d14071960 0x7f3d14114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.696+0000 7f3d1919b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d14071010 0x7f3d14113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.696+0000 7f3d1899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d14071960 0x7f3d14114090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:47694/0 (socket says 172.21.15.69:47694) 2023-07-06T20:25:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.696+0000 7f3d1919b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d14071010 0x7f3d14113b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:56166/0 (socket says 172.21.15.69:56166) 2023-07-06T20:25:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.696+0000 7f3d1b3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d1410be90 0x7f3d141b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.696+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/417461599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d14117170 con 0x7f3d1410be90 2023-07-06T20:25:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.696+0000 7f3d1899a700 1 -- 172.21.15.69:0/417461599 learned_addr learned my addr 172.21.15.69:0/417461599 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:25:24.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.696+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/417461599 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d141172e0 con 0x7f3d14071010 2023-07-06T20:25:24.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.696+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/417461599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d141b4ec0 con 0x7f3d14071960 2023-07-06T20:25:24.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.697+0000 7f3d1919b700 1 -- 172.21.15.69:0/417461599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d14071960 msgr2=0x7f3d14114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:24.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.697+0000 7f3d1919b700 1 --2- 172.21.15.69:0/417461599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d14071960 0x7f3d14114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.697+0000 7f3d1919b700 1 -- 172.21.15.69:0/417461599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d1410be90 msgr2=0x7f3d141b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:24.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.697+0000 7f3d1919b700 1 --2- 172.21.15.69:0/417461599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d1410be90 0x7f3d141b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.697+0000 7f3d1919b700 1 -- 172.21.15.69:0/417461599 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c009c40 con 0x7f3d14071010 2023-07-06T20:25:24.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.697+0000 7f3d1919b700 1 --2- 172.21.15.69:0/417461599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d14071010 0x7f3d14113b50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400d950 tx=0x7f3d0400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:25:24.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.697+0000 7f3d0a7fc700 1 -- 172.21.15.69:0/417461599 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d040098d0 con 0x7f3d14071010 2023-07-06T20:25:24.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.698+0000 7f3d0a7fc700 1 -- 172.21.15.69:0/417461599 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d04010460 con 0x7f3d14071010 2023-07-06T20:25:24.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.698+0000 7f3d0a7fc700 1 -- 172.21.15.69:0/417461599 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0400b5d0 con 0x7f3d14071010 2023-07-06T20:25:24.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.698+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/417461599 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d141b50c0 con 0x7f3d14071010 2023-07-06T20:25:24.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.698+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/417461599 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d141b5550 con 0x7f3d14071010 2023-07-06T20:25:24.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.705+0000 7f3d2eda6b80 1 Processor -- start 2023-07-06T20:25:24.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.706+0000 7f3d2eda6b80 1 -- start start 2023-07-06T20:25:24.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.707+0000 7f3d2eda6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb44f20 0x55a99eb45330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.707+0000 7f3d2eda6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb45870 0x55a99eb45e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.707+0000 7f3d2eda6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb46830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.707+0000 7f3d2eda6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99ea49cb0 con 0x55a99eb45870 2023-07-06T20:25:24.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.707+0000 7f3d2eda6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99ea9ca40 con 0x55a99eb44f20 2023-07-06T20:25:24.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.707+0000 7f3d2eda6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99ea95ed0 con 0x55a99eb46390 2023-07-06T20:25:24.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.707+0000 7f3d1899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb46830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.707+0000 7f3d1999c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb44f20 0x55a99eb45330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.707+0000 7f3d1899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb46830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:47698/0 (socket says 172.21.15.69:47698) 2023-07-06T20:25:24.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.707+0000 7f3d1899a700 1 -- 172.21.15.69:0/3868791384 learned_addr learned my addr 172.21.15.69:0/3868791384 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:25:24.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.707+0000 7f3d1999c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb44f20 0x55a99eb45330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:56176/0 (socket says 172.21.15.69:56176) 2023-07-06T20:25:24.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.707+0000 7f3d0a7fc700 1 -- 172.21.15.69:0/417461599 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3d04010820 con 0x7f3d14071010 2023-07-06T20:25:24.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.708+0000 7f3d1999c700 1 -- 172.21.15.69:0/3868791384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 msgr2=0x55a99eb46830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:24.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.708+0000 7f3d1999c700 1 --2- 172.21.15.69:0/3868791384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb46830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.708+0000 7f3d1999c700 1 -- 172.21.15.69:0/3868791384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb45870 msgr2=0x55a99eb45e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:24.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.708+0000 7f3d1999c700 1 --2- 172.21.15.69:0/3868791384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb45870 0x55a99eb45e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.708+0000 7f3d1999c700 1 -- 172.21.15.69:0/3868791384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a99e8c3de0 con 0x55a99eb44f20 2023-07-06T20:25:24.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.708+0000 7f3d1899a700 1 --2- 172.21.15.69:0/3868791384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb46830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:25:24.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.708+0000 7f3d1999c700 1 --2- 172.21.15.69:0/3868791384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb44f20 0x55a99eb45330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009f90 tx=0x7f3d0c007420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d85e4a44cb463560 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:25:24.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.708+0000 7f3cff7fe700 1 -- 172.21.15.69:0/3868791384 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00e070 con 0x55a99eb44f20 2023-07-06T20:25:24.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.708+0000 7f3cff7fe700 1 -- 172.21.15.69:0/3868791384 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d0c007790 con 0x55a99eb44f20 2023-07-06T20:25:24.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.709+0000 7f3cff7fe700 1 -- 172.21.15.69:0/3868791384 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c005490 con 0x55a99eb44f20 2023-07-06T20:25:24.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.709+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/3868791384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb44f20 msgr2=0x55a99eb45330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:24.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.709+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/3868791384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb44f20 0x55a99eb45330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009f90 tx=0x7f3d0c007420 comp rx=0 tx=0).stop 2023-07-06T20:25:24.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.710+0000 7f3d0a7fc700 1 --2- 172.21.15.69:0/417461599 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3d000a1220 0x7f3d000a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.710+0000 7f3d0a7fc700 1 -- 172.21.15.69:0/417461599 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f3d040cc3d0 con 0x7f3d14071010 2023-07-06T20:25:24.718 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.711+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/3868791384 shutdown_connections 2023-07-06T20:25:24.718 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.711+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/3868791384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb46830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.711+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/3868791384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb44f20 0x55a99eb45330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.711+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/3868791384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb45870 0x55a99eb45e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.711+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/3868791384 >> 172.21.15.69:0/3868791384 conn(0x55a99eb3a640 msgr2=0x55a99eb3f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:25:24.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.712+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/3868791384 shutdown_connections 2023-07-06T20:25:24.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.712+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/3868791384 wait complete. 2023-07-06T20:25:24.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.713+0000 7f3d2eda6b80 1 -- start start 2023-07-06T20:25:24.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.713+0000 7f3d2eda6b80 1 Processor -- start 2023-07-06T20:25:24.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.713+0000 7f3d1999c700 1 --2- 172.21.15.69:0/417461599 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3d000a1220 0x7f3d000a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.714+0000 7f3d2eda6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb44f20 0x55a99eb4e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.714+0000 7f3d2eda6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb45870 0x55a99eb4eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.714+0000 7f3d2eda6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb55e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.714+0000 7f3d2eda6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99ea35ac0 con 0x55a99eb45870 2023-07-06T20:25:24.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.714+0000 7f3d2eda6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99ea357c0 con 0x55a99eb44f20 2023-07-06T20:25:24.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.714+0000 7f3d2eda6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99e919920 con 0x55a99eb46390 2023-07-06T20:25:24.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.714+0000 7f3d2eda6b80 10 client.0 _ll_register_callbacks cb 0x55a99ea33760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:25:24.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.715+0000 7f3d1999c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb55e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.715+0000 7f3d1999c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb55e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:47712/0 (socket says 172.21.15.69:47712) 2023-07-06T20:25:24.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.715+0000 7f3d1999c700 1 -- 172.21.15.69:0/2507930068 learned_addr learned my addr 172.21.15.69:0/2507930068 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:25:24.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.715+0000 7f3d1999c700 1 --2- 172.21.15.69:0/417461599 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3d000a1220 0x7f3d000a36d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00be40 tx=0x7f3d0c005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:25:24.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.715+0000 7f3d1919b700 1 --2- 172.21.15.69:0/2507930068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb44f20 0x55a99eb4e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.715+0000 7f3d1999c700 1 -- 172.21.15.69:0/2507930068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb44f20 msgr2=0x55a99eb4e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:24.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.715+0000 7f3d1999c700 1 --2- 172.21.15.69:0/2507930068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb44f20 0x55a99eb4e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.715+0000 7f3d1999c700 1 -- 172.21.15.69:0/2507930068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb45870 msgr2=0x55a99eb4eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:24.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.715+0000 7f3d1999c700 1 --2- 172.21.15.69:0/2507930068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb45870 0x55a99eb4eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.715+0000 7f3d1999c700 1 -- 172.21.15.69:0/2507930068 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d10009a70 con 0x55a99eb46390 2023-07-06T20:25:24.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.716+0000 7f3d1999c700 1 --2- 172.21.15.69:0/2507930068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb55e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c01c000 tx=0x7f3d0c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a094449c4a204f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:25:24.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.716+0000 7f3cfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:25:24.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.717+0000 7f3cfcff9700 1 -- 172.21.15.69:0/2507930068 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00e070 con 0x55a99eb46390 2023-07-06T20:25:24.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.717+0000 7f3cfcff9700 1 -- 172.21.15.69:0/2507930068 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f3d0c01b450 con 0x55a99eb46390 2023-07-06T20:25:24.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.717+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/2507930068 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a99e9ef4f0 con 0x55a99eb46390 2023-07-06T20:25:24.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.718+0000 7f3cfcff9700 1 -- 172.21.15.69:0/2507930068 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d0c036070 con 0x55a99eb46390 2023-07-06T20:25:24.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.718+0000 7f3d2eda6b80 10 client.25558 fetch_fsmap learned FSMap version 30 2023-07-06T20:25:24.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.718+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/2507930068 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f3d14117770 con 0x55a99eb46390 2023-07-06T20:25:24.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.718+0000 7f3cfcff9700 1 -- 172.21.15.69:0/2507930068 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a99e9ef4f0 con 0x55a99eb46390 2023-07-06T20:25:24.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.718+0000 7f3cfcff9700 1 -- 172.21.15.69:0/2507930068 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3d0c027640 con 0x55a99eb46390 2023-07-06T20:25:24.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.718+0000 7f3d2eda6b80 10 client.25558 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:25:24.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.718+0000 7f3d2eda6b80 10 client.25558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:25:24.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.719+0000 7f3d2eda6b80 20 client.25558 populate_metadata read hostname 'smithi069' 2023-07-06T20:25:24.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.719+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/2507930068 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a99eb5a270 0x55a99eb5c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.719+0000 7f3d2eda6b80 4 client.25558 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:25:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.719+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/2507930068 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a99eb13d60 con 0x55a99eb5a270 2023-07-06T20:25:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.719+0000 7f3d1919b700 1 --2- 172.21.15.69:0/2507930068 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a99eb5a270 0x55a99eb5c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.720+0000 7f3d1919b700 1 --2- 172.21.15.69:0/2507930068 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a99eb5a270 0x55a99eb5c680 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=557b1f91b6c0347c server_cookie=c8c5ec875144dfb in_seq=0 out_seq=0 2023-07-06T20:25:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.720+0000 7f3cfcff9700 10 client.25558 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:25:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.720+0000 7f3cfcff9700 1 -- 172.21.15.69:0/2507930068 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a99eb13d60 con 0x55a99eb5a270 2023-07-06T20:25:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.720+0000 7f3cfcff9700 10 client.25558 handle_command_reply: tid=0 2023-07-06T20:25:24.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.721+0000 7f3d2eda6b80 1 client.25558 shutdown 2023-07-06T20:25:24.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.721+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/2507930068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 msgr2=0x55a99eb55e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:24.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.721+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/2507930068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb55e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c01c000 tx=0x7f3d0c009510 comp rx=0 tx=0).stop 2023-07-06T20:25:24.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.722+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/2507930068 shutdown_connections 2023-07-06T20:25:24.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.722+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/2507930068 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a99eb5a270 0x55a99eb5c680 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.722+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/2507930068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb55e00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.722+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/2507930068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb44f20 0x55a99eb4e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.722+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/2507930068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb45870 0x55a99eb4eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.722+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/2507930068 >> 172.21.15.69:0/2507930068 conn(0x55a99eb3a640 msgr2=0x55a99eb3f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:25:24.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.722+0000 7f3cfcff9700 0 client.25558 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:25:24.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.722+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/2507930068 shutdown_connections 2023-07-06T20:25:24.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.722+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/2507930068 wait complete. 2023-07-06T20:25:24.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.722+0000 7f3d2eda6b80 20 client.25558 trim_cache size 0 max 16384 2023-07-06T20:25:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.748+0000 7f3d2eda6b80 1 Processor -- start 2023-07-06T20:25:24.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.748+0000 7f3d2eda6b80 1 -- start start 2023-07-06T20:25:24.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.749+0000 7f3d2eda6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb44f20 0x55a99eb3f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.749+0000 7f3d2eda6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb45870 0x55a99eb55dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.751 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.749+0000 7f3d2eda6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb46390 0x55a99eb5c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.749+0000 7f3d2eda6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99e9ef4f0 con 0x55a99eb45870 2023-07-06T20:25:24.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.749+0000 7f3d2eda6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99eb413b0 con 0x55a99eb46390 2023-07-06T20:25:24.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.749+0000 7f3d2eda6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99eb3a450 con 0x55a99eb44f20 2023-07-06T20:25:24.752 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.749+0000 7f3d1999c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb46390 0x55a99eb5c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.749+0000 7f3d1999c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb46390 0x55a99eb5c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:56186/0 (socket says 172.21.15.69:56186) 2023-07-06T20:25:24.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.749+0000 7f3d1999c700 1 -- 172.21.15.69:0/1636582361 learned_addr learned my addr 172.21.15.69:0/1636582361 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:25:24.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.749+0000 7f3d1919b700 1 --2- 172.21.15.69:0/1636582361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb44f20 0x55a99eb3f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.753 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.750+0000 7f3d1999c700 1 -- 172.21.15.69:0/1636582361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb44f20 msgr2=0x55a99eb3f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:24.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.750+0000 7f3d1999c700 1 --2- 172.21.15.69:0/1636582361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb44f20 0x55a99eb3f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.750+0000 7f3d1999c700 1 -- 172.21.15.69:0/1636582361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb45870 msgr2=0x55a99eb55dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:24.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.750+0000 7f3d1999c700 1 --2- 172.21.15.69:0/1636582361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb45870 0x55a99eb55dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.754 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.750+0000 7f3d1999c700 1 -- 172.21.15.69:0/1636582361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a99e8acc60 con 0x55a99eb46390 2023-07-06T20:25:24.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.750+0000 7f3d1999c700 1 --2- 172.21.15.69:0/1636582361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb46390 0x55a99eb5c280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0279a0 tx=0x7f3d0c00be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70f2a2314f23f2e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:25:24.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.750+0000 7f3cfffff700 1 -- 172.21.15.69:0/1636582361 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c036070 con 0x55a99eb46390 2023-07-06T20:25:24.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.751+0000 7f3cfffff700 1 -- 172.21.15.69:0/1636582361 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d0c0057a0 con 0x55a99eb46390 2023-07-06T20:25:24.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.751+0000 7f3cfffff700 1 -- 172.21.15.69:0/1636582361 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00e040 con 0x55a99eb46390 2023-07-06T20:25:24.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.752+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/1636582361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb46390 msgr2=0x55a99eb5c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:24.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.752+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/1636582361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb46390 0x55a99eb5c280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0279a0 tx=0x7f3d0c00be70 comp rx=0 tx=0).stop 2023-07-06T20:25:24.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.752+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/1636582361 shutdown_connections 2023-07-06T20:25:24.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.752+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/1636582361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb46390 0x55a99eb5c280 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.752+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/1636582361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb44f20 0x55a99eb3f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.752+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/1636582361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb45870 0x55a99eb55dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.752+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/1636582361 >> 172.21.15.69:0/1636582361 conn(0x55a99eb3a640 msgr2=0x55a99eb4d540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:25:24.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.752+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/1636582361 shutdown_connections 2023-07-06T20:25:24.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.752+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/1636582361 wait complete. 2023-07-06T20:25:24.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.753+0000 7f3d2eda6b80 1 -- start start 2023-07-06T20:25:24.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.753+0000 7f3d2eda6b80 1 Processor -- start 2023-07-06T20:25:24.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.754+0000 7f3d2eda6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb44f20 0x55a99eb555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.754+0000 7f3d2eda6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb45870 0x55a99eb55b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.762 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.754+0000 7f3d2eda6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb53bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.754+0000 7f3d2eda6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99eb54150 con 0x55a99eb44f20 2023-07-06T20:25:24.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.754+0000 7f3d2eda6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99eb542c0 con 0x55a99eb45870 2023-07-06T20:25:24.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.754+0000 7f3d2eda6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99eb54480 con 0x55a99eb46390 2023-07-06T20:25:24.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.754+0000 7f3d2eda6b80 10 client.0 _ll_register_callbacks cb 0x7f3d0c00bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:25:24.763 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.754+0000 7f3d1919b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb44f20 0x55a99eb555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.754+0000 7f3d1899a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb45870 0x55a99eb55b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.754+0000 7f3d1919b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb44f20 0x55a99eb555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54232/0 (socket says 172.21.15.69:54232) 2023-07-06T20:25:24.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.754+0000 7f3d1919b700 1 -- 172.21.15.69:0/152153544 learned_addr learned my addr 172.21.15.69:0/152153544 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:25:24.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.754+0000 7f3d1999c700 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb53bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.755+0000 7f3d1899a700 1 -- 172.21.15.69:0/152153544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 msgr2=0x55a99eb53bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:24.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.755+0000 7f3d1899a700 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb53bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.755+0000 7f3d1899a700 1 -- 172.21.15.69:0/152153544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb44f20 msgr2=0x55a99eb555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:24.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.755+0000 7f3d1899a700 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb44f20 0x55a99eb555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.755+0000 7f3d1899a700 1 -- 172.21.15.69:0/152153544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d14117770 con 0x55a99eb45870 2023-07-06T20:25:24.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.755+0000 7f3d1999c700 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb53bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:25:24.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.755+0000 7f3d1899a700 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb45870 0x55a99eb55b00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55a99eb44c70 tx=0x7f3d10012c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d595474523d7f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:25:24.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.755+0000 7f3cfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:25:24.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.755+0000 7f3cfcff9700 1 -- 172.21.15.69:0/152153544 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10017070 con 0x55a99eb45870 2023-07-06T20:25:24.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.755+0000 7f3cfcff9700 1 -- 172.21.15.69:0/152153544 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f3d10005050 con 0x55a99eb45870 2023-07-06T20:25:24.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.755+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/152153544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d0c007790 con 0x55a99eb45870 2023-07-06T20:25:24.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.756+0000 7f3cfcff9700 1 -- 172.21.15.69:0/152153544 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d10005210 con 0x55a99eb45870 2023-07-06T20:25:24.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.756+0000 7f3d2eda6b80 10 client.25616 fetch_fsmap learned FSMap version 30 2023-07-06T20:25:24.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.756+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/152153544 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a99e8ea250 con 0x55a99eb45870 2023-07-06T20:25:24.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.756+0000 7f3cfcff9700 1 -- 172.21.15.69:0/152153544 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d0c007790 con 0x55a99eb45870 2023-07-06T20:25:24.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.756+0000 7f3cfcff9700 1 -- 172.21.15.69:0/152153544 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3d1000f760 con 0x55a99eb45870 2023-07-06T20:25:24.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.756+0000 7f3d2eda6b80 10 client.25616 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:25:24.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.756+0000 7f3d2eda6b80 10 client.25616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:25:24.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.756+0000 7f3d2eda6b80 20 client.25616 populate_metadata read hostname 'smithi069' 2023-07-06T20:25:24.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.756+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a99ebaf440 0x55a99ebb1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:24.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.756+0000 7f3d2eda6b80 4 client.25616 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:25:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.756+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/152153544 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a99ea8eeb0 con 0x55a99ebaf440 2023-07-06T20:25:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.757+0000 7f3d1919b700 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a99ebaf440 0x55a99ebb1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.757+0000 7f3cfcff9700 10 client.25616 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:25:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.757+0000 7f3d1919b700 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a99ebaf440 0x55a99ebb1850 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b13701f460447b server_cookie=ddd5ad127757f2e2 in_seq=0 out_seq=0 2023-07-06T20:25:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3cfcff9700 1 -- 172.21.15.69:0/152153544 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a99ea8eeb0 con 0x55a99ebaf440 2023-07-06T20:25:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3cfcff9700 10 client.25616 handle_command_reply: tid=0 2023-07-06T20:25:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3d2eda6b80 1 client.25616 shutdown 2023-07-06T20:25:24.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/152153544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb45870 msgr2=0x55a99eb55b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:24.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb45870 0x55a99eb55b00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55a99eb44c70 tx=0x7f3d10012c50 comp rx=0 tx=0).stop 2023-07-06T20:25:24.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/152153544 shutdown_connections 2023-07-06T20:25:24.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55a99ebaf440 0x55a99ebb1850 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a99eb46390 0x55a99eb53bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a99eb44f20 0x55a99eb555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3d2eda6b80 1 --2- 172.21.15.69:0/152153544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a99eb45870 0x55a99eb55b00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/152153544 >> 172.21.15.69:0/152153544 conn(0x55a99eb3a640 msgr2=0x55a99eb3f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:25:24.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3cfcff9700 0 client.25616 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:25:24.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/152153544 shutdown_connections 2023-07-06T20:25:24.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.758+0000 7f3d2eda6b80 1 -- 172.21.15.69:0/152153544 wait complete. 2023-07-06T20:25:24.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.760+0000 7f3d2eda6b80 20 client.25616 trim_cache size 0 max 16384 2023-07-06T20:25:24.771 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:25:24.772 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:25:24.772 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:25:24.772 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:25:24.772 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:25:24.772 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:25:24.772 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:25:24.772 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:25:24.773 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:25:24.773 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:25:24.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.761+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/417461599 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3d000a1220 msgr2=0x7f3d000a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:24.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.761+0000 7f3d1b3ff700 1 --2- 172.21.15.69:0/417461599 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3d000a1220 0x7f3d000a36d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00be40 tx=0x7f3d0c005040 comp rx=0 tx=0).stop 2023-07-06T20:25:24.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.761+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/417461599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d14071010 msgr2=0x7f3d14113b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:24.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.761+0000 7f3d1b3ff700 1 --2- 172.21.15.69:0/417461599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d14071010 0x7f3d14113b50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400d950 tx=0x7f3d0400dc60 comp rx=0 tx=0).stop 2023-07-06T20:25:24.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.762+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/417461599 shutdown_connections 2023-07-06T20:25:24.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.762+0000 7f3d1b3ff700 1 --2- 172.21.15.69:0/417461599 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f3d000a1220 0x7f3d000a36d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.762+0000 7f3d1b3ff700 1 --2- 172.21.15.69:0/417461599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d1410be90 0x7f3d141b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.762+0000 7f3d1b3ff700 1 --2- 172.21.15.69:0/417461599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d14071010 0x7f3d14113b50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.762+0000 7f3d1b3ff700 1 --2- 172.21.15.69:0/417461599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d14071960 0x7f3d14114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:24.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.762+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/417461599 >> 172.21.15.69:0/417461599 conn(0x7f3d1406cbc0 msgr2=0x7f3d1406dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:25:24.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.763+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/417461599 shutdown_connections 2023-07-06T20:25:24.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:24.763+0000 7f3d1b3ff700 1 -- 172.21.15.69:0/417461599 wait complete. 2023-07-06T20:25:24.791 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:25:25.258 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:25 smithi069 ceph-mon[126399]: pgmap v510: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 60 MiB/s wr, 3.91k op/s 2023-07-06T20:25:25.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:25 smithi156 ceph-mon[129280]: pgmap v510: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 60 MiB/s wr, 3.91k op/s 2023-07-06T20:25:25.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:25 smithi098 ceph-mon[128968]: pgmap v510: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 60 MiB/s wr, 3.91k op/s 2023-07-06T20:25:27.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:27 smithi156 ceph-mon[129280]: pgmap v511: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 58 MiB/s wr, 3.90k op/s 2023-07-06T20:25:27.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:27 smithi098 ceph-mon[128968]: pgmap v511: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 58 MiB/s wr, 3.90k op/s 2023-07-06T20:25:27.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:27 smithi069 ceph-mon[126399]: pgmap v511: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 58 MiB/s wr, 3.90k op/s 2023-07-06T20:25:29.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 220.0 s, 2507.2 tps, lat 39.858 ms stddev 36.791 2023-07-06T20:25:29.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:29 smithi156 ceph-mon[129280]: pgmap v512: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 56 MiB/s wr, 3.85k op/s 2023-07-06T20:25:29.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:29 smithi098 ceph-mon[128968]: pgmap v512: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 56 MiB/s wr, 3.85k op/s 2023-07-06T20:25:29.764 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:29 smithi069 ceph-mon[126399]: pgmap v512: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 56 MiB/s wr, 3.85k op/s 2023-07-06T20:25:31.629 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:31 smithi098 ceph-mon[128968]: pgmap v513: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 62 MiB/s wr, 3.92k op/s 2023-07-06T20:25:31.766 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:31 smithi069 ceph-mon[126399]: pgmap v513: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 62 MiB/s wr, 3.92k op/s 2023-07-06T20:25:31.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:31 smithi156 ceph-mon[129280]: pgmap v513: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 62 MiB/s wr, 3.92k op/s 2023-07-06T20:25:33.769 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:33 smithi069 ceph-mon[126399]: pgmap v514: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 MiB/s wr, 2.82k op/s 2023-07-06T20:25:33.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:33 smithi156 ceph-mon[129280]: pgmap v514: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 MiB/s wr, 2.82k op/s 2023-07-06T20:25:33.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:33 smithi098 ceph-mon[128968]: pgmap v514: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 MiB/s wr, 2.82k op/s 2023-07-06T20:25:34.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 225.0 s, 1528.7 tps, lat 59.768 ms stddev 89.035 2023-07-06T20:25:35.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:35 smithi069 ceph-mon[126399]: pgmap v515: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 56 MiB/s wr, 3.07k op/s 2023-07-06T20:25:35.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:35 smithi156 ceph-mon[129280]: pgmap v515: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 56 MiB/s wr, 3.07k op/s 2023-07-06T20:25:35.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:35 smithi098 ceph-mon[128968]: pgmap v515: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 56 MiB/s wr, 3.07k op/s 2023-07-06T20:25:37.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:37 smithi069 ceph-mon[126399]: pgmap v516: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 55 MiB/s wr, 3.08k op/s 2023-07-06T20:25:37.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:37 smithi156 ceph-mon[129280]: pgmap v516: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 55 MiB/s wr, 3.08k op/s 2023-07-06T20:25:37.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:37 smithi098 ceph-mon[128968]: pgmap v516: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 55 MiB/s wr, 3.08k op/s 2023-07-06T20:25:39.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 230.0 s, 2896.8 tps, lat 37.478 ms stddev 64.062 2023-07-06T20:25:39.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:39 smithi069 ceph-mon[126399]: pgmap v517: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 56 MiB/s wr, 3.49k op/s 2023-07-06T20:25:39.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:39 smithi156 ceph-mon[129280]: pgmap v517: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 56 MiB/s wr, 3.49k op/s 2023-07-06T20:25:39.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:39 smithi098 ceph-mon[128968]: pgmap v517: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 56 MiB/s wr, 3.49k op/s 2023-07-06T20:25:41.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:41 smithi069 ceph-mon[126399]: pgmap v518: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 67 MiB/s wr, 4.16k op/s 2023-07-06T20:25:41.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:41 smithi156 ceph-mon[129280]: pgmap v518: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 67 MiB/s wr, 4.16k op/s 2023-07-06T20:25:41.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:41 smithi098 ceph-mon[128968]: pgmap v518: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 67 MiB/s wr, 4.16k op/s 2023-07-06T20:25:43.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:43 smithi069 ceph-mon[126399]: pgmap v519: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 MiB/s wr, 3.97k op/s 2023-07-06T20:25:43.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:43 smithi156 ceph-mon[129280]: pgmap v519: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 MiB/s wr, 3.97k op/s 2023-07-06T20:25:43.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:43 smithi098 ceph-mon[128968]: pgmap v519: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 MiB/s wr, 3.97k op/s 2023-07-06T20:25:44.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 235.0 s, 2569.7 tps, lat 38.643 ms stddev 37.013 2023-07-06T20:25:45.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:45 smithi069 ceph-mon[126399]: pgmap v520: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 MiB/s wr, 3.94k op/s 2023-07-06T20:25:45.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:45 smithi156 ceph-mon[129280]: pgmap v520: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 MiB/s wr, 3.94k op/s 2023-07-06T20:25:45.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:45 smithi098 ceph-mon[128968]: pgmap v520: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 MiB/s wr, 3.94k op/s 2023-07-06T20:25:47.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:47 smithi069 ceph-mon[126399]: pgmap v521: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 52 MiB/s wr, 2.87k op/s 2023-07-06T20:25:47.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:47 smithi156 ceph-mon[129280]: pgmap v521: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 52 MiB/s wr, 2.87k op/s 2023-07-06T20:25:47.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:47 smithi098 ceph-mon[128968]: pgmap v521: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 52 MiB/s wr, 2.87k op/s 2023-07-06T20:25:49.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 240.0 s, 1519.3 tps, lat 60.011 ms stddev 94.779 2023-07-06T20:25:49.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:49 smithi156 ceph-mon[129280]: pgmap v522: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 2.06k op/s 2023-07-06T20:25:49.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:49 smithi098 ceph-mon[128968]: pgmap v522: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 2.06k op/s 2023-07-06T20:25:50.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:49 smithi069 ceph-mon[126399]: pgmap v522: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 MiB/s wr, 2.06k op/s 2023-07-06T20:25:50.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:50 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:25:50.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:50 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:25:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:50 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:25:51.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:51 smithi156 ceph-mon[129280]: pgmap v523: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 57 MiB/s wr, 3.12k op/s 2023-07-06T20:25:51.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:25:51.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:25:51.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:51 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:25:51.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:51 smithi098 ceph-mon[128968]: pgmap v523: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 57 MiB/s wr, 3.12k op/s 2023-07-06T20:25:51.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:25:51.881 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:25:51.881 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:51 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:25:52.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:51 smithi069 ceph-mon[126399]: pgmap v523: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 57 MiB/s wr, 3.12k op/s 2023-07-06T20:25:52.104 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:25:52.104 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:25:52.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:51 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:25:52.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:52 smithi156 ceph-mon[129280]: pgmap v524: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 52 MiB/s wr, 3.20k op/s 2023-07-06T20:25:52.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:52 smithi098 ceph-mon[128968]: pgmap v524: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 52 MiB/s wr, 3.20k op/s 2023-07-06T20:25:53.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:52 smithi069 ceph-mon[126399]: pgmap v524: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 52 MiB/s wr, 3.20k op/s 2023-07-06T20:25:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 245.0 s, 2760.6 tps, lat 39.773 ms stddev 49.862 2023-07-06T20:25:54.792 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:25:55.144 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.141+0000 7f38603ff700 1 -- 172.21.15.69:0/2895186613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3858103940 msgr2=0x7f385810a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:55.144 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.141+0000 7f38603ff700 1 --2- 172.21.15.69:0/2895186613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3858103940 0x7f385810a270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f385000b0a0 tx=0x7f385000b3b0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.144 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.141+0000 7f38603ff700 1 -- 172.21.15.69:0/2895186613 shutdown_connections 2023-07-06T20:25:55.145 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.141+0000 7f38603ff700 1 --2- 172.21.15.69:0/2895186613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3858103940 0x7f385810a270 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.145 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.141+0000 7f38603ff700 1 --2- 172.21.15.69:0/2895186613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3858101e40 0x7f3858102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.145 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.141+0000 7f38603ff700 1 --2- 172.21.15.69:0/2895186613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3858102f90 0x7f3858103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.145 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.141+0000 7f38603ff700 1 -- 172.21.15.69:0/2895186613 >> 172.21.15.69:0/2895186613 conn(0x7f38580fd480 msgr2=0x7f38580ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:25:55.145 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.141+0000 7f38603ff700 1 -- 172.21.15.69:0/2895186613 shutdown_connections 2023-07-06T20:25:55.146 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.142+0000 7f38603ff700 1 -- 172.21.15.69:0/2895186613 wait complete. 2023-07-06T20:25:55.146 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.143+0000 7f38603ff700 1 Processor -- start 2023-07-06T20:25:55.148 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.145+0000 7f38603ff700 1 -- start start 2023-07-06T20:25:55.148 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.145+0000 7f38603ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3858101e40 0x7f38581938b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.145+0000 7f38603ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3858102f90 0x7f3858193df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.145+0000 7f38603ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3858103940 0x7f385819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.145+0000 7f38603ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385819b430 con 0x7f3858103940 2023-07-06T20:25:55.150 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.145+0000 7f38603ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385819b570 con 0x7f3858102f90 2023-07-06T20:25:55.150 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.145+0000 7f38603ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385806a0b0 con 0x7f3858101e40 2023-07-06T20:25:55.151 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.146+0000 7f385d99a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3858102f90 0x7f3858193df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.151 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.146+0000 7f385d99a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3858102f90 0x7f3858193df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:58610/0 (socket says 172.21.15.69:58610) 2023-07-06T20:25:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.146+0000 7f385d99a700 1 -- 172.21.15.69:0/1304503572 learned_addr learned my addr 172.21.15.69:0/1304503572 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:25:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.146+0000 7f385e19b700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3858101e40 0x7f38581938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.147+0000 7f385e99c700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3858103940 0x7f385819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.147+0000 7f385d99a700 1 -- 172.21.15.69:0/1304503572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3858101e40 msgr2=0x7f38581938b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.147+0000 7f385d99a700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3858101e40 0x7f38581938b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.148+0000 7f385d99a700 1 -- 172.21.15.69:0/1304503572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3858103940 msgr2=0x7f385819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:55.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.148+0000 7f385d99a700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3858103940 0x7f385819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.148+0000 7f385d99a700 1 -- 172.21.15.69:0/1304503572 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3850009d00 con 0x7f3858102f90 2023-07-06T20:25:55.153 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.148+0000 7f385e19b700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3858101e40 0x7f38581938b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:25:55.154 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.148+0000 7f385d99a700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3858102f90 0x7f3858193df0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f384800b550 tx=0x7f384800b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:25:55.154 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.149+0000 7f384f7fe700 1 -- 172.21.15.69:0/1304503572 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848011830 con 0x7f3858102f90 2023-07-06T20:25:55.154 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.149+0000 7f384f7fe700 1 -- 172.21.15.69:0/1304503572 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3848011e50 con 0x7f3858102f90 2023-07-06T20:25:55.154 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.149+0000 7f384f7fe700 1 -- 172.21.15.69:0/1304503572 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384800f2e0 con 0x7f3858102f90 2023-07-06T20:25:55.155 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.149+0000 7f38603ff700 1 -- 172.21.15.69:0/1304503572 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385806a390 con 0x7f3858102f90 2023-07-06T20:25:55.155 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.150+0000 7f38603ff700 1 -- 172.21.15.69:0/1304503572 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385806a8d0 con 0x7f3858102f90 2023-07-06T20:25:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.153+0000 7f384f7fe700 1 -- 172.21.15.69:0/1304503572 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3848011990 con 0x7f3858102f90 2023-07-06T20:25:55.157 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.155+0000 7f3873c3fb80 1 Processor -- start 2023-07-06T20:25:55.158 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.155+0000 7f3873c3fb80 1 -- start start 2023-07-06T20:25:55.159 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.156+0000 7f3873c3fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8cf20 0x5585f0f8d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.159 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.156+0000 7f3873c3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f8d870 0x5585f0f8de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.159 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.156+0000 7f3873c3fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f8e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.159 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.156+0000 7f3873c3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0e91cb0 con 0x5585f0f8d870 2023-07-06T20:25:55.159 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.156+0000 7f3873c3fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0ee4a40 con 0x5585f0f8cf20 2023-07-06T20:25:55.159 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.156+0000 7f3873c3fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0edded0 con 0x5585f0f8e390 2023-07-06T20:25:55.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.157+0000 7f385d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f8e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.157+0000 7f385d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f8e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:46554/0 (socket says 172.21.15.69:46554) 2023-07-06T20:25:55.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.157+0000 7f385d99a700 1 -- 172.21.15.69:0/1705133613 learned_addr learned my addr 172.21.15.69:0/1705133613 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:25:55.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.158+0000 7f385d99a700 1 -- 172.21.15.69:0/1705133613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8cf20 msgr2=0x5585f0f8d330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:25:55.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.158+0000 7f385d99a700 1 --2- 172.21.15.69:0/1705133613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8cf20 0x5585f0f8d330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.158+0000 7f385d99a700 1 -- 172.21.15.69:0/1705133613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f8d870 msgr2=0x5585f0f8de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:55.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.158+0000 7f385d99a700 1 --2- 172.21.15.69:0/1705133613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f8d870 0x5585f0f8de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.158+0000 7f385d99a700 1 -- 172.21.15.69:0/1705133613 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585f0d0bde0 con 0x5585f0f8e390 2023-07-06T20:25:55.163 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.160+0000 7f384f7fe700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f38440a11d0 0x7f38440a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.163 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.161+0000 7f384f7fe700 1 -- 172.21.15.69:0/1304503572 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f38480cc380 con 0x7f3858102f90 2023-07-06T20:25:55.164 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.161+0000 7f385d99a700 1 --2- 172.21.15.69:0/1705133613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f8e830 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f38480d4310 tx=0x7f38480d4620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ee28a153a260095 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:25:55.164 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.162+0000 7f384cbfe700 1 -- 172.21.15.69:0/1705133613 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848095a40 con 0x5585f0f8e390 2023-07-06T20:25:55.164 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.162+0000 7f384cbfe700 1 -- 172.21.15.69:0/1705133613 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38480994d0 con 0x5585f0f8e390 2023-07-06T20:25:55.164 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.162+0000 7f384cbfe700 1 -- 172.21.15.69:0/1705133613 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848099700 con 0x5585f0f8e390 2023-07-06T20:25:55.165 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.163+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1705133613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 msgr2=0x5585f0f8e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:55.165 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.163+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/1705133613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f8e830 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f38480d4310 tx=0x7f38480d4620 comp rx=0 tx=0).stop 2023-07-06T20:25:55.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.165+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1705133613 shutdown_connections 2023-07-06T20:25:55.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.165+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/1705133613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f8e830 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.165+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/1705133613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8cf20 0x5585f0f8d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.165+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/1705133613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f8d870 0x5585f0f8de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.165+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1705133613 >> 172.21.15.69:0/1705133613 conn(0x5585f0f82640 msgr2=0x5585f0f879f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:25:55.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.166+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1705133613 shutdown_connections 2023-07-06T20:25:55.170 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.166+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1705133613 wait complete. 2023-07-06T20:25:55.170 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.167+0000 7f3873c3fb80 1 -- start start 2023-07-06T20:25:55.170 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.167+0000 7f3873c3fb80 1 Processor -- start 2023-07-06T20:25:55.170 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.167+0000 7f385e99c700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f38440a11d0 0x7f38440a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.169+0000 7f3873c3fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8cf20 0x5585f0f96960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.169+0000 7f3873c3fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8d870 0x5585f0f96ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.169+0000 7f3873c3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f8e390 0x5585f0f9dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.169+0000 7f3873c3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0e7dac0 con 0x5585f0f8e390 2023-07-06T20:25:55.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.169+0000 7f3873c3fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0e7d7c0 con 0x5585f0f8d870 2023-07-06T20:25:55.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.169+0000 7f3873c3fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0d61920 con 0x5585f0f8cf20 2023-07-06T20:25:55.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.169+0000 7f3873c3fb80 10 client.0 _ll_register_callbacks cb 0x5585f0e7b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:25:55.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.170+0000 7f385e99c700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f38440a11d0 0x7f38440a3680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3850009fd0 tx=0x7f385000b7d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:25:55.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.174+0000 7f385e19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8cf20 0x5585f0f96960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.174+0000 7f385d99a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8d870 0x5585f0f96ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.175+0000 7f385d99a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8d870 0x5585f0f96ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:58634/0 (socket says 172.21.15.69:58634) 2023-07-06T20:25:55.192 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.175+0000 7f385d99a700 1 -- 172.21.15.69:0/3703953069 learned_addr learned my addr 172.21.15.69:0/3703953069 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:25:55.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.176+0000 7f385d99a700 1 -- 172.21.15.69:0/3703953069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8cf20 msgr2=0x5585f0f96960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:55.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.176+0000 7f385d99a700 1 --2- 172.21.15.69:0/3703953069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8cf20 0x5585f0f96960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.176+0000 7f385d99a700 1 -- 172.21.15.69:0/3703953069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f8e390 msgr2=0x5585f0f9dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:55.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.176+0000 7f385d99a700 1 --2- 172.21.15.69:0/3703953069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f8e390 0x5585f0f9dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.176+0000 7f385d99a700 1 -- 172.21.15.69:0/3703953069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385806a8d0 con 0x5585f0f8d870 2023-07-06T20:25:55.193 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.176+0000 7f385d99a700 1 --2- 172.21.15.69:0/3703953069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8d870 0x5585f0f96ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3848000c00 tx=0x7f3848004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35c3963320663e30 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.185+0000 7f3841ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.185+0000 7f3841ffb700 1 -- 172.21.15.69:0/3703953069 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848095a40 con 0x5585f0f8d870 2023-07-06T20:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.185+0000 7f3841ffb700 1 -- 172.21.15.69:0/3703953069 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f384809d730 con 0x5585f0f8d870 2023-07-06T20:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.185+0000 7f3873c3fb80 1 -- 172.21.15.69:0/3703953069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585f0e374f0 con 0x5585f0f8d870 2023-07-06T20:25:55.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.191+0000 7f3841ffb700 1 -- 172.21.15.69:0/3703953069 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38480a1b50 con 0x5585f0f8d870 2023-07-06T20:25:55.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.194+0000 7f3873c3fb80 10 client.25625 fetch_fsmap learned FSMap version 30 2023-07-06T20:25:55.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.194+0000 7f3873c3fb80 1 -- 172.21.15.69:0/3703953069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5585f0cf4c60 con 0x5585f0f8d870 2023-07-06T20:25:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.195+0000 7f3841ffb700 1 -- 172.21.15.69:0/3703953069 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5585f0e374f0 con 0x5585f0f8d870 2023-07-06T20:25:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.195+0000 7f3841ffb700 1 -- 172.21.15.69:0/3703953069 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f38480cb030 con 0x5585f0f8d870 2023-07-06T20:25:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.196+0000 7f3873c3fb80 10 client.25625 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:25:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.196+0000 7f3873c3fb80 10 client.25625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:25:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.196+0000 7f3873c3fb80 20 client.25625 populate_metadata read hostname 'smithi069' 2023-07-06T20:25:55.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.196+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/3703953069 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5585f0fa2300 0x5585f0fa4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.196+0000 7f3873c3fb80 4 client.25625 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:25:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.196+0000 7f3873c3fb80 1 -- 172.21.15.69:0/3703953069 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585f0f5bd60 con 0x5585f0fa2300 2023-07-06T20:25:55.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.196+0000 7f385e19b700 1 --2- 172.21.15.69:0/3703953069 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5585f0fa2300 0x5585f0fa4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.198+0000 7f385e19b700 1 --2- 172.21.15.69:0/3703953069 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5585f0fa2300 0x5585f0fa4710 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4111e194f79c1b server_cookie=dd697ca4fb4cfe2 in_seq=0 out_seq=0 2023-07-06T20:25:55.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.198+0000 7f3841ffb700 10 client.25625 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:25:55.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.199+0000 7f3841ffb700 1 -- 172.21.15.69:0/3703953069 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5585f0f5bd60 con 0x5585f0fa2300 2023-07-06T20:25:55.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.199+0000 7f3841ffb700 10 client.25625 handle_command_reply: tid=0 2023-07-06T20:25:55.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.205+0000 7f3873c3fb80 1 client.25625 shutdown 2023-07-06T20:25:55.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.207+0000 7f3873c3fb80 1 -- 172.21.15.69:0/3703953069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8d870 msgr2=0x5585f0f96ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:55.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.207+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/3703953069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8d870 0x5585f0f96ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3848000c00 tx=0x7f3848004770 comp rx=0 tx=0).stop 2023-07-06T20:25:55.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.207+0000 7f3873c3fb80 1 -- 172.21.15.69:0/3703953069 shutdown_connections 2023-07-06T20:25:55.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.207+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/3703953069 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5585f0fa2300 0x5585f0fa4710 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.207+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/3703953069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f8e390 0x5585f0f9dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.207+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/3703953069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8cf20 0x5585f0f96960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.207+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/3703953069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8d870 0x5585f0f96ea0 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3848000c00 tx=0x7f3848004770 comp rx=0 tx=0).stop 2023-07-06T20:25:55.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.207+0000 7f3873c3fb80 1 -- 172.21.15.69:0/3703953069 >> 172.21.15.69:0/3703953069 conn(0x5585f0f82640 msgr2=0x5585f0f870b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:25:55.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.207+0000 7f3841ffb700 0 client.25625 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:25:55.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.208+0000 7f3873c3fb80 1 -- 172.21.15.69:0/3703953069 shutdown_connections 2023-07-06T20:25:55.215 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.213+0000 7f3873c3fb80 1 -- 172.21.15.69:0/3703953069 wait complete. 2023-07-06T20:25:55.216 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.214+0000 7f3873c3fb80 20 client.25625 trim_cache size 0 max 16384 2023-07-06T20:25:55.247 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.245+0000 7f3873c3fb80 1 Processor -- start 2023-07-06T20:25:55.247 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.245+0000 7f3873c3fb80 1 -- start start 2023-07-06T20:25:55.247 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.245+0000 7f3873c3fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8cf20 0x5585f0f870b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.245+0000 7f3873c3fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f9dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.245+0000 7f3873c3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f94800 0x5585f0fa4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.245+0000 7f3873c3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0f5bd90 con 0x5585f0f94800 2023-07-06T20:25:55.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.245+0000 7f3873c3fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0e37520 con 0x5585f0f8cf20 2023-07-06T20:25:55.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.245+0000 7f3873c3fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0f82210 con 0x5585f0f8e390 2023-07-06T20:25:55.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.245+0000 7f385d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f9dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.245+0000 7f385e99c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f94800 0x5585f0fa4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.245+0000 7f385d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f9dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:46570/0 (socket says 172.21.15.69:46570) 2023-07-06T20:25:55.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.246+0000 7f385d99a700 1 -- 172.21.15.69:0/900643020 learned_addr learned my addr 172.21.15.69:0/900643020 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:25:55.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.246+0000 7f385e19b700 1 --2- 172.21.15.69:0/900643020 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8cf20 0x5585f0f870b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.246+0000 7f385d99a700 1 -- 172.21.15.69:0/900643020 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8cf20 msgr2=0x5585f0f870b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:55.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.246+0000 7f385d99a700 1 --2- 172.21.15.69:0/900643020 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8cf20 0x5585f0f870b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.246+0000 7f385d99a700 1 -- 172.21.15.69:0/900643020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f94800 msgr2=0x5585f0fa4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:55.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.246+0000 7f385d99a700 1 --2- 172.21.15.69:0/900643020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f94800 0x5585f0fa4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.246+0000 7f385d99a700 1 -- 172.21.15.69:0/900643020 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585f0d32250 con 0x5585f0f8e390 2023-07-06T20:25:55.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.246+0000 7f385d99a700 1 --2- 172.21.15.69:0/900643020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f9dea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3848099400 tx=0x7f38480d4620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb34a8ac8f8fef0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:25:55.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.246+0000 7f384d3ff700 1 -- 172.21.15.69:0/900643020 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848095a40 con 0x5585f0f8e390 2023-07-06T20:25:55.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.246+0000 7f384d3ff700 1 -- 172.21.15.69:0/900643020 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3848099240 con 0x5585f0f8e390 2023-07-06T20:25:55.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.246+0000 7f384d3ff700 1 -- 172.21.15.69:0/900643020 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384809d730 con 0x5585f0f8e390 2023-07-06T20:25:55.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.247+0000 7f3873c3fb80 1 -- 172.21.15.69:0/900643020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 msgr2=0x5585f0f9dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:55.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.247+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/900643020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f9dea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3848099400 tx=0x7f38480d4620 comp rx=0 tx=0).stop 2023-07-06T20:25:55.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.248+0000 7f3873c3fb80 1 -- 172.21.15.69:0/900643020 shutdown_connections 2023-07-06T20:25:55.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.248+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/900643020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f94800 0x5585f0fa4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.248+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/900643020 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f8cf20 0x5585f0f870b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.248+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/900643020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f9dea0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.248+0000 7f3873c3fb80 1 -- 172.21.15.69:0/900643020 >> 172.21.15.69:0/900643020 conn(0x5585f0f82640 msgr2=0x5585f0f955d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:25:55.253 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.248+0000 7f3873c3fb80 1 -- 172.21.15.69:0/900643020 shutdown_connections 2023-07-06T20:25:55.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.248+0000 7f3873c3fb80 1 -- 172.21.15.69:0/900643020 wait complete. 2023-07-06T20:25:55.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.248+0000 7f3873c3fb80 1 -- start start 2023-07-06T20:25:55.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.249+0000 7f3873c3fb80 1 Processor -- start 2023-07-06T20:25:55.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.249+0000 7f3873c3fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f8cf20 0x5585f0f9d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.249+0000 7f3873c3fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f9dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.249+0000 7f3873c3fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f94800 0x5585f0f9be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.249+0000 7f3873c3fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0f9c470 con 0x5585f0f8cf20 2023-07-06T20:25:55.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.249+0000 7f3873c3fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0f9c5b0 con 0x5585f0f94800 2023-07-06T20:25:55.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.249+0000 7f3873c3fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585f0f9c770 con 0x5585f0f8e390 2023-07-06T20:25:55.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.249+0000 7f3873c3fb80 10 client.0 _ll_register_callbacks cb 0x7f38480d4190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:25:55.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.250+0000 7f385d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f9dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.250+0000 7f385d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f9dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:46586/0 (socket says 172.21.15.69:46586) 2023-07-06T20:25:55.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.250+0000 7f385d99a700 1 -- 172.21.15.69:0/1618675536 learned_addr learned my addr 172.21.15.69:0/1618675536 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:25:55.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.250+0000 7f385e99c700 1 --2- 172.21.15.69:0/1618675536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f94800 0x5585f0f9be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.251+0000 7f385d99a700 1 -- 172.21.15.69:0/1618675536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f94800 msgr2=0x5585f0f9be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:55.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.251+0000 7f385d99a700 1 --2- 172.21.15.69:0/1618675536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f94800 0x5585f0f9be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.251+0000 7f385d99a700 1 -- 172.21.15.69:0/1618675536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f8cf20 msgr2=0x5585f0f9d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:25:55.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.251+0000 7f385d99a700 1 --2- 172.21.15.69:0/1618675536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f8cf20 0x5585f0f9d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.251+0000 7f385d99a700 1 -- 172.21.15.69:0/1618675536 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585f0cf4c60 con 0x5585f0f8e390 2023-07-06T20:25:55.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.251+0000 7f385d99a700 1 --2- 172.21.15.69:0/1618675536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f9dd80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3848099640 tx=0x7f38480d4bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7f4ebf4818e22a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:25:55.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.251+0000 7f3841ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:25:55.259 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.254+0000 7f3841ffb700 1 -- 172.21.15.69:0/1618675536 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480a1b50 con 0x5585f0f8e390 2023-07-06T20:25:55.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.255+0000 7f3841ffb700 1 -- 172.21.15.69:0/1618675536 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f3848099700 con 0x5585f0f8e390 2023-07-06T20:25:55.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.255+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1618675536 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38480994d0 con 0x5585f0f8e390 2023-07-06T20:25:55.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.255+0000 7f3841ffb700 1 -- 172.21.15.69:0/1618675536 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f384809d730 con 0x5585f0f8e390 2023-07-06T20:25:55.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.255+0000 7f3873c3fb80 10 client.25579 fetch_fsmap learned FSMap version 30 2023-07-06T20:25:55.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.255+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1618675536 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5585f0d0a2b0 con 0x5585f0f8e390 2023-07-06T20:25:55.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.256+0000 7f3841ffb700 1 -- 172.21.15.69:0/1618675536 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3848081020 con 0x5585f0f8e390 2023-07-06T20:25:55.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.256+0000 7f3841ffb700 1 -- 172.21.15.69:0/1618675536 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f38480cb050 con 0x5585f0f8e390 2023-07-06T20:25:55.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.256+0000 7f3873c3fb80 10 client.25579 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:25:55.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.256+0000 7f3873c3fb80 10 client.25579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:25:55.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.256+0000 7f3873c3fb80 20 client.25579 populate_metadata read hostname 'smithi069' 2023-07-06T20:25:55.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.256+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/1618675536 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5585f0ff6ec0 0x5585f0ff92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:25:55.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.256+0000 7f3873c3fb80 4 client.25579 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:25:55.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.256+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1618675536 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5585f0ed6eb0 con 0x5585f0ff6ec0 2023-07-06T20:25:55.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.257+0000 7f385e19b700 1 --2- 172.21.15.69:0/1618675536 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5585f0ff6ec0 0x5585f0ff92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:25:55.263 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:54 smithi069 ceph-mon[126399]: pgmap v525: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 62 MiB/s wr, 3.93k op/s 2023-07-06T20:25:55.265 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.258+0000 7f385e19b700 1 --2- 172.21.15.69:0/1618675536 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5585f0ff6ec0 0x5585f0ff92a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=194a9bfbfd636670 server_cookie=eeb592af6a8ffc39 in_seq=0 out_seq=0 2023-07-06T20:25:55.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.258+0000 7f3841ffb700 10 client.25579 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:25:55.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.258+0000 7f3841ffb700 1 -- 172.21.15.69:0/1618675536 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5585f0ed6eb0 con 0x5585f0ff6ec0 2023-07-06T20:25:55.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.258+0000 7f3841ffb700 10 client.25579 handle_command_reply: tid=0 2023-07-06T20:25:55.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.258+0000 7f3873c3fb80 1 client.25579 shutdown 2023-07-06T20:25:55.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.258+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1618675536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 msgr2=0x5585f0f9dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:55.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.258+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/1618675536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f9dd80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3848099640 tx=0x7f38480d4bd0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.258+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1618675536 shutdown_connections 2023-07-06T20:25:55.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.259+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/1618675536 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5585f0ff6ec0 0x5585f0ff92a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.259+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/1618675536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5585f0f94800 0x5585f0f9be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.259+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/1618675536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585f0f8cf20 0x5585f0f9d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.269 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.259+0000 7f3873c3fb80 1 --2- 172.21.15.69:0/1618675536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5585f0f8e390 0x5585f0f9dd80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.269 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.259+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1618675536 >> 172.21.15.69:0/1618675536 conn(0x5585f0f82640 msgr2=0x5585f0fa4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:25:55.269 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.259+0000 7f3841ffb700 0 client.25579 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:25:55.270 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.259+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1618675536 shutdown_connections 2023-07-06T20:25:55.270 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:25:55.271 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:25:55.271 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:25:55.271 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:25:55.271 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:25:55.271 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:25:55.271 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:25:55.271 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:25:55.272 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:25:55.272 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:25:55.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.259+0000 7f3873c3fb80 1 -- 172.21.15.69:0/1618675536 wait complete. 2023-07-06T20:25:55.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.259+0000 7f3873c3fb80 20 client.25579 trim_cache size 0 max 16384 2023-07-06T20:25:55.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.262+0000 7f38603ff700 1 -- 172.21.15.69:0/1304503572 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f38440a11d0 msgr2=0x7f38440a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:55.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.262+0000 7f38603ff700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f38440a11d0 0x7f38440a3680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3850009fd0 tx=0x7f385000b7d0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.262+0000 7f38603ff700 1 -- 172.21.15.69:0/1304503572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3858102f90 msgr2=0x7f3858193df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:25:55.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.262+0000 7f38603ff700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3858102f90 0x7f3858193df0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f384800b550 tx=0x7f384800b860 comp rx=0 tx=0).stop 2023-07-06T20:25:55.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.262+0000 7f38603ff700 1 -- 172.21.15.69:0/1304503572 shutdown_connections 2023-07-06T20:25:55.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.262+0000 7f38603ff700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f38440a11d0 0x7f38440a3680 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.262+0000 7f38603ff700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3858103940 0x7f385819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.262+0000 7f38603ff700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3858101e40 0x7f38581938b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.262+0000 7f38603ff700 1 --2- 172.21.15.69:0/1304503572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3858102f90 0x7f3858193df0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:25:55.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.262+0000 7f38603ff700 1 -- 172.21.15.69:0/1304503572 >> 172.21.15.69:0/1304503572 conn(0x7f38580fd480 msgr2=0x7f3858076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:25:55.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.263+0000 7f38603ff700 1 -- 172.21.15.69:0/1304503572 shutdown_connections 2023-07-06T20:25:55.277 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:25:55.263+0000 7f38603ff700 1 -- 172.21.15.69:0/1304503572 wait complete. 2023-07-06T20:25:55.294 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:25:55.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:54 smithi156 ceph-mon[129280]: pgmap v525: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 62 MiB/s wr, 3.93k op/s 2023-07-06T20:25:55.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:54 smithi098 ceph-mon[128968]: pgmap v525: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 62 MiB/s wr, 3.93k op/s 2023-07-06T20:25:57.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:56 smithi069 ceph-mon[126399]: pgmap v526: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 57 MiB/s wr, 3.83k op/s 2023-07-06T20:25:57.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:56 smithi156 ceph-mon[129280]: pgmap v526: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 57 MiB/s wr, 3.83k op/s 2023-07-06T20:25:57.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:56 smithi098 ceph-mon[128968]: pgmap v526: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 57 MiB/s wr, 3.83k op/s 2023-07-06T20:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 250.0 s, 2551.4 tps, lat 39.175 ms stddev 42.777 2023-07-06T20:25:59.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:25:59 smithi069 ceph-mon[126399]: pgmap v527: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 55 MiB/s wr, 3.79k op/s 2023-07-06T20:25:59.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:25:59 smithi156 ceph-mon[129280]: pgmap v527: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 55 MiB/s wr, 3.79k op/s 2023-07-06T20:25:59.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:25:59 smithi098 ceph-mon[128968]: pgmap v527: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 55 MiB/s wr, 3.79k op/s 2023-07-06T20:26:01.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:01 smithi156 ceph-mon[129280]: pgmap v528: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 59 MiB/s wr, 3.74k op/s 2023-07-06T20:26:01.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:01 smithi098 ceph-mon[128968]: pgmap v528: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 59 MiB/s wr, 3.74k op/s 2023-07-06T20:26:01.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:01 smithi069 ceph-mon[126399]: pgmap v528: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 59 MiB/s wr, 3.74k op/s 2023-07-06T20:26:03.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:03 smithi069 ceph-mon[126399]: pgmap v529: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 2.17k op/s 2023-07-06T20:26:03.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:03 smithi156 ceph-mon[129280]: pgmap v529: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 2.17k op/s 2023-07-06T20:26:03.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:03 smithi098 ceph-mon[128968]: pgmap v529: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 2.17k op/s 2023-07-06T20:26:04.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 255.0 s, 1712.8 tps, lat 58.428 ms stddev 85.150 2023-07-06T20:26:05.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:05 smithi069 ceph-mon[126399]: pgmap v530: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 51 MiB/s wr, 2.42k op/s 2023-07-06T20:26:05.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:05 smithi156 ceph-mon[129280]: pgmap v530: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 51 MiB/s wr, 2.42k op/s 2023-07-06T20:26:05.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:05 smithi098 ceph-mon[128968]: pgmap v530: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 51 MiB/s wr, 2.42k op/s 2023-07-06T20:26:07.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:07 smithi069 ceph-mon[126399]: pgmap v531: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 49 MiB/s wr, 2.66k op/s 2023-07-06T20:26:07.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:07 smithi156 ceph-mon[129280]: pgmap v531: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 49 MiB/s wr, 2.66k op/s 2023-07-06T20:26:07.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:07 smithi098 ceph-mon[128968]: pgmap v531: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 49 MiB/s wr, 2.66k op/s 2023-07-06T20:26:09.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 260.0 s, 2330.0 tps, lat 42.924 ms stddev 75.049 2023-07-06T20:26:09.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:09 smithi069 ceph-mon[126399]: pgmap v532: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 52 MiB/s wr, 3.15k op/s 2023-07-06T20:26:09.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:09 smithi156 ceph-mon[129280]: pgmap v532: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 52 MiB/s wr, 3.15k op/s 2023-07-06T20:26:09.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:09 smithi098 ceph-mon[128968]: pgmap v532: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 52 MiB/s wr, 3.15k op/s 2023-07-06T20:26:11.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:11 smithi069 ceph-mon[126399]: pgmap v533: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 62 MiB/s wr, 4.08k op/s 2023-07-06T20:26:11.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:11 smithi156 ceph-mon[129280]: pgmap v533: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 62 MiB/s wr, 4.08k op/s 2023-07-06T20:26:11.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:11 smithi098 ceph-mon[128968]: pgmap v533: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 62 MiB/s wr, 4.08k op/s 2023-07-06T20:26:13.615 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:13 smithi156 ceph-mon[129280]: pgmap v534: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 55 MiB/s wr, 3.88k op/s 2023-07-06T20:26:13.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:13 smithi098 ceph-mon[128968]: pgmap v534: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 55 MiB/s wr, 3.88k op/s 2023-07-06T20:26:13.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:13 smithi069 ceph-mon[126399]: pgmap v534: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 55 MiB/s wr, 3.88k op/s 2023-07-06T20:26:14.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 265.0 s, 2396.8 tps, lat 41.759 ms stddev 60.063 2023-07-06T20:26:15.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:15 smithi156 ceph-mon[129280]: pgmap v535: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 63 MiB/s wr, 4.02k op/s 2023-07-06T20:26:15.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:15 smithi098 ceph-mon[128968]: pgmap v535: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 63 MiB/s wr, 4.02k op/s 2023-07-06T20:26:15.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:15 smithi069 ceph-mon[126399]: pgmap v535: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 63 MiB/s wr, 4.02k op/s 2023-07-06T20:26:17.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:17 smithi069 ceph-mon[126399]: pgmap v536: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 MiB/s wr, 3.05k op/s 2023-07-06T20:26:17.860 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:17 smithi156 ceph-mon[129280]: pgmap v536: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 MiB/s wr, 3.05k op/s 2023-07-06T20:26:17.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:17 smithi098 ceph-mon[128968]: pgmap v536: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 50 MiB/s wr, 3.05k op/s 2023-07-06T20:26:19.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 270.0 s, 2266.0 tps, lat 42.520 ms stddev 54.116 2023-07-06T20:26:19.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:19 smithi069 ceph-mon[126399]: pgmap v537: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 MiB/s wr, 2.10k op/s 2023-07-06T20:26:19.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:19 smithi156 ceph-mon[129280]: pgmap v537: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 MiB/s wr, 2.10k op/s 2023-07-06T20:26:19.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:19 smithi098 ceph-mon[128968]: pgmap v537: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 MiB/s wr, 2.10k op/s 2023-07-06T20:26:21.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:21 smithi069 ceph-mon[126399]: pgmap v538: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 49 MiB/s wr, 2.29k op/s 2023-07-06T20:26:21.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:21 smithi156 ceph-mon[129280]: pgmap v538: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 49 MiB/s wr, 2.29k op/s 2023-07-06T20:26:21.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:21 smithi098 ceph-mon[128968]: pgmap v538: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 49 MiB/s wr, 2.29k op/s 2023-07-06T20:26:23.764 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:23 smithi069 ceph-mon[126399]: pgmap v539: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 40 MiB/s wr, 1.70k op/s 2023-07-06T20:26:23.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:23 smithi156 ceph-mon[129280]: pgmap v539: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 40 MiB/s wr, 1.70k op/s 2023-07-06T20:26:23.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:23 smithi098 ceph-mon[128968]: pgmap v539: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 40 MiB/s wr, 1.70k op/s 2023-07-06T20:26:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 275.0 s, 2067.4 tps, lat 50.076 ms stddev 72.867 2023-07-06T20:26:25.295 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:26:25.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.690+0000 7f574b5ff700 1 -- 172.21.15.69:0/3391912572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5744103940 msgr2=0x7f574410a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:25.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.690+0000 7f574b5ff700 1 --2- 172.21.15.69:0/3391912572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5744103940 0x7f574410a270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f573c00b0a0 tx=0x7f573c00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.690+0000 7f574b5ff700 1 -- 172.21.15.69:0/3391912572 shutdown_connections 2023-07-06T20:26:25.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.690+0000 7f574b5ff700 1 --2- 172.21.15.69:0/3391912572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5744103940 0x7f574410a270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.690+0000 7f574b5ff700 1 --2- 172.21.15.69:0/3391912572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5744101e40 0x7f5744102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.690+0000 7f574b5ff700 1 --2- 172.21.15.69:0/3391912572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744102f90 0x7f5744103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.690+0000 7f574b5ff700 1 -- 172.21.15.69:0/3391912572 >> 172.21.15.69:0/3391912572 conn(0x7f57440fd480 msgr2=0x7f57440ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:26:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.690+0000 7f574b5ff700 1 -- 172.21.15.69:0/3391912572 shutdown_connections 2023-07-06T20:26:25.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.690+0000 7f574b5ff700 1 -- 172.21.15.69:0/3391912572 wait complete. 2023-07-06T20:26:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.692+0000 7f574b5ff700 1 Processor -- start 2023-07-06T20:26:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.692+0000 7f574b5ff700 1 -- start start 2023-07-06T20:26:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574b5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5744101e40 0x7f5744197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574b5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744102f90 0x7f57441980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574b5ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5744103940 0x7f574419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574b5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574419f730 con 0x7f5744102f90 2023-07-06T20:26:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574b5ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574419f870 con 0x7f5744103940 2023-07-06T20:26:25.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574b5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574419fa00 con 0x7f5744101e40 2023-07-06T20:26:25.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5744101e40 0x7f5744197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5744101e40 0x7f5744197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:60300/0 (socket says 172.21.15.69:60300) 2023-07-06T20:26:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574939b700 1 -- 172.21.15.69:0/3299688019 learned_addr learned my addr 172.21.15.69:0/3299688019 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:26:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f5749b9c700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5744103940 0x7f574419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f5748b9a700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744102f90 0x7f57441980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574939b700 1 -- 172.21.15.69:0/3299688019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5744103940 msgr2=0x7f574419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574939b700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5744103940 0x7f574419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574939b700 1 -- 172.21.15.69:0/3299688019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744102f90 msgr2=0x7f57441980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:25.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.693+0000 7f574939b700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744102f90 0x7f57441980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.694+0000 7f574939b700 1 -- 172.21.15.69:0/3299688019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c009d00 con 0x7f5744101e40 2023-07-06T20:26:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.694+0000 7f5748b9a700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744102f90 0x7f57441980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:26:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.694+0000 7f574939b700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5744101e40 0x7f5744197bb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f573400b550 tx=0x7f573400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:26:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.694+0000 7f573a7fc700 1 -- 172.21.15.69:0/3299688019 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734011830 con 0x7f5744101e40 2023-07-06T20:26:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.694+0000 7f573a7fc700 1 -- 172.21.15.69:0/3299688019 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5734011e50 con 0x7f5744101e40 2023-07-06T20:26:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.694+0000 7f574b5ff700 1 -- 172.21.15.69:0/3299688019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574419fce0 con 0x7f5744101e40 2023-07-06T20:26:25.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.694+0000 7f573a7fc700 1 -- 172.21.15.69:0/3299688019 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400f2e0 con 0x7f5744101e40 2023-07-06T20:26:25.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.694+0000 7f574b5ff700 1 -- 172.21.15.69:0/3299688019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57441a01a0 con 0x7f5744101e40 2023-07-06T20:26:25.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.705+0000 7f575eef0b80 1 Processor -- start 2023-07-06T20:26:25.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.706+0000 7f575eef0b80 1 -- start start 2023-07-06T20:26:25.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.706+0000 7f575eef0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.706+0000 7f575eef0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc0870 0x55ba2bdc0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.706+0000 7f575eef0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc1390 0x55ba2bdc1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.706+0000 7f575eef0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2bcc4cb0 con 0x55ba2bdc1390 2023-07-06T20:26:25.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.706+0000 7f575eef0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2bd17a40 con 0x55ba2bdbff20 2023-07-06T20:26:25.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.706+0000 7f575eef0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2bd10ed0 con 0x55ba2bdc0870 2023-07-06T20:26:25.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.706+0000 7f573a7fc700 1 -- 172.21.15.69:0/3299688019 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5734010580 con 0x7f5744101e40 2023-07-06T20:26:25.710 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.706+0000 7f5749b9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.706+0000 7f5749b9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:53672/0 (socket says 172.21.15.69:53672) 2023-07-06T20:26:25.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.706+0000 7f5749b9c700 1 -- 172.21.15.69:0/3085655192 learned_addr learned my addr 172.21.15.69:0/3085655192 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:26:25.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.706+0000 7f5748b9a700 1 --2- 172.21.15.69:0/3085655192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc1390 0x55ba2bdc1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.711 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.707+0000 7f574939b700 1 --2- 172.21.15.69:0/3085655192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc0870 0x55ba2bdc0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.707+0000 7f5749b9c700 1 -- 172.21.15.69:0/3085655192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc0870 msgr2=0x55ba2bdc0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:25.712 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.707+0000 7f5749b9c700 1 --2- 172.21.15.69:0/3085655192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc0870 0x55ba2bdc0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.707+0000 7f5749b9c700 1 -- 172.21.15.69:0/3085655192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc1390 msgr2=0x55ba2bdc1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:25.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.707+0000 7f5749b9c700 1 --2- 172.21.15.69:0/3085655192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc1390 0x55ba2bdc1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.707+0000 7f5749b9c700 1 -- 172.21.15.69:0/3085655192 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba2bb3ede0 con 0x55ba2bdbff20 2023-07-06T20:26:25.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.707+0000 7f5749b9c700 1 --2- 172.21.15.69:0/3085655192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc0330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f573c015000 tx=0x7f573c009470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f61f2cb018e4c978 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:26:25.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.708+0000 7f572f7fe700 1 -- 172.21.15.69:0/3085655192 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c00e050 con 0x55ba2bdbff20 2023-07-06T20:26:25.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.708+0000 7f572f7fe700 1 -- 172.21.15.69:0/3085655192 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f573c00bd30 con 0x55ba2bdbff20 2023-07-06T20:26:25.713 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.708+0000 7f572f7fe700 1 -- 172.21.15.69:0/3085655192 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c0054a0 con 0x55ba2bdbff20 2023-07-06T20:26:25.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.708+0000 7f573a7fc700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f57300a11d0 0x7f57300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.714 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.708+0000 7f575eef0b80 1 -- 172.21.15.69:0/3085655192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 msgr2=0x55ba2bdc0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:25.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.708+0000 7f575eef0b80 1 --2- 172.21.15.69:0/3085655192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc0330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f573c015000 tx=0x7f573c009470 comp rx=0 tx=0).stop 2023-07-06T20:26:25.715 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.709+0000 7f573a7fc700 1 -- 172.21.15.69:0/3299688019 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f57340cc480 con 0x7f5744101e40 2023-07-06T20:26:25.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.709+0000 7f575eef0b80 1 -- 172.21.15.69:0/3085655192 shutdown_connections 2023-07-06T20:26:25.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.709+0000 7f575eef0b80 1 --2- 172.21.15.69:0/3085655192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc1390 0x55ba2bdc1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.709+0000 7f575eef0b80 1 --2- 172.21.15.69:0/3085655192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc0330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.709+0000 7f575eef0b80 1 --2- 172.21.15.69:0/3085655192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc0870 0x55ba2bdc0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.709+0000 7f575eef0b80 1 -- 172.21.15.69:0/3085655192 >> 172.21.15.69:0/3085655192 conn(0x55ba2bdb5640 msgr2=0x55ba2bdba9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:26:25.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.709+0000 7f574939b700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f57300a11d0 0x7f57300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.709+0000 7f575eef0b80 1 -- 172.21.15.69:0/3085655192 shutdown_connections 2023-07-06T20:26:25.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.709+0000 7f575eef0b80 1 -- 172.21.15.69:0/3085655192 wait complete. 2023-07-06T20:26:25.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.710+0000 7f575eef0b80 1 -- start start 2023-07-06T20:26:25.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.710+0000 7f575eef0b80 1 Processor -- start 2023-07-06T20:26:25.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.711+0000 7f575eef0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.711+0000 7f575eef0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdc0870 0x55ba2bdc9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.711+0000 7f575eef0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc1390 0x55ba2bdd0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.711+0000 7f575eef0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2bcb0ac0 con 0x55ba2bdbff20 2023-07-06T20:26:25.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.711+0000 7f575eef0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2bcb07c0 con 0x55ba2bdc0870 2023-07-06T20:26:25.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.711+0000 7f575eef0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2bb94920 con 0x55ba2bdc1390 2023-07-06T20:26:25.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.712+0000 7f575eef0b80 10 client.0 _ll_register_callbacks cb 0x55ba2bcae760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:26:25.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.713+0000 7f5749b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.713+0000 7f5749b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57468/0 (socket says 172.21.15.69:57468) 2023-07-06T20:26:25.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.713+0000 7f5749b9c700 1 -- 172.21.15.69:0/2624823252 learned_addr learned my addr 172.21.15.69:0/2624823252 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:26:25.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.713+0000 7f5748b9a700 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdc0870 0x55ba2bdc9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.713+0000 7f5749b9c700 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc1390 0x55ba2bdd0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.713+0000 7f574939b700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f57300a11d0 0x7f57300a3680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f57340c9ef0 tx=0x7f5734010980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:26:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.713+0000 7f5748b9a700 1 -- 172.21.15.69:0/2624823252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc1390 msgr2=0x55ba2bdd0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.713+0000 7f5748b9a700 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc1390 0x55ba2bdd0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.714+0000 7f5748b9a700 1 -- 172.21.15.69:0/2624823252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdbff20 msgr2=0x55ba2bdc9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.714+0000 7f5748b9a700 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.714+0000 7f5748b9a700 1 -- 172.21.15.69:0/2624823252 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574410ab70 con 0x55ba2bdc0870 2023-07-06T20:26:25.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.719+0000 7f5748b9a700 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdc0870 0x55ba2bdc9d70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5740009870 tx=0x7f5740009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d63cdca71286b6c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:26:25.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.719+0000 7f572cff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:26:25.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.719+0000 7f572cff9700 1 -- 172.21.15.69:0/2624823252 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740003cb0 con 0x55ba2bdc0870 2023-07-06T20:26:25.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.719+0000 7f572cff9700 1 -- 172.21.15.69:0/2624823252 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f5740003e10 con 0x55ba2bdc0870 2023-07-06T20:26:25.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.719+0000 7f575eef0b80 1 -- 172.21.15.69:0/2624823252 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba2bc6a4f0 con 0x55ba2bdc0870 2023-07-06T20:26:25.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.720+0000 7f572cff9700 1 -- 172.21.15.69:0/2624823252 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f574001b440 con 0x55ba2bdc0870 2023-07-06T20:26:25.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.720+0000 7f575eef0b80 10 client.25640 fetch_fsmap learned FSMap version 30 2023-07-06T20:26:25.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.720+0000 7f575eef0b80 1 -- 172.21.15.69:0/2624823252 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ba2bb27c60 con 0x55ba2bdc0870 2023-07-06T20:26:25.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.720+0000 7f572cff9700 1 -- 172.21.15.69:0/2624823252 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba2bc6a4f0 con 0x55ba2bdc0870 2023-07-06T20:26:25.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.721+0000 7f572cff9700 1 -- 172.21.15.69:0/2624823252 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f57400127e0 con 0x55ba2bdc0870 2023-07-06T20:26:25.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.721+0000 7f575eef0b80 10 client.25640 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:26:25.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.721+0000 7f575eef0b80 10 client.25640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:26:25.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.721+0000 7f575eef0b80 20 client.25640 populate_metadata read hostname 'smithi069' 2023-07-06T20:26:25.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.721+0000 7f575eef0b80 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2bdd3dd0 0x55ba2bdd61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.721+0000 7f575eef0b80 4 client.25640 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:26:25.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.721+0000 7f575eef0b80 1 -- 172.21.15.69:0/2624823252 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba2bd8ed60 con 0x55ba2bdd3dd0 2023-07-06T20:26:25.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.721+0000 7f5749b9c700 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2bdd3dd0 0x55ba2bdd61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.721+0000 7f572cff9700 10 client.25640 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:26:25.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.721+0000 7f5749b9c700 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2bdd3dd0 0x55ba2bdd61e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d712138a6f0064 server_cookie=ca3caf60034c31af in_seq=0 out_seq=0 2023-07-06T20:26:25.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.722+0000 7f572cff9700 1 -- 172.21.15.69:0/2624823252 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ba2bd8ed60 con 0x55ba2bdd3dd0 2023-07-06T20:26:25.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.722+0000 7f572cff9700 10 client.25640 handle_command_reply: tid=0 2023-07-06T20:26:25.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.722+0000 7f575eef0b80 1 client.25640 shutdown 2023-07-06T20:26:25.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.723+0000 7f575eef0b80 1 -- 172.21.15.69:0/2624823252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdc0870 msgr2=0x55ba2bdc9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:25.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.723+0000 7f575eef0b80 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdc0870 0x55ba2bdc9d70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5740009870 tx=0x7f5740009b80 comp rx=0 tx=0).stop 2023-07-06T20:26:25.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.723+0000 7f575eef0b80 1 -- 172.21.15.69:0/2624823252 shutdown_connections 2023-07-06T20:26:25.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.723+0000 7f575eef0b80 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2bdd3dd0 0x55ba2bdd61e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.723+0000 7f575eef0b80 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc1390 0x55ba2bdd0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.723+0000 7f575eef0b80 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.723+0000 7f575eef0b80 1 --2- 172.21.15.69:0/2624823252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdc0870 0x55ba2bdc9d70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.723+0000 7f575eef0b80 1 -- 172.21.15.69:0/2624823252 >> 172.21.15.69:0/2624823252 conn(0x55ba2bdb5640 msgr2=0x55ba2bdba0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:26:25.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.723+0000 7f572cff9700 0 client.25640 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:26:25.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.723+0000 7f575eef0b80 1 -- 172.21.15.69:0/2624823252 shutdown_connections 2023-07-06T20:26:25.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.724+0000 7f575eef0b80 1 -- 172.21.15.69:0/2624823252 wait complete. 2023-07-06T20:26:25.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.724+0000 7f575eef0b80 20 client.25640 trim_cache size 0 max 16384 2023-07-06T20:26:25.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.752+0000 7f575eef0b80 1 Processor -- start 2023-07-06T20:26:25.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.753+0000 7f575eef0b80 1 -- start start 2023-07-06T20:26:25.755 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.753+0000 7f575eef0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.753+0000 7f575eef0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc0870 0x55ba2bdd0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.753+0000 7f575eef0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc1390 0x55ba2bdc7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.753+0000 7f575eef0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2bdb51c0 con 0x55ba2bdc1390 2023-07-06T20:26:25.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.753+0000 7f575eef0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2bdc8270 con 0x55ba2bdbff20 2023-07-06T20:26:25.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.753+0000 7f575eef0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2bdc7530 con 0x55ba2bdc0870 2023-07-06T20:26:25.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.753+0000 7f5748b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc0870 0x55ba2bdd0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.753+0000 7f5748b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc0870 0x55ba2bdd0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:60316/0 (socket says 172.21.15.69:60316) 2023-07-06T20:26:25.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.753+0000 7f5748b9a700 1 -- 172.21.15.69:0/1547941325 learned_addr learned my addr 172.21.15.69:0/1547941325 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:26:25.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.753+0000 7f5749b9c700 1 --2- 172.21.15.69:0/1547941325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.753+0000 7f5749b9c700 1 --2- 172.21.15.69:0/1547941325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc1390 0x55ba2bdc7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.754+0000 7f5748b9a700 1 -- 172.21.15.69:0/1547941325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 msgr2=0x55ba2bdc8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:25.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.754+0000 7f5748b9a700 1 --2- 172.21.15.69:0/1547941325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.754+0000 7f5748b9a700 1 -- 172.21.15.69:0/1547941325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc1390 msgr2=0x55ba2bdc7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:25.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.754+0000 7f5748b9a700 1 --2- 172.21.15.69:0/1547941325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc1390 0x55ba2bdc7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.754+0000 7f5748b9a700 1 -- 172.21.15.69:0/1547941325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba2bb65250 con 0x55ba2bdc0870 2023-07-06T20:26:25.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.754+0000 7f5748b9a700 1 --2- 172.21.15.69:0/1547941325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc0870 0x55ba2bdd0d70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5740012b90 tx=0x7f5740009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcf05641599fe68d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:26:25.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.755+0000 7f572ffff700 1 -- 172.21.15.69:0/1547941325 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740012920 con 0x55ba2bdc0870 2023-07-06T20:26:25.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.755+0000 7f572ffff700 1 -- 172.21.15.69:0/1547941325 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5740012440 con 0x55ba2bdc0870 2023-07-06T20:26:25.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.755+0000 7f572ffff700 1 -- 172.21.15.69:0/1547941325 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740025df0 con 0x55ba2bdc0870 2023-07-06T20:26:25.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.756+0000 7f575eef0b80 1 -- 172.21.15.69:0/1547941325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc0870 msgr2=0x55ba2bdd0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:25.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.756+0000 7f575eef0b80 1 --2- 172.21.15.69:0/1547941325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc0870 0x55ba2bdd0d70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5740012b90 tx=0x7f5740009f50 comp rx=0 tx=0).stop 2023-07-06T20:26:25.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.756+0000 7f575eef0b80 1 -- 172.21.15.69:0/1547941325 shutdown_connections 2023-07-06T20:26:25.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.756+0000 7f575eef0b80 1 --2- 172.21.15.69:0/1547941325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc1390 0x55ba2bdc7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.756+0000 7f575eef0b80 1 --2- 172.21.15.69:0/1547941325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdc8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.756+0000 7f575eef0b80 1 --2- 172.21.15.69:0/1547941325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc0870 0x55ba2bdd0d70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.756+0000 7f575eef0b80 1 -- 172.21.15.69:0/1547941325 >> 172.21.15.69:0/1547941325 conn(0x55ba2bdb5640 msgr2=0x55ba2bde1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:26:25.760 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.757+0000 7f575eef0b80 1 -- 172.21.15.69:0/1547941325 shutdown_connections 2023-07-06T20:26:25.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.757+0000 7f575eef0b80 1 -- 172.21.15.69:0/1547941325 wait complete. 2023-07-06T20:26:25.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.758+0000 7f575eef0b80 1 -- start start 2023-07-06T20:26:25.761 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.758+0000 7f575eef0b80 1 Processor -- start 2023-07-06T20:26:25.764 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:25 smithi069 ceph-mon[126399]: pgmap v540: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 60 MiB/s wr, 3.64k op/s 2023-07-06T20:26:25.764 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.762+0000 7f575eef0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdcced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.762+0000 7f575eef0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc0870 0x55ba2bdcd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.762+0000 7f575eef0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc1390 0x55ba2bdd0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.762+0000 7f575eef0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2bdd0970 con 0x55ba2bdc0870 2023-07-06T20:26:25.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.762+0000 7f575eef0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2bdd0ae0 con 0x55ba2bdbff20 2023-07-06T20:26:25.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.762+0000 7f575eef0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2bdd0ca0 con 0x55ba2bdc1390 2023-07-06T20:26:25.765 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.762+0000 7f575eef0b80 10 client.0 _ll_register_callbacks cb 0x55ba2bcae760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:26:25.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.763+0000 7f5748b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc0870 0x55ba2bdcd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.763+0000 7f5748b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc0870 0x55ba2bdcd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57488/0 (socket says 172.21.15.69:57488) 2023-07-06T20:26:25.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.763+0000 7f5748b9a700 1 -- 172.21.15.69:0/2371726036 learned_addr learned my addr 172.21.15.69:0/2371726036 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:26:25.766 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.763+0000 7f5748b9a700 1 -- 172.21.15.69:0/2371726036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc1390 msgr2=0x55ba2bdd0430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:26:25.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.763+0000 7f5748b9a700 1 --2- 172.21.15.69:0/2371726036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc1390 0x55ba2bdd0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.763+0000 7f5748b9a700 1 -- 172.21.15.69:0/2371726036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 msgr2=0x55ba2bdcced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:25.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.763+0000 7f5748b9a700 1 --2- 172.21.15.69:0/2371726036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdcced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.763+0000 7f5748b9a700 1 -- 172.21.15.69:0/2371726036 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba2bb27c60 con 0x55ba2bdc0870 2023-07-06T20:26:25.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.764+0000 7f5748b9a700 1 --2- 172.21.15.69:0/2371726036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc0870 0x55ba2bdcd410 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5740009bc0 tx=0x7f5740026010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f142d7d2f1c7259a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:26:25.767 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.764+0000 7f572cff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:26:25.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.764+0000 7f572cff9700 1 -- 172.21.15.69:0/2371726036 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57400253e0 con 0x55ba2bdc0870 2023-07-06T20:26:25.768 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.764+0000 7f572cff9700 1 -- 172.21.15.69:0/2371726036 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f5740000e40 con 0x55ba2bdc0870 2023-07-06T20:26:25.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.764+0000 7f575eef0b80 1 -- 172.21.15.69:0/2371726036 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f573c00bd30 con 0x55ba2bdc0870 2023-07-06T20:26:25.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.765+0000 7f572cff9700 1 -- 172.21.15.69:0/2371726036 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5740025540 con 0x55ba2bdc0870 2023-07-06T20:26:25.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.765+0000 7f575eef0b80 10 client.15516 fetch_fsmap learned FSMap version 30 2023-07-06T20:26:25.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.765+0000 7f575eef0b80 1 -- 172.21.15.69:0/2371726036 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ba2bb3d2b0 con 0x55ba2bdc0870 2023-07-06T20:26:25.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.765+0000 7f572cff9700 1 -- 172.21.15.69:0/2371726036 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f573c00bd30 con 0x55ba2bdc0870 2023-07-06T20:26:25.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.766+0000 7f572cff9700 1 -- 172.21.15.69:0/2371726036 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5740037070 con 0x55ba2bdc0870 2023-07-06T20:26:25.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.766+0000 7f575eef0b80 10 client.15516 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:26:25.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.766+0000 7f575eef0b80 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:26:25.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.766+0000 7f575eef0b80 20 client.15516 populate_metadata read hostname 'smithi069' 2023-07-06T20:26:25.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.766+0000 7f575eef0b80 1 --2- 172.21.15.69:0/2371726036 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2be334f0 0x55ba2be35900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:25.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.766+0000 7f575eef0b80 4 client.15516 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:26:25.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.766+0000 7f575eef0b80 1 -- 172.21.15.69:0/2371726036 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba2bd09eb0 con 0x55ba2be334f0 2023-07-06T20:26:25.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.766+0000 7f5749b9c700 1 --2- 172.21.15.69:0/2371726036 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2be334f0 0x55ba2be35900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:25.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.767+0000 7f572cff9700 10 client.15516 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:26:25.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.767+0000 7f5749b9c700 1 --2- 172.21.15.69:0/2371726036 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2be334f0 0x55ba2be35900 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7995578410f59d1d server_cookie=cd4b3ffe59831e23 in_seq=0 out_seq=0 2023-07-06T20:26:25.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.767+0000 7f572cff9700 1 -- 172.21.15.69:0/2371726036 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ba2bd09eb0 con 0x55ba2be334f0 2023-07-06T20:26:25.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.767+0000 7f572cff9700 10 client.15516 handle_command_reply: tid=0 2023-07-06T20:26:25.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.767+0000 7f575eef0b80 1 client.15516 shutdown 2023-07-06T20:26:25.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.767+0000 7f575eef0b80 1 -- 172.21.15.69:0/2371726036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc0870 msgr2=0x55ba2bdcd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:25.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.767+0000 7f575eef0b80 1 --2- 172.21.15.69:0/2371726036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc0870 0x55ba2bdcd410 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5740009bc0 tx=0x7f5740026010 comp rx=0 tx=0).stop 2023-07-06T20:26:25.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.768+0000 7f575eef0b80 1 -- 172.21.15.69:0/2371726036 shutdown_connections 2023-07-06T20:26:25.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.768+0000 7f575eef0b80 1 --2- 172.21.15.69:0/2371726036 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55ba2be334f0 0x55ba2be35900 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.768+0000 7f575eef0b80 1 --2- 172.21.15.69:0/2371726036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2bdc1390 0x55ba2bdd0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.768+0000 7f575eef0b80 1 --2- 172.21.15.69:0/2371726036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba2bdbff20 0x55ba2bdcced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.768+0000 7f575eef0b80 1 --2- 172.21.15.69:0/2371726036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba2bdc0870 0x55ba2bdcd410 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.768+0000 7f575eef0b80 1 -- 172.21.15.69:0/2371726036 >> 172.21.15.69:0/2371726036 conn(0x55ba2bdb5640 msgr2=0x55ba2bdc8500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:26:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.768+0000 7f572cff9700 0 client.15516 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:26:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.768+0000 7f575eef0b80 1 -- 172.21.15.69:0/2371726036 shutdown_connections 2023-07-06T20:26:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.768+0000 7f575eef0b80 1 -- 172.21.15.69:0/2371726036 wait complete. 2023-07-06T20:26:25.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.768+0000 7f575eef0b80 20 client.15516 trim_cache size 0 max 16384 2023-07-06T20:26:25.774 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:26:25.774 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:26:25.774 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:26:25.774 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:26:25.774 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:26:25.774 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:26:25.775 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:26:25.775 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:26:25.775 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:26:25.775 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:26:25.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.769+0000 7f574b5ff700 1 -- 172.21.15.69:0/3299688019 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f57300a11d0 msgr2=0x7f57300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.769+0000 7f574b5ff700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f57300a11d0 0x7f57300a3680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f57340c9ef0 tx=0x7f5734010980 comp rx=0 tx=0).stop 2023-07-06T20:26:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.769+0000 7f574b5ff700 1 -- 172.21.15.69:0/3299688019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5744101e40 msgr2=0x7f5744197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.769+0000 7f574b5ff700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5744101e40 0x7f5744197bb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f573400b550 tx=0x7f573400b910 comp rx=0 tx=0).stop 2023-07-06T20:26:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.769+0000 7f574b5ff700 1 -- 172.21.15.69:0/3299688019 shutdown_connections 2023-07-06T20:26:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.769+0000 7f574b5ff700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f57300a11d0 0x7f57300a3680 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.769+0000 7f574b5ff700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5744103940 0x7f574419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.769+0000 7f574b5ff700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5744101e40 0x7f5744197bb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.769+0000 7f574b5ff700 1 --2- 172.21.15.69:0/3299688019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5744102f90 0x7f57441980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:25.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.769+0000 7f574b5ff700 1 -- 172.21.15.69:0/3299688019 >> 172.21.15.69:0/3299688019 conn(0x7f57440fd480 msgr2=0x7f5744106400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:26:25.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.769+0000 7f574b5ff700 1 -- 172.21.15.69:0/3299688019 shutdown_connections 2023-07-06T20:26:25.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:25.769+0000 7f574b5ff700 1 -- 172.21.15.69:0/3299688019 wait complete. 2023-07-06T20:26:25.804 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:26:25.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:25 smithi156 ceph-mon[129280]: pgmap v540: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 60 MiB/s wr, 3.64k op/s 2023-07-06T20:26:25.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:25 smithi098 ceph-mon[128968]: pgmap v540: 97 pgs: 97 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 60 MiB/s wr, 3.64k op/s 2023-07-06T20:26:27.767 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:27 smithi069 ceph-mon[126399]: pgmap v541: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 56 MiB/s wr, 3.87k op/s 2023-07-06T20:26:27.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:27 smithi156 ceph-mon[129280]: pgmap v541: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 56 MiB/s wr, 3.87k op/s 2023-07-06T20:26:27.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:27 smithi098 ceph-mon[128968]: pgmap v541: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 56 MiB/s wr, 3.87k op/s 2023-07-06T20:26:29.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 280.0 s, 2560.8 tps, lat 39.017 ms stddev 44.142 2023-07-06T20:26:29.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:29 smithi156 ceph-mon[129280]: pgmap v542: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 56 MiB/s wr, 3.80k op/s 2023-07-06T20:26:29.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:29 smithi098 ceph-mon[128968]: pgmap v542: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 56 MiB/s wr, 3.80k op/s 2023-07-06T20:26:30.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:29 smithi069 ceph-mon[126399]: pgmap v542: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 56 MiB/s wr, 3.80k op/s 2023-07-06T20:26:31.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:31 smithi156 ceph-mon[129280]: pgmap v543: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 61 MiB/s wr, 3.93k op/s 2023-07-06T20:26:31.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:31 smithi098 ceph-mon[128968]: pgmap v543: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 61 MiB/s wr, 3.93k op/s 2023-07-06T20:26:32.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:31 smithi069 ceph-mon[126399]: pgmap v543: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 61 MiB/s wr, 3.93k op/s 2023-07-06T20:26:34.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:33 smithi069 ceph-mon[126399]: pgmap v544: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 49 MiB/s wr, 3.19k op/s 2023-07-06T20:26:34.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:33 smithi156 ceph-mon[129280]: pgmap v544: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 49 MiB/s wr, 3.19k op/s 2023-07-06T20:26:34.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:33 smithi098 ceph-mon[128968]: pgmap v544: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 49 MiB/s wr, 3.19k op/s 2023-07-06T20:26:34.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 285.0 s, 2497.4 tps, lat 40.060 ms stddev 52.137 2023-07-06T20:26:35.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:34 smithi069 ceph-mon[126399]: pgmap v545: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 56 MiB/s wr, 3.04k op/s 2023-07-06T20:26:35.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:34 smithi156 ceph-mon[129280]: pgmap v545: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 56 MiB/s wr, 3.04k op/s 2023-07-06T20:26:35.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:34 smithi098 ceph-mon[128968]: pgmap v545: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 56 MiB/s wr, 3.04k op/s 2023-07-06T20:26:37.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:36 smithi069 ceph-mon[126399]: pgmap v546: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 42 MiB/s wr, 1.76k op/s 2023-07-06T20:26:37.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:36 smithi156 ceph-mon[129280]: pgmap v546: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 42 MiB/s wr, 1.76k op/s 2023-07-06T20:26:37.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:36 smithi098 ceph-mon[128968]: pgmap v546: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 42 MiB/s wr, 1.76k op/s 2023-07-06T20:26:39.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 290.0 s, 1545.3 tps, lat 64.789 ms stddev 120.739 2023-07-06T20:26:39.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:38 smithi069 ceph-mon[126399]: pgmap v547: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 MiB/s wr, 2.15k op/s 2023-07-06T20:26:39.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:38 smithi156 ceph-mon[129280]: pgmap v547: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 MiB/s wr, 2.15k op/s 2023-07-06T20:26:39.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:38 smithi098 ceph-mon[128968]: pgmap v547: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 MiB/s wr, 2.15k op/s 2023-07-06T20:26:41.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:41 smithi156 ceph-mon[129280]: pgmap v548: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 59 MiB/s wr, 3.90k op/s 2023-07-06T20:26:41.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:41 smithi098 ceph-mon[128968]: pgmap v548: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 59 MiB/s wr, 3.90k op/s 2023-07-06T20:26:41.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:41 smithi069 ceph-mon[126399]: pgmap v548: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 59 MiB/s wr, 3.90k op/s 2023-07-06T20:26:43.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:43 smithi156 ceph-mon[129280]: pgmap v549: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 MiB/s wr, 3.93k op/s 2023-07-06T20:26:43.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:43 smithi098 ceph-mon[128968]: pgmap v549: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 MiB/s wr, 3.93k op/s 2023-07-06T20:26:43.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:43 smithi069 ceph-mon[126399]: pgmap v549: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 MiB/s wr, 3.93k op/s 2023-07-06T20:26:44.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 295.0 s, 2748.4 tps, lat 34.562 ms stddev 19.642 2023-07-06T20:26:45.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:45 smithi156 ceph-mon[129280]: pgmap v550: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 MiB/s wr, 3.99k op/s 2023-07-06T20:26:45.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:45 smithi098 ceph-mon[128968]: pgmap v550: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 MiB/s wr, 3.99k op/s 2023-07-06T20:26:45.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:45 smithi069 ceph-mon[126399]: pgmap v550: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 MiB/s wr, 3.99k op/s 2023-07-06T20:26:47.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:47 smithi156 ceph-mon[129280]: pgmap v551: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 55 MiB/s wr, 3.85k op/s 2023-07-06T20:26:47.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:47 smithi098 ceph-mon[128968]: pgmap v551: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 55 MiB/s wr, 3.85k op/s 2023-07-06T20:26:47.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:47 smithi069 ceph-mon[126399]: pgmap v551: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 55 MiB/s wr, 3.85k op/s 2023-07-06T20:26:49.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 300.0 s, 2541.9 tps, lat 41.121 ms stddev 44.691 2023-07-06T20:26:49.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:49 smithi156 ceph-mon[129280]: pgmap v552: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 49 MiB/s wr, 3.24k op/s 2023-07-06T20:26:49.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:49 smithi098 ceph-mon[128968]: pgmap v552: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 49 MiB/s wr, 3.24k op/s 2023-07-06T20:26:49.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:49 smithi069 ceph-mon[126399]: pgmap v552: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 49 MiB/s wr, 3.24k op/s 2023-07-06T20:26:51.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:51 smithi069 ceph-mon[126399]: pgmap v553: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 51 MiB/s wr, 2.63k op/s 2023-07-06T20:26:51.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:51 smithi156 ceph-mon[129280]: pgmap v553: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 51 MiB/s wr, 2.63k op/s 2023-07-06T20:26:51.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:51 smithi098 ceph-mon[128968]: pgmap v553: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 51 MiB/s wr, 2.63k op/s 2023-07-06T20:26:52.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:52 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:26:52.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:52 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:26:52.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:52 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:26:53.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: pgmap v554: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 MiB/s wr, 981 op/s 2023-07-06T20:26:53.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-07-06T20:26:53.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-06T20:26:53.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-07-06T20:26:53.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-06T20:26:53.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.514 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:26:53.514 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:26:53.514 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:53 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:26:53.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: pgmap v554: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 MiB/s wr, 981 op/s 2023-07-06T20:26:53.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-07-06T20:26:53.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-06T20:26:53.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-07-06T20:26:53.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-06T20:26:53.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:26:53.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:26:53.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:53 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:26:53.636 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: pgmap v554: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 MiB/s wr, 981 op/s 2023-07-06T20:26:53.637 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.637 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.637 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.638 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.638 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-07-06T20:26:53.638 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-06T20:26:53.638 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.639 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.639 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.639 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.639 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-07-06T20:26:53.640 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-06T20:26:53.640 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.640 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.641 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.641 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:26:53.641 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:26:53.641 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:26:53.642 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:53 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:26:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 305.0 s, 879.8 tps, lat 113.520 ms stddev 149.994 2023-07-06T20:26:54.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:54 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi069 to 220.8M 2023-07-06T20:26:54.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:54 smithi069 ceph-mon[126399]: Unable to set osd_memory_target on smithi069 to 231614566: error parsing value: Value '231614566' is below minimum 939524096 2023-07-06T20:26:54.613 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:54 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi069 to 220.8M 2023-07-06T20:26:54.613 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:54 smithi156 ceph-mon[129280]: Unable to set osd_memory_target on smithi069 to 231614566: error parsing value: Value '231614566' is below minimum 939524096 2023-07-06T20:26:54.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:54 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi069 to 220.8M 2023-07-06T20:26:54.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:54 smithi098 ceph-mon[128968]: Unable to set osd_memory_target on smithi069 to 231614566: error parsing value: Value '231614566' is below minimum 939524096 2023-07-06T20:26:55.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:55 smithi069 ceph-mon[126399]: pgmap v555: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 48 MiB/s wr, 2.30k op/s 2023-07-06T20:26:55.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:55 smithi156 ceph-mon[129280]: pgmap v555: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 48 MiB/s wr, 2.30k op/s 2023-07-06T20:26:55.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:55 smithi098 ceph-mon[128968]: pgmap v555: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 48 MiB/s wr, 2.30k op/s 2023-07-06T20:26:55.805 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:26:56.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.176+0000 7fce357ff700 1 -- 172.21.15.69:0/1197542262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3010be90 msgr2=0x7fce3010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:56.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.176+0000 7fce357ff700 1 --2- 172.21.15.69:0/1197542262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3010be90 0x7fce3010e320 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fce280098f0 tx=0x7fce28009c00 comp rx=0 tx=0).stop 2023-07-06T20:26:56.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.176+0000 7fce357ff700 1 -- 172.21.15.69:0/1197542262 shutdown_connections 2023-07-06T20:26:56.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.176+0000 7fce357ff700 1 --2- 172.21.15.69:0/1197542262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3010be90 0x7fce3010e320 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.176+0000 7fce357ff700 1 --2- 172.21.15.69:0/1197542262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce30071010 0x7fce30071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.176+0000 7fce357ff700 1 --2- 172.21.15.69:0/1197542262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce30071960 0x7fce3010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.176+0000 7fce357ff700 1 -- 172.21.15.69:0/1197542262 >> 172.21.15.69:0/1197542262 conn(0x7fce3006cbc0 msgr2=0x7fce3006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:26:56.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.176+0000 7fce357ff700 1 -- 172.21.15.69:0/1197542262 shutdown_connections 2023-07-06T20:26:56.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.177+0000 7fce357ff700 1 -- 172.21.15.69:0/1197542262 wait complete. 2023-07-06T20:26:56.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.178+0000 7fce357ff700 1 Processor -- start 2023-07-06T20:26:56.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.178+0000 7fce357ff700 1 -- start start 2023-07-06T20:26:56.181 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.178+0000 7fce357ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce30071010 0x7fce301a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.181 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.178+0000 7fce357ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce30071960 0x7fce301a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce357ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3010be90 0x7fce301abd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce357ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce301ac350 con 0x7fce3010be90 2023-07-06T20:26:56.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce357ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce301ac490 con 0x7fce30071010 2023-07-06T20:26:56.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce357ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce301ac620 con 0x7fce30071960 2023-07-06T20:26:56.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce2effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce30071010 0x7fce301a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce2effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce30071010 0x7fce301a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:52692/0 (socket says 172.21.15.69:52692) 2023-07-06T20:26:56.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce2effd700 1 -- 172.21.15.69:0/401680301 learned_addr learned my addr 172.21.15.69:0/401680301 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:26:56.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce267fc700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce30071960 0x7fce301a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce2f7fe700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3010be90 0x7fce301abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce2effd700 1 -- 172.21.15.69:0/401680301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce30071960 msgr2=0x7fce301a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:56.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce2effd700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce30071960 0x7fce301a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce2effd700 1 -- 172.21.15.69:0/401680301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3010be90 msgr2=0x7fce301abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:56.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce2effd700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3010be90 0x7fce301abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.183 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.179+0000 7fce2effd700 1 -- 172.21.15.69:0/401680301 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce28009520 con 0x7fce30071010 2023-07-06T20:26:56.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.180+0000 7fce2effd700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce30071010 0x7fce301a47d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce2000b530 tx=0x7fce2000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:26:56.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.180+0000 7fce2cff9700 1 -- 172.21.15.69:0/401680301 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20011830 con 0x7fce30071010 2023-07-06T20:26:56.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.180+0000 7fce357ff700 1 -- 172.21.15.69:0/401680301 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce301ac900 con 0x7fce30071010 2023-07-06T20:26:56.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.180+0000 7fce2cff9700 1 -- 172.21.15.69:0/401680301 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce20011e50 con 0x7fce30071010 2023-07-06T20:26:56.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.180+0000 7fce2cff9700 1 -- 172.21.15.69:0/401680301 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2000f2e0 con 0x7fce30071010 2023-07-06T20:26:56.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.180+0000 7fce357ff700 1 -- 172.21.15.69:0/401680301 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce301acee0 con 0x7fce30071010 2023-07-06T20:26:56.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.192+0000 7fce4910db80 1 Processor -- start 2023-07-06T20:26:56.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.192+0000 7fce4910db80 1 -- start start 2023-07-06T20:26:56.194 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce4910db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a14f20 0x55bde0a15330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce4910db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a15e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.195 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce4910db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a16390 0x55bde0a16830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce4910db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde0919cb0 con 0x55bde0a14f20 2023-07-06T20:26:56.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce4910db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde096ca40 con 0x55bde0a16390 2023-07-06T20:26:56.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce4910db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde0965ed0 con 0x55bde0a15870 2023-07-06T20:26:56.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce2cff9700 1 -- 172.21.15.69:0/401680301 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fce20010580 con 0x7fce30071010 2023-07-06T20:26:56.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce267fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a16390 0x55bde0a16830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce267fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a16390 0x55bde0a16830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:52694/0 (socket says 172.21.15.69:52694) 2023-07-06T20:26:56.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce267fc700 1 -- 172.21.15.69:0/1492957080 learned_addr learned my addr 172.21.15.69:0/1492957080 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:26:56.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce2f7fe700 1 --2- 172.21.15.69:0/1492957080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a14f20 0x55bde0a15330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce2effd700 1 --2- 172.21.15.69:0/1492957080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a15e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce267fc700 1 -- 172.21.15.69:0/1492957080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 msgr2=0x55bde0a15e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:56.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce267fc700 1 --2- 172.21.15.69:0/1492957080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a15e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce267fc700 1 -- 172.21.15.69:0/1492957080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a14f20 msgr2=0x55bde0a15330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:56.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce267fc700 1 --2- 172.21.15.69:0/1492957080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a14f20 0x55bde0a15330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce267fc700 1 -- 172.21.15.69:0/1492957080 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bde0793de0 con 0x55bde0a16390 2023-07-06T20:26:56.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.193+0000 7fce2effd700 1 --2- 172.21.15.69:0/1492957080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a15e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:26:56.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.194+0000 7fce267fc700 1 --2- 172.21.15.69:0/1492957080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a16390 0x55bde0a16830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce180093b0 tx=0x7fce180099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87a7f08aa6a4be3a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:26:56.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.194+0000 7fce257fa700 1 -- 172.21.15.69:0/1492957080 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1800e070 con 0x55bde0a16390 2023-07-06T20:26:56.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.194+0000 7fce257fa700 1 -- 172.21.15.69:0/1492957080 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce1800f040 con 0x55bde0a16390 2023-07-06T20:26:56.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.194+0000 7fce257fa700 1 -- 172.21.15.69:0/1492957080 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce180147a0 con 0x55bde0a16390 2023-07-06T20:26:56.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.195+0000 7fce4910db80 1 -- 172.21.15.69:0/1492957080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a16390 msgr2=0x55bde0a16830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:56.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.195+0000 7fce4910db80 1 --2- 172.21.15.69:0/1492957080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a16390 0x55bde0a16830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce180093b0 tx=0x7fce180099e0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.195+0000 7fce2cff9700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fce100a1150 0x7fce100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.195+0000 7fce4910db80 1 -- 172.21.15.69:0/1492957080 shutdown_connections 2023-07-06T20:26:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.195+0000 7fce4910db80 1 --2- 172.21.15.69:0/1492957080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a16390 0x55bde0a16830 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.195+0000 7fce4910db80 1 --2- 172.21.15.69:0/1492957080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a14f20 0x55bde0a15330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.195+0000 7fce4910db80 1 --2- 172.21.15.69:0/1492957080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a15e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.195+0000 7fce4910db80 1 -- 172.21.15.69:0/1492957080 >> 172.21.15.69:0/1492957080 conn(0x55bde0a0a640 msgr2=0x55bde0a0f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:26:56.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.195+0000 7fce2cff9700 1 -- 172.21.15.69:0/401680301 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fce200cc440 con 0x7fce30071010 2023-07-06T20:26:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.196+0000 7fce4910db80 1 -- 172.21.15.69:0/1492957080 shutdown_connections 2023-07-06T20:26:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.196+0000 7fce4910db80 1 -- 172.21.15.69:0/1492957080 wait complete. 2023-07-06T20:26:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.196+0000 7fce2f7fe700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fce100a1150 0x7fce100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.197+0000 7fce4910db80 1 -- start start 2023-07-06T20:26:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.197+0000 7fce4910db80 1 Processor -- start 2023-07-06T20:26:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.198+0000 7fce4910db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a14f20 0x55bde0a1e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.198+0000 7fce4910db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a1edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.198+0000 7fce4910db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a16390 0x55bde0a25e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.198+0000 7fce4910db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde0905ac0 con 0x55bde0a14f20 2023-07-06T20:26:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.198+0000 7fce4910db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde09057c0 con 0x55bde0a16390 2023-07-06T20:26:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.198+0000 7fce4910db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde07e9920 con 0x55bde0a15870 2023-07-06T20:26:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.198+0000 7fce4910db80 10 client.0 _ll_register_callbacks cb 0x55bde0903760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:26:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.198+0000 7fce267fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a1edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.198+0000 7fce267fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a1edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:56208/0 (socket says 172.21.15.69:56208) 2023-07-06T20:26:56.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.198+0000 7fce267fc700 1 -- 172.21.15.69:0/3653193259 learned_addr learned my addr 172.21.15.69:0/3653193259 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:26:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.198+0000 7fce2f7fe700 1 --2- 172.21.15.69:0/3653193259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a16390 0x55bde0a25e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.199+0000 7fce267fc700 1 -- 172.21.15.69:0/3653193259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a16390 msgr2=0x55bde0a25e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.199+0000 7fce267fc700 1 --2- 172.21.15.69:0/3653193259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a16390 0x55bde0a25e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.199+0000 7fce267fc700 1 -- 172.21.15.69:0/3653193259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a14f20 msgr2=0x55bde0a1e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.199+0000 7fce267fc700 1 --2- 172.21.15.69:0/3653193259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a14f20 0x55bde0a1e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.199+0000 7fce267fc700 1 -- 172.21.15.69:0/3653193259 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce18009090 con 0x55bde0a15870 2023-07-06T20:26:56.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.200+0000 7fce267fc700 1 --2- 172.21.15.69:0/3653193259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a1edf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce18015bc0 tx=0x7fce18015bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff333d87fd503938 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:26:56.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.200+0000 7fce1e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:26:56.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.200+0000 7fce2f7fe700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fce100a1150 0x7fce100a3600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fce28009fd0 tx=0x7fce28019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:26:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.202+0000 7fce1e7fc700 1 -- 172.21.15.69:0/3653193259 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1800e040 con 0x55bde0a15870 2023-07-06T20:26:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.202+0000 7fce1e7fc700 1 -- 172.21.15.69:0/3653193259 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fce18004ce0 con 0x55bde0a15870 2023-07-06T20:26:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.202+0000 7fce4910db80 1 -- 172.21.15.69:0/3653193259 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bde08bf4f0 con 0x55bde0a15870 2023-07-06T20:26:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.203+0000 7fce1e7fc700 1 -- 172.21.15.69:0/3653193259 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce18004ea0 con 0x55bde0a15870 2023-07-06T20:26:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.204+0000 7fce4910db80 10 client.25603 fetch_fsmap learned FSMap version 30 2023-07-06T20:26:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.204+0000 7fce4910db80 1 -- 172.21.15.69:0/3653193259 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bde077cc60 con 0x55bde0a15870 2023-07-06T20:26:56.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.205+0000 7fce1e7fc700 1 -- 172.21.15.69:0/3653193259 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bde08bf4f0 con 0x55bde0a15870 2023-07-06T20:26:56.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.205+0000 7fce1e7fc700 1 -- 172.21.15.69:0/3653193259 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fce18014740 con 0x55bde0a15870 2023-07-06T20:26:56.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.206+0000 7fce4910db80 10 client.25603 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:26:56.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.206+0000 7fce4910db80 10 client.25603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:26:56.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.206+0000 7fce4910db80 20 client.25603 populate_metadata read hostname 'smithi069' 2023-07-06T20:26:56.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.206+0000 7fce4910db80 1 --2- 172.21.15.69:0/3653193259 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bde0a4c640 0x55bde0a4ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.206+0000 7fce4910db80 4 client.25603 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:26:56.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.206+0000 7fce4910db80 1 -- 172.21.15.69:0/3653193259 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bde09e3d60 con 0x55bde0a4c640 2023-07-06T20:26:56.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.206+0000 7fce2effd700 1 --2- 172.21.15.69:0/3653193259 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bde0a4c640 0x55bde0a4ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.207+0000 7fce2effd700 1 --2- 172.21.15.69:0/3653193259 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bde0a4c640 0x55bde0a4ea50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=300533e70b10ea54 server_cookie=816b23c5bc48b7e2 in_seq=0 out_seq=0 2023-07-06T20:26:56.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.207+0000 7fce1e7fc700 10 client.25603 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:26:56.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce1e7fc700 1 -- 172.21.15.69:0/3653193259 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bde09e3d60 con 0x55bde0a4c640 2023-07-06T20:26:56.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce1e7fc700 10 client.25603 handle_command_reply: tid=0 2023-07-06T20:26:56.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce4910db80 1 client.25603 shutdown 2023-07-06T20:26:56.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce4910db80 1 -- 172.21.15.69:0/3653193259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 msgr2=0x55bde0a1edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:56.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce4910db80 1 --2- 172.21.15.69:0/3653193259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a1edf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce18015bc0 tx=0x7fce18015bf0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce4910db80 1 -- 172.21.15.69:0/3653193259 shutdown_connections 2023-07-06T20:26:56.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce4910db80 1 --2- 172.21.15.69:0/3653193259 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bde0a4c640 0x55bde0a4ea50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce4910db80 1 --2- 172.21.15.69:0/3653193259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a16390 0x55bde0a25e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce1e7fc700 0 client.25603 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:26:56.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce4910db80 1 --2- 172.21.15.69:0/3653193259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a14f20 0x55bde0a1e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce4910db80 1 --2- 172.21.15.69:0/3653193259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a1edf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce4910db80 1 -- 172.21.15.69:0/3653193259 >> 172.21.15.69:0/3653193259 conn(0x55bde0a0a640 msgr2=0x55bde0a0f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:26:56.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce4910db80 1 -- 172.21.15.69:0/3653193259 shutdown_connections 2023-07-06T20:26:56.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.208+0000 7fce4910db80 1 -- 172.21.15.69:0/3653193259 wait complete. 2023-07-06T20:26:56.214 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.209+0000 7fce4910db80 20 client.25603 trim_cache size 0 max 16384 2023-07-06T20:26:56.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.238+0000 7fce4910db80 1 Processor -- start 2023-07-06T20:26:56.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.239+0000 7fce4910db80 1 -- start start 2023-07-06T20:26:56.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.241+0000 7fce4910db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a14f20 0x55bde0a1d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.241+0000 7fce4910db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a15870 0x55bde0a25df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.241+0000 7fce4910db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a16390 0x55bde0a19330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.241+0000 7fce4910db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde0a1e480 con 0x55bde0a15870 2023-07-06T20:26:56.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.241+0000 7fce4910db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde0a10d90 con 0x55bde0a14f20 2023-07-06T20:26:56.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.241+0000 7fce4910db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde0a16760 con 0x55bde0a16390 2023-07-06T20:26:56.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.241+0000 7fce2f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a16390 0x55bde0a19330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.241+0000 7fce2f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a16390 0x55bde0a19330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:56218/0 (socket says 172.21.15.69:56218) 2023-07-06T20:26:56.244 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.241+0000 7fce2f7fe700 1 -- 172.21.15.69:0/3494708882 learned_addr learned my addr 172.21.15.69:0/3494708882 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:26:56.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.241+0000 7fce267fc700 1 --2- 172.21.15.69:0/3494708882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a15870 0x55bde0a25df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.242+0000 7fce2f7fe700 1 -- 172.21.15.69:0/3494708882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a14f20 msgr2=0x55bde0a1d5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:26:56.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.242+0000 7fce2f7fe700 1 --2- 172.21.15.69:0/3494708882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a14f20 0x55bde0a1d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.242+0000 7fce2f7fe700 1 -- 172.21.15.69:0/3494708882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a15870 msgr2=0x55bde0a25df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:56.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.242+0000 7fce2f7fe700 1 --2- 172.21.15.69:0/3494708882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a15870 0x55bde0a25df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.245 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.242+0000 7fce2f7fe700 1 -- 172.21.15.69:0/3494708882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bde07ba250 con 0x55bde0a16390 2023-07-06T20:26:56.246 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.242+0000 7fce2f7fe700 1 --2- 172.21.15.69:0/3494708882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a16390 0x55bde0a19330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fce2800f550 tx=0x7fce2801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a12db04cf874b41 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:26:56.246 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.243+0000 7fce25ffb700 1 -- 172.21.15.69:0/3494708882 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2800bc00 con 0x55bde0a16390 2023-07-06T20:26:56.246 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.243+0000 7fce25ffb700 1 -- 172.21.15.69:0/3494708882 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce2801d040 con 0x55bde0a16390 2023-07-06T20:26:56.246 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.243+0000 7fce25ffb700 1 -- 172.21.15.69:0/3494708882 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce28004cf0 con 0x55bde0a16390 2023-07-06T20:26:56.246 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.244+0000 7fce4910db80 1 -- 172.21.15.69:0/3494708882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a16390 msgr2=0x55bde0a19330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:56.247 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.244+0000 7fce4910db80 1 --2- 172.21.15.69:0/3494708882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a16390 0x55bde0a19330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fce2800f550 tx=0x7fce2801f000 comp rx=0 tx=0).stop 2023-07-06T20:26:56.247 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.244+0000 7fce4910db80 1 -- 172.21.15.69:0/3494708882 shutdown_connections 2023-07-06T20:26:56.247 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.244+0000 7fce4910db80 1 --2- 172.21.15.69:0/3494708882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a16390 0x55bde0a19330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.247 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.244+0000 7fce4910db80 1 --2- 172.21.15.69:0/3494708882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a14f20 0x55bde0a1d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.244+0000 7fce4910db80 1 --2- 172.21.15.69:0/3494708882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a15870 0x55bde0a25df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.244+0000 7fce4910db80 1 -- 172.21.15.69:0/3494708882 >> 172.21.15.69:0/3494708882 conn(0x55bde0a0a640 msgr2=0x55bde0a10680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:26:56.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.245+0000 7fce4910db80 1 -- 172.21.15.69:0/3494708882 shutdown_connections 2023-07-06T20:26:56.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.245+0000 7fce4910db80 1 -- 172.21.15.69:0/3494708882 wait complete. 2023-07-06T20:26:56.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.245+0000 7fce4910db80 1 -- start start 2023-07-06T20:26:56.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.246+0000 7fce4910db80 1 Processor -- start 2023-07-06T20:26:56.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.246+0000 7fce4910db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a14f20 0x55bde0a256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.246+0000 7fce4910db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a25c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.246+0000 7fce4910db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a16390 0x55bde0a23ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.246+0000 7fce4910db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde0a24330 con 0x55bde0a16390 2023-07-06T20:26:56.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.246+0000 7fce4910db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde0a244a0 con 0x55bde0a14f20 2023-07-06T20:26:56.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.246+0000 7fce4910db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde0a24660 con 0x55bde0a15870 2023-07-06T20:26:56.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.246+0000 7fce4910db80 10 client.0 _ll_register_callbacks cb 0x7fce28009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:26:56.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.247+0000 7fce2effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a14f20 0x55bde0a256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.247+0000 7fce2effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a14f20 0x55bde0a256d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:52718/0 (socket says 172.21.15.69:52718) 2023-07-06T20:26:56.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.247+0000 7fce2effd700 1 -- 172.21.15.69:0/52846910 learned_addr learned my addr 172.21.15.69:0/52846910 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:26:56.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.247+0000 7fce267fc700 1 --2- 172.21.15.69:0/52846910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a25c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.247+0000 7fce2effd700 1 -- 172.21.15.69:0/52846910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 msgr2=0x55bde0a25c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:56.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.247+0000 7fce2effd700 1 --2- 172.21.15.69:0/52846910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a25c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.247+0000 7fce2effd700 1 -- 172.21.15.69:0/52846910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a16390 msgr2=0x55bde0a23ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:26:56.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.247+0000 7fce2effd700 1 --2- 172.21.15.69:0/52846910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a16390 0x55bde0a23ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.247+0000 7fce2effd700 1 -- 172.21.15.69:0/52846910 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce30117770 con 0x55bde0a14f20 2023-07-06T20:26:56.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.248+0000 7fce2effd700 1 --2- 172.21.15.69:0/52846910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a14f20 0x55bde0a256d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fce2000b500 tx=0x7fce20095bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1437d2ec69125441 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:26:56.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.248+0000 7fce1e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:26:56.253 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce1e7fc700 1 -- 172.21.15.69:0/52846910 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20099180 con 0x55bde0a14f20 2023-07-06T20:26:56.253 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce1e7fc700 1 -- 172.21.15.69:0/52846910 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fce20010380 con 0x55bde0a14f20 2023-07-06T20:26:56.253 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce4910db80 1 -- 172.21.15.69:0/52846910 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce1800f040 con 0x55bde0a14f20 2023-07-06T20:26:56.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce1e7fc700 1 -- 172.21.15.69:0/52846910 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce20099820 con 0x55bde0a14f20 2023-07-06T20:26:56.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce4910db80 10 client.25658 fetch_fsmap learned FSMap version 30 2023-07-06T20:26:56.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce4910db80 1 -- 172.21.15.69:0/52846910 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bde07922b0 con 0x55bde0a14f20 2023-07-06T20:26:56.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce1e7fc700 1 -- 172.21.15.69:0/52846910 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce1800f040 con 0x55bde0a14f20 2023-07-06T20:26:56.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce1e7fc700 1 -- 172.21.15.69:0/52846910 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fce20099a80 con 0x55bde0a14f20 2023-07-06T20:26:56.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce4910db80 10 client.25658 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:26:56.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce4910db80 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:26:56.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce4910db80 20 client.25658 populate_metadata read hostname 'smithi069' 2023-07-06T20:26:56.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce4910db80 1 --2- 172.21.15.69:0/52846910 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bde0aa17e0 0x55bde0aa3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:26:56.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce4910db80 4 client.25658 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:26:56.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.249+0000 7fce4910db80 1 -- 172.21.15.69:0/52846910 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bde095eeb0 con 0x55bde0aa17e0 2023-07-06T20:26:56.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.250+0000 7fce267fc700 1 --2- 172.21.15.69:0/52846910 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bde0aa17e0 0x55bde0aa3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:26:56.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.250+0000 7fce1e7fc700 10 client.25658 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:26:56.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.250+0000 7fce267fc700 1 --2- 172.21.15.69:0/52846910 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bde0aa17e0 0x55bde0aa3bf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a4759709b8f9bf server_cookie=c65572228e1e0167 in_seq=0 out_seq=0 2023-07-06T20:26:56.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce1e7fc700 1 -- 172.21.15.69:0/52846910 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55bde095eeb0 con 0x55bde0aa17e0 2023-07-06T20:26:56.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce1e7fc700 10 client.25658 handle_command_reply: tid=0 2023-07-06T20:26:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce4910db80 1 client.25658 shutdown 2023-07-06T20:26:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce4910db80 1 -- 172.21.15.69:0/52846910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a14f20 msgr2=0x55bde0a256d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce4910db80 1 --2- 172.21.15.69:0/52846910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a14f20 0x55bde0a256d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fce2000b500 tx=0x7fce20095bc0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce4910db80 1 -- 172.21.15.69:0/52846910 shutdown_connections 2023-07-06T20:26:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce4910db80 1 --2- 172.21.15.69:0/52846910 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55bde0aa17e0 0x55bde0aa3bf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce4910db80 1 --2- 172.21.15.69:0/52846910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bde0a16390 0x55bde0a23ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce4910db80 1 --2- 172.21.15.69:0/52846910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bde0a14f20 0x55bde0a256d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce4910db80 1 --2- 172.21.15.69:0/52846910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde0a15870 0x55bde0a25c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce4910db80 1 -- 172.21.15.69:0/52846910 >> 172.21.15.69:0/52846910 conn(0x55bde0a0a640 msgr2=0x55bde0a19330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:26:56.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce1e7fc700 0 client.25658 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:26:56.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce4910db80 1 -- 172.21.15.69:0/52846910 shutdown_connections 2023-07-06T20:26:56.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.251+0000 7fce4910db80 1 -- 172.21.15.69:0/52846910 wait complete. 2023-07-06T20:26:56.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.252+0000 7fce4910db80 20 client.25658 trim_cache size 0 max 16384 2023-07-06T20:26:56.258 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:26:56.258 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:26:56.258 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:26:56.258 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:26:56.259 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:26:56.259 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:26:56.259 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:26:56.259 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:26:56.259 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:26:56.259 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:26:56.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.252+0000 7fce357ff700 1 -- 172.21.15.69:0/401680301 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fce100a1150 msgr2=0x7fce100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:56.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.252+0000 7fce357ff700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fce100a1150 0x7fce100a3600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fce28009fd0 tx=0x7fce28019040 comp rx=0 tx=0).stop 2023-07-06T20:26:56.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.252+0000 7fce357ff700 1 -- 172.21.15.69:0/401680301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce30071010 msgr2=0x7fce301a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:26:56.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.252+0000 7fce357ff700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce30071010 0x7fce301a47d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce2000b530 tx=0x7fce2000b8f0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.260 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.252+0000 7fce357ff700 1 -- 172.21.15.69:0/401680301 shutdown_connections 2023-07-06T20:26:56.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.252+0000 7fce357ff700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fce100a1150 0x7fce100a3600 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.252+0000 7fce357ff700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce3010be90 0x7fce301abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.252+0000 7fce357ff700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce30071010 0x7fce301a47d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.252+0000 7fce357ff700 1 --2- 172.21.15.69:0/401680301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce30071960 0x7fce301a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:26:56.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.252+0000 7fce357ff700 1 -- 172.21.15.69:0/401680301 >> 172.21.15.69:0/401680301 conn(0x7fce3006cbc0 msgr2=0x7fce3006de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:26:56.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.253+0000 7fce357ff700 1 -- 172.21.15.69:0/401680301 shutdown_connections 2023-07-06T20:26:56.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:26:56.253+0000 7fce357ff700 1 -- 172.21.15.69:0/401680301 wait complete. 2023-07-06T20:26:56.308 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:26:57.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:57 smithi069 ceph-mon[126399]: pgmap v556: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 56 MiB/s wr, 3.58k op/s 2023-07-06T20:26:57.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:57 smithi156 ceph-mon[129280]: pgmap v556: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 56 MiB/s wr, 3.58k op/s 2023-07-06T20:26:57.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:57 smithi098 ceph-mon[128968]: pgmap v556: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 56 MiB/s wr, 3.58k op/s 2023-07-06T20:26:59.232 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 310.0 s, 2931.6 tps, lat 33.945 ms stddev 19.997 2023-07-06T20:26:59.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:26:59 smithi069 ceph-mon[126399]: pgmap v557: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 52 MiB/s wr, 3.72k op/s 2023-07-06T20:26:59.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:26:59 smithi156 ceph-mon[129280]: pgmap v557: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 52 MiB/s wr, 3.72k op/s 2023-07-06T20:26:59.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:26:59 smithi098 ceph-mon[128968]: pgmap v557: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 52 MiB/s wr, 3.72k op/s 2023-07-06T20:27:01.519 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:01 smithi069 ceph-mon[126399]: pgmap v558: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 59 MiB/s wr, 3.90k op/s 2023-07-06T20:27:01.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:01 smithi156 ceph-mon[129280]: pgmap v558: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 59 MiB/s wr, 3.90k op/s 2023-07-06T20:27:01.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:01 smithi098 ceph-mon[128968]: pgmap v558: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 59 MiB/s wr, 3.90k op/s 2023-07-06T20:27:03.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:03 smithi156 ceph-mon[129280]: pgmap v559: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 50 MiB/s wr, 3.73k op/s 2023-07-06T20:27:03.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:03 smithi098 ceph-mon[128968]: pgmap v559: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 50 MiB/s wr, 3.73k op/s 2023-07-06T20:27:03.764 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:03 smithi069 ceph-mon[126399]: pgmap v559: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 50 MiB/s wr, 3.73k op/s 2023-07-06T20:27:04.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 315.0 s, 2345.3 tps, lat 43.104 ms stddev 47.188 2023-07-06T20:27:05.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:05 smithi156 ceph-mon[129280]: pgmap v560: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 56 MiB/s wr, 3.82k op/s 2023-07-06T20:27:05.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:05 smithi098 ceph-mon[128968]: pgmap v560: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 56 MiB/s wr, 3.82k op/s 2023-07-06T20:27:05.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:05 smithi069 ceph-mon[126399]: pgmap v560: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 56 MiB/s wr, 3.82k op/s 2023-07-06T20:27:07.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:07 smithi156 ceph-mon[129280]: pgmap v561: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 45 MiB/s wr, 2.45k op/s 2023-07-06T20:27:07.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:07 smithi098 ceph-mon[128968]: pgmap v561: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 45 MiB/s wr, 2.45k op/s 2023-07-06T20:27:07.764 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:07 smithi069 ceph-mon[126399]: pgmap v561: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 45 MiB/s wr, 2.45k op/s 2023-07-06T20:27:09.239 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 320.0 s, 1749.5 tps, lat 55.358 ms stddev 104.073 2023-07-06T20:27:09.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:09 smithi069 ceph-mon[126399]: pgmap v562: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.35k op/s 2023-07-06T20:27:09.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:09 smithi156 ceph-mon[129280]: pgmap v562: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.35k op/s 2023-07-06T20:27:09.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:09 smithi098 ceph-mon[128968]: pgmap v562: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.35k op/s 2023-07-06T20:27:11.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:11 smithi069 ceph-mon[126399]: pgmap v563: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 49 MiB/s wr, 2.44k op/s 2023-07-06T20:27:11.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:11 smithi156 ceph-mon[129280]: pgmap v563: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 49 MiB/s wr, 2.44k op/s 2023-07-06T20:27:11.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:11 smithi098 ceph-mon[128968]: pgmap v563: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 49 MiB/s wr, 2.44k op/s 2023-07-06T20:27:13.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:13 smithi069 ceph-mon[126399]: pgmap v564: 97 pgs: 97 active+clean; 8.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 MiB/s wr, 2.69k op/s 2023-07-06T20:27:13.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:13 smithi156 ceph-mon[129280]: pgmap v564: 97 pgs: 97 active+clean; 8.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 MiB/s wr, 2.69k op/s 2023-07-06T20:27:13.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:13 smithi098 ceph-mon[128968]: pgmap v564: 97 pgs: 97 active+clean; 8.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 MiB/s wr, 2.69k op/s 2023-07-06T20:27:14.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 325.0 s, 2180.5 tps, lat 47.203 ms stddev 48.734 2023-07-06T20:27:15.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:15 smithi069 ceph-mon[126399]: pgmap v565: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 MiB/s wr, 3.70k op/s 2023-07-06T20:27:15.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:15 smithi156 ceph-mon[129280]: pgmap v565: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 MiB/s wr, 3.70k op/s 2023-07-06T20:27:15.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:15 smithi098 ceph-mon[128968]: pgmap v565: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 MiB/s wr, 3.70k op/s 2023-07-06T20:27:17.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:17 smithi069 ceph-mon[126399]: pgmap v566: 97 pgs: 97 active+clean; 8.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 MiB/s wr, 3.79k op/s 2023-07-06T20:27:17.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:17 smithi156 ceph-mon[129280]: pgmap v566: 97 pgs: 97 active+clean; 8.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 MiB/s wr, 3.79k op/s 2023-07-06T20:27:17.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:17 smithi098 ceph-mon[128968]: pgmap v566: 97 pgs: 97 active+clean; 8.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 MiB/s wr, 3.79k op/s 2023-07-06T20:27:19.237 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 330.0 s, 2269.2 tps, lat 44.170 ms stddev 59.862 2023-07-06T20:27:19.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:19 smithi156 ceph-mon[129280]: pgmap v567: 97 pgs: 97 active+clean; 8.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 51 MiB/s wr, 3.65k op/s 2023-07-06T20:27:19.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:19 smithi098 ceph-mon[128968]: pgmap v567: 97 pgs: 97 active+clean; 8.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 51 MiB/s wr, 3.65k op/s 2023-07-06T20:27:20.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:19 smithi069 ceph-mon[126399]: pgmap v567: 97 pgs: 97 active+clean; 8.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 51 MiB/s wr, 3.65k op/s 2023-07-06T20:27:21.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:21 smithi156 ceph-mon[129280]: pgmap v568: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 MiB/s wr, 3.52k op/s 2023-07-06T20:27:21.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:21 smithi098 ceph-mon[128968]: pgmap v568: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 MiB/s wr, 3.52k op/s 2023-07-06T20:27:22.015 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:21 smithi069 ceph-mon[126399]: pgmap v568: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 MiB/s wr, 3.52k op/s 2023-07-06T20:27:24.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:23 smithi069 ceph-mon[126399]: pgmap v569: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 MiB/s wr, 2.22k op/s 2023-07-06T20:27:24.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:23 smithi156 ceph-mon[129280]: pgmap v569: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 MiB/s wr, 2.22k op/s 2023-07-06T20:27:24.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:23 smithi098 ceph-mon[128968]: pgmap v569: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 MiB/s wr, 2.22k op/s 2023-07-06T20:27:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 335.0 s, 2671.5 tps, lat 37.465 ms stddev 23.196 2023-07-06T20:27:26.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:25 smithi069 ceph-mon[126399]: pgmap v570: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 1.88k op/s 2023-07-06T20:27:26.114 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:25 smithi156 ceph-mon[129280]: pgmap v570: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 1.88k op/s 2023-07-06T20:27:26.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:25 smithi098 ceph-mon[128968]: pgmap v570: 97 pgs: 97 active+clean; 8.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 MiB/s wr, 1.88k op/s 2023-07-06T20:27:26.309 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:27:26.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.753+0000 7f11fdfff700 1 -- 172.21.15.69:0/2239747450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8077340 msgr2=0x7f11f80797c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:26.757 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.753+0000 7f11fdfff700 1 --2- 172.21.15.69:0/2239747450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8077340 0x7f11f80797c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f11f000b1a0 tx=0x7f11f000b4b0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.758 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.756+0000 7f11fdfff700 1 -- 172.21.15.69:0/2239747450 shutdown_connections 2023-07-06T20:27:26.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.756+0000 7f11fdfff700 1 --2- 172.21.15.69:0/2239747450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8077340 0x7f11f80797c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.756+0000 7f11fdfff700 1 --2- 172.21.15.69:0/2239747450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11f8071330 0x7f11f8071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.756+0000 7f11fdfff700 1 --2- 172.21.15.69:0/2239747450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11f8074980 0x7f11f8076e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.759 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.756+0000 7f11fdfff700 1 -- 172.21.15.69:0/2239747450 >> 172.21.15.69:0/2239747450 conn(0x7f11f806cbc0 msgr2=0x7f11f806eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:27:26.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.766+0000 7f11fdfff700 1 -- 172.21.15.69:0/2239747450 shutdown_connections 2023-07-06T20:27:26.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.766+0000 7f11fdfff700 1 -- 172.21.15.69:0/2239747450 wait complete. 2023-07-06T20:27:26.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.767+0000 7f11fdfff700 1 Processor -- start 2023-07-06T20:27:26.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.767+0000 7f11fdfff700 1 -- start start 2023-07-06T20:27:26.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.768+0000 7f11fdfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11f8071330 0x7f11f8082400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.768+0000 7f11fdfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8074980 0x7f11f8082940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.768+0000 7f11fdfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11f8077340 0x7f11f81b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.768+0000 7f11fdfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81b4a90 con 0x7f11f8071330 2023-07-06T20:27:26.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.768+0000 7f11fdfff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81b4bd0 con 0x7f11f8077340 2023-07-06T20:27:26.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.768+0000 7f11fdfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81b4d90 con 0x7f11f8074980 2023-07-06T20:27:26.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.768+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11f8071330 0x7f11f8082400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.768+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11f8071330 0x7f11f8082400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57832/0 (socket says 172.21.15.69:57832) 2023-07-06T20:27:26.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.768+0000 7f11f77fe700 1 -- 172.21.15.69:0/78814484 learned_addr learned my addr 172.21.15.69:0/78814484 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:27:26.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.775+0000 7f11f6ffd700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8074980 0x7f11f8082940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.775+0000 7f11f77fe700 1 -- 172.21.15.69:0/78814484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8074980 msgr2=0x7f11f8082940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:26.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.775+0000 7f11f77fe700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8074980 0x7f11f8082940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.776+0000 7f11f77fe700 1 -- 172.21.15.69:0/78814484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11f8077340 msgr2=0x7f11f81b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:26.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.776+0000 7f11f77fe700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11f8077340 0x7f11f81b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.776+0000 7f11f77fe700 1 -- 172.21.15.69:0/78814484 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f0009d00 con 0x7f11f8071330 2023-07-06T20:27:26.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.782+0000 7f11f77fe700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11f8071330 0x7f11f8082400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f11e80078a0 tx=0x7f11e8007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:27:26.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.795+0000 7f11f4ff9700 1 -- 172.21.15.69:0/78814484 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8007ea0 con 0x7f11f8071330 2023-07-06T20:27:26.808 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.795+0000 7f11f4ff9700 1 -- 172.21.15.69:0/78814484 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11e800de80 con 0x7f11f8071330 2023-07-06T20:27:26.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.795+0000 7f11f4ff9700 1 -- 172.21.15.69:0/78814484 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e801a6a0 con 0x7f11f8071330 2023-07-06T20:27:26.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.796+0000 7f11fdfff700 1 -- 172.21.15.69:0/78814484 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f81b5070 con 0x7f11f8071330 2023-07-06T20:27:26.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.796+0000 7f11fdfff700 1 -- 172.21.15.69:0/78814484 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f81bdcb0 con 0x7f11f8071330 2023-07-06T20:27:26.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.802+0000 7f1211994b80 1 Processor -- start 2023-07-06T20:27:26.809 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.803+0000 7f1211994b80 1 -- start start 2023-07-06T20:27:26.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.803+0000 7f1211994b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d22f30 0x557c98d23340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.803+0000 7f1211994b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d23880 0x557c98d23e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.803+0000 7f1211994b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d243a0 0x557c98d24840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.803+0000 7f1211994b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c98c16cb0 con 0x557c98d22f30 2023-07-06T20:27:26.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.803+0000 7f1211994b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c98c69a40 con 0x557c98d23880 2023-07-06T20:27:26.811 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.803+0000 7f1211994b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c98c62ed0 con 0x557c98d243a0 2023-07-06T20:27:26.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.811+0000 7f11f6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d243a0 0x557c98d24840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.813+0000 7f11f7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d22f30 0x557c98d23340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.816 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.813+0000 7f11f7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d22f30 0x557c98d23340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57846/0 (socket says 172.21.15.69:57846) 2023-07-06T20:27:26.817 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.813+0000 7f11f7fff700 1 -- 172.21.15.69:0/1976958967 learned_addr learned my addr 172.21.15.69:0/1976958967 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:27:26.821 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.816+0000 7f11f4ff9700 1 -- 172.21.15.69:0/78814484 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f11e80127d0 con 0x7f11f8071330 2023-07-06T20:27:26.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.818+0000 7f11f4ff9700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f11e00a1350 0x7f11e00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.822 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.819+0000 7f11f4ff9700 1 -- 172.21.15.69:0/78814484 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f11e80cbe00 con 0x7f11f8071330 2023-07-06T20:27:26.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.825+0000 7f11f7fff700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f11e00a1350 0x7f11e00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.825+0000 7f11f77fe700 1 --2- 172.21.15.69:0/1976958967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d23880 0x557c98d23e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.833+0000 7f11f7fff700 1 -- 172.21.15.69:0/1976958967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d243a0 msgr2=0x557c98d24840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:26.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.833+0000 7f11f7fff700 1 --2- 172.21.15.69:0/1976958967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d243a0 0x557c98d24840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.833+0000 7f11f7fff700 1 -- 172.21.15.69:0/1976958967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d23880 msgr2=0x557c98d23e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:26.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.833+0000 7f11f7fff700 1 --2- 172.21.15.69:0/1976958967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d23880 0x557c98d23e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.833+0000 7f11f7fff700 1 -- 172.21.15.69:0/1976958967 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c98a90de0 con 0x557c98d22f30 2023-07-06T20:27:26.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.834+0000 7f11f7fff700 1 --2- 172.21.15.69:0/1976958967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d22f30 0x557c98d23340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f11f00062a0 tx=0x7f11f0008d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70040b7e735e49 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:27:26.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.835+0000 7f11f7fff700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f11e00a1350 0x7f11e00a3800 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f11f000b8b0 tx=0x7f11f0026000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:27:26.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.844+0000 7f11ddffb700 1 -- 172.21.15.69:0/1976958967 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0007bc0 con 0x557c98d22f30 2023-07-06T20:27:26.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.844+0000 7f11ddffb700 1 -- 172.21.15.69:0/1976958967 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11f0021070 con 0x557c98d22f30 2023-07-06T20:27:26.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.844+0000 7f11ddffb700 1 -- 172.21.15.69:0/1976958967 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0027440 con 0x557c98d22f30 2023-07-06T20:27:26.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.845+0000 7f1211994b80 1 -- 172.21.15.69:0/1976958967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d22f30 msgr2=0x557c98d23340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:26.849 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.845+0000 7f1211994b80 1 --2- 172.21.15.69:0/1976958967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d22f30 0x557c98d23340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f11f00062a0 tx=0x7f11f0008d90 comp rx=0 tx=0).stop 2023-07-06T20:27:26.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.846+0000 7f1211994b80 1 -- 172.21.15.69:0/1976958967 shutdown_connections 2023-07-06T20:27:26.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.846+0000 7f1211994b80 1 --2- 172.21.15.69:0/1976958967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d243a0 0x557c98d24840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.846+0000 7f1211994b80 1 --2- 172.21.15.69:0/1976958967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d22f30 0x557c98d23340 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.850 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.846+0000 7f1211994b80 1 --2- 172.21.15.69:0/1976958967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d23880 0x557c98d23e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.846+0000 7f1211994b80 1 -- 172.21.15.69:0/1976958967 >> 172.21.15.69:0/1976958967 conn(0x557c98d07640 msgr2=0x557c98d0c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:27:26.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.846+0000 7f1211994b80 1 -- 172.21.15.69:0/1976958967 shutdown_connections 2023-07-06T20:27:26.851 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.846+0000 7f1211994b80 1 -- 172.21.15.69:0/1976958967 wait complete. 2023-07-06T20:27:26.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.847+0000 7f1211994b80 1 -- start start 2023-07-06T20:27:26.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.847+0000 7f1211994b80 1 Processor -- start 2023-07-06T20:27:26.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.848+0000 7f1211994b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 0x557c98d2c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.848+0000 7f1211994b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d23880 0x557c98d2ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.848+0000 7f1211994b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d243a0 0x557c98d33e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.848+0000 7f1211994b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c98c02ac0 con 0x557c98d23880 2023-07-06T20:27:26.852 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.848+0000 7f1211994b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c98c027c0 con 0x557c98d243a0 2023-07-06T20:27:26.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.848+0000 7f1211994b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c98ae6920 con 0x557c98d22f30 2023-07-06T20:27:26.853 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.848+0000 7f1211994b80 10 client.0 _ll_register_callbacks cb 0x557c98c00760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:27:26.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.853+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 0x557c98d2c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.853+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 0x557c98d2c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43108/0 (socket says 172.21.15.69:43108) 2023-07-06T20:27:26.855 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.853+0000 7f11f77fe700 1 -- 172.21.15.69:0/1487697495 learned_addr learned my addr 172.21.15.69:0/1487697495 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:27:26.858 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.856+0000 7f11f7fff700 1 --2- 172.21.15.69:0/1487697495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d243a0 0x557c98d33e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.856+0000 7f11f7fff700 1 -- 172.21.15.69:0/1487697495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 msgr2=0x557c98d2c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:26.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.856+0000 7f11f7fff700 1 --2- 172.21.15.69:0/1487697495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 0x557c98d2c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.856+0000 7f11f7fff700 1 -- 172.21.15.69:0/1487697495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d23880 msgr2=0x557c98d2ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:26.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.856+0000 7f11f7fff700 1 --2- 172.21.15.69:0/1487697495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d23880 0x557c98d2ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.859 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.856+0000 7f11f7fff700 1 -- 172.21.15.69:0/1487697495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f807a1f0 con 0x557c98d243a0 2023-07-06T20:27:26.862 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.860+0000 7f11f7fff700 1 --2- 172.21.15.69:0/1487697495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d243a0 0x557c98d33e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009610 tx=0x7f11f00118f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9666a82b81bd44e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:27:26.867 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.863+0000 7f11d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:27:26.869 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.867+0000 7f11d6ffd700 1 -- 172.21.15.69:0/1487697495 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0027640 con 0x557c98d243a0 2023-07-06T20:27:26.869 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.867+0000 7f11d6ffd700 1 -- 172.21.15.69:0/1487697495 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f11f00277a0 con 0x557c98d243a0 2023-07-06T20:27:26.870 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.867+0000 7f1211994b80 1 -- 172.21.15.69:0/1487697495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c98bbc4f0 con 0x557c98d243a0 2023-07-06T20:27:26.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.874+0000 7f11d6ffd700 1 -- 172.21.15.69:0/1487697495 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11f0027960 con 0x557c98d243a0 2023-07-06T20:27:26.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.874+0000 7f1211994b80 10 client.25667 fetch_fsmap learned FSMap version 30 2023-07-06T20:27:26.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.874+0000 7f1211994b80 1 -- 172.21.15.69:0/1487697495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557c98a79c60 con 0x557c98d243a0 2023-07-06T20:27:26.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.876+0000 7f11d6ffd700 1 -- 172.21.15.69:0/1487697495 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c98bbc4f0 con 0x557c98d243a0 2023-07-06T20:27:26.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.876+0000 7f11d6ffd700 1 -- 172.21.15.69:0/1487697495 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f11f00057e0 con 0x557c98d243a0 2023-07-06T20:27:26.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.876+0000 7f1211994b80 10 client.25667 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:27:26.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.876+0000 7f1211994b80 10 client.25667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:27:26.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.876+0000 7f1211994b80 20 client.25667 populate_metadata read hostname 'smithi069' 2023-07-06T20:27:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.876+0000 7f1211994b80 1 --2- 172.21.15.69:0/1487697495 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x557c98d38170 0x557c98d3a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.876+0000 7f1211994b80 4 client.25667 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:27:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.876+0000 7f1211994b80 1 -- 172.21.15.69:0/1487697495 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c98ce0d60 con 0x557c98d38170 2023-07-06T20:27:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.879+0000 7f11f77fe700 1 --2- 172.21.15.69:0/1487697495 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x557c98d38170 0x557c98d3a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.880+0000 7f11f77fe700 1 --2- 172.21.15.69:0/1487697495 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x557c98d38170 0x557c98d3a550 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3152f5ddb60342e server_cookie=47397d1401a0b370 in_seq=0 out_seq=0 2023-07-06T20:27:26.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.880+0000 7f11d6ffd700 10 client.25667 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:27:26.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.881+0000 7f11d6ffd700 1 -- 172.21.15.69:0/1487697495 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c98ce0d60 con 0x557c98d38170 2023-07-06T20:27:26.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.881+0000 7f11d6ffd700 10 client.25667 handle_command_reply: tid=0 2023-07-06T20:27:26.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.881+0000 7f1211994b80 1 client.25667 shutdown 2023-07-06T20:27:26.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.883+0000 7f1211994b80 1 -- 172.21.15.69:0/1487697495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d243a0 msgr2=0x557c98d33e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:26.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.883+0000 7f1211994b80 1 --2- 172.21.15.69:0/1487697495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d243a0 0x557c98d33e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009610 tx=0x7f11f00118f0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.883+0000 7f1211994b80 1 -- 172.21.15.69:0/1487697495 shutdown_connections 2023-07-06T20:27:26.892 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.883+0000 7f1211994b80 1 --2- 172.21.15.69:0/1487697495 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x557c98d38170 0x557c98d3a550 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.892 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.883+0000 7f1211994b80 1 --2- 172.21.15.69:0/1487697495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d243a0 0x557c98d33e60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.893 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.883+0000 7f1211994b80 1 --2- 172.21.15.69:0/1487697495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 0x557c98d2c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.893 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.883+0000 7f1211994b80 1 --2- 172.21.15.69:0/1487697495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d23880 0x557c98d2ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.893 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.883+0000 7f1211994b80 1 -- 172.21.15.69:0/1487697495 >> 172.21.15.69:0/1487697495 conn(0x557c98d07640 msgr2=0x557c98d0c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:27:26.893 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.883+0000 7f11d6ffd700 0 client.25667 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:27:26.894 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.884+0000 7f1211994b80 1 -- 172.21.15.69:0/1487697495 shutdown_connections 2023-07-06T20:27:26.894 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.884+0000 7f1211994b80 1 -- 172.21.15.69:0/1487697495 wait complete. 2023-07-06T20:27:26.894 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.884+0000 7f1211994b80 20 client.25667 trim_cache size 0 max 16384 2023-07-06T20:27:26.926 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.923+0000 7f1211994b80 1 Processor -- start 2023-07-06T20:27:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.923+0000 7f1211994b80 1 -- start start 2023-07-06T20:27:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.924+0000 7f1211994b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d22f30 0x557c98d2b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.924+0000 7f1211994b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d23880 0x557c98d33e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.924+0000 7f1211994b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d243a0 0x557c98d346f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.924+0000 7f1211994b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c98d2dad0 con 0x557c98d243a0 2023-07-06T20:27:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.924+0000 7f1211994b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c98d0dc20 con 0x557c98d22f30 2023-07-06T20:27:26.927 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.924+0000 7f1211994b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c98d2bde0 con 0x557c98d23880 2023-07-06T20:27:26.928 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.924+0000 7f11f6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d23880 0x557c98d33e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.924+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d22f30 0x557c98d2b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.925+0000 7f11f6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d23880 0x557c98d33e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43120/0 (socket says 172.21.15.69:43120) 2023-07-06T20:27:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.925+0000 7f11f6ffd700 1 -- 172.21.15.69:0/1083592126 learned_addr learned my addr 172.21.15.69:0/1083592126 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:27:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.925+0000 7f11f7fff700 1 --2- 172.21.15.69:0/1083592126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d243a0 0x557c98d346f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.925+0000 7f11f6ffd700 1 -- 172.21.15.69:0/1083592126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d22f30 msgr2=0x557c98d2b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.925+0000 7f11f6ffd700 1 --2- 172.21.15.69:0/1083592126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d22f30 0x557c98d2b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.925+0000 7f11f6ffd700 1 -- 172.21.15.69:0/1083592126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d243a0 msgr2=0x557c98d346f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:26.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.925+0000 7f11f6ffd700 1 --2- 172.21.15.69:0/1083592126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d243a0 0x557c98d346f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.925+0000 7f11f6ffd700 1 -- 172.21.15.69:0/1083592126 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c98ab7250 con 0x557c98d23880 2023-07-06T20:27:26.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.926+0000 7f11f6ffd700 1 --2- 172.21.15.69:0/1083592126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d23880 0x557c98d33e20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00bcd0 tx=0x7f11ec012450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d39f5a083725c985 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:27:26.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.927+0000 7f11de7fc700 1 -- 172.21.15.69:0/1083592126 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec012c50 con 0x557c98d23880 2023-07-06T20:27:26.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.927+0000 7f11de7fc700 1 -- 172.21.15.69:0/1083592126 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ec0044f0 con 0x557c98d23880 2023-07-06T20:27:26.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.927+0000 7f11de7fc700 1 -- 172.21.15.69:0/1083592126 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec01b440 con 0x557c98d23880 2023-07-06T20:27:26.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.928+0000 7f1211994b80 1 -- 172.21.15.69:0/1083592126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d23880 msgr2=0x557c98d33e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:26.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.928+0000 7f1211994b80 1 --2- 172.21.15.69:0/1083592126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d23880 0x557c98d33e20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00bcd0 tx=0x7f11ec012450 comp rx=0 tx=0).stop 2023-07-06T20:27:26.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.928+0000 7f1211994b80 1 -- 172.21.15.69:0/1083592126 shutdown_connections 2023-07-06T20:27:26.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.928+0000 7f1211994b80 1 --2- 172.21.15.69:0/1083592126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d243a0 0x557c98d346f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.928+0000 7f1211994b80 1 --2- 172.21.15.69:0/1083592126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d22f30 0x557c98d2b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.928+0000 7f1211994b80 1 --2- 172.21.15.69:0/1083592126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d23880 0x557c98d33e20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.928+0000 7f1211994b80 1 -- 172.21.15.69:0/1083592126 >> 172.21.15.69:0/1083592126 conn(0x557c98d07640 msgr2=0x557c98d3ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:27:26.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.928+0000 7f1211994b80 1 -- 172.21.15.69:0/1083592126 shutdown_connections 2023-07-06T20:27:26.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.928+0000 7f1211994b80 1 -- 172.21.15.69:0/1083592126 wait complete. 2023-07-06T20:27:26.933 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.928+0000 7f1211994b80 1 -- start start 2023-07-06T20:27:26.933 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.929+0000 7f1211994b80 1 Processor -- start 2023-07-06T20:27:26.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.929+0000 7f1211994b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 0x557c98d33900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.929+0000 7f1211994b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d23880 0x557c98d33e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.929+0000 7f1211994b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d243a0 0x557c98d30490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.929+0000 7f1211994b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c98d30ba0 con 0x557c98d243a0 2023-07-06T20:27:26.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.929+0000 7f1211994b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c98d30d10 con 0x557c98d23880 2023-07-06T20:27:26.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.929+0000 7f1211994b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c98d30ed0 con 0x557c98d22f30 2023-07-06T20:27:26.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.929+0000 7f1211994b80 10 client.0 _ll_register_callbacks cb 0x7f11e8012ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:27:26.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.930+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 0x557c98d33900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.930+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 0x557c98d33900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43126/0 (socket says 172.21.15.69:43126) 2023-07-06T20:27:26.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.930+0000 7f11f77fe700 1 -- 172.21.15.69:0/3637371614 learned_addr learned my addr 172.21.15.69:0/3637371614 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:27:26.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.930+0000 7f11f7fff700 1 --2- 172.21.15.69:0/3637371614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d243a0 0x557c98d30490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.930+0000 7f11f77fe700 1 -- 172.21.15.69:0/3637371614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d23880 msgr2=0x557c98d33e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:27:26.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.930+0000 7f11f77fe700 1 --2- 172.21.15.69:0/3637371614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d23880 0x557c98d33e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.930+0000 7f11f77fe700 1 -- 172.21.15.69:0/3637371614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d243a0 msgr2=0x557c98d30490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:26.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.930+0000 7f11f77fe700 1 --2- 172.21.15.69:0/3637371614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d243a0 0x557c98d30490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.930+0000 7f11f77fe700 1 -- 172.21.15.69:0/3637371614 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ec00b9b0 con 0x557c98d22f30 2023-07-06T20:27:26.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.931+0000 7f11f77fe700 1 --2- 172.21.15.69:0/3637371614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 0x557c98d33900 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f11e8099110 tx=0x7f11e80cb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae9ad7e3b53e928d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:27:26.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.931+0000 7f11d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:27:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.931+0000 7f11d6ffd700 1 -- 172.21.15.69:0/3637371614 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800f020 con 0x557c98d22f30 2023-07-06T20:27:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.931+0000 7f11d6ffd700 1 -- 172.21.15.69:0/3637371614 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f11e80b1070 con 0x557c98d22f30 2023-07-06T20:27:26.937 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.931+0000 7f1211994b80 1 -- 172.21.15.69:0/3637371614 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11f0021070 con 0x557c98d22f30 2023-07-06T20:27:26.938 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.932+0000 7f11d6ffd700 1 -- 172.21.15.69:0/3637371614 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11e80993e0 con 0x557c98d22f30 2023-07-06T20:27:26.938 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.932+0000 7f1211994b80 10 client.25627 fetch_fsmap learned FSMap version 30 2023-07-06T20:27:26.938 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.932+0000 7f1211994b80 1 -- 172.21.15.69:0/3637371614 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557c98a8f2b0 con 0x557c98d22f30 2023-07-06T20:27:26.938 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.932+0000 7f11d6ffd700 1 -- 172.21.15.69:0/3637371614 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11e80bc020 con 0x557c98d22f30 2023-07-06T20:27:26.938 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.932+0000 7f11d6ffd700 1 -- 172.21.15.69:0/3637371614 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f11e80a1140 con 0x557c98d22f30 2023-07-06T20:27:26.938 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.933+0000 7f1211994b80 10 client.25627 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:27:26.938 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.933+0000 7f1211994b80 10 client.25627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:27:26.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.933+0000 7f1211994b80 20 client.25627 populate_metadata read hostname 'smithi069' 2023-07-06T20:27:26.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.933+0000 7f1211994b80 1 --2- 172.21.15.69:0/3637371614 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x557c98d8d720 0x557c98d8fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:26.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.933+0000 7f1211994b80 4 client.25627 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:27:26.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.933+0000 7f1211994b80 1 -- 172.21.15.69:0/3637371614 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557c98c5beb0 con 0x557c98d8d720 2023-07-06T20:27:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.941+0000 7f11f6ffd700 1 --2- 172.21.15.69:0/3637371614 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x557c98d8d720 0x557c98d8fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:26.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.943+0000 7f11f6ffd700 1 --2- 172.21.15.69:0/3637371614 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x557c98d8d720 0x557c98d8fb30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=422216920f6886a3 server_cookie=605121d2dbc42963 in_seq=0 out_seq=0 2023-07-06T20:27:26.946 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.943+0000 7f11d6ffd700 10 client.25627 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:27:26.946 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.944+0000 7f11d6ffd700 1 -- 172.21.15.69:0/3637371614 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557c98c5beb0 con 0x557c98d8d720 2023-07-06T20:27:26.946 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.944+0000 7f11d6ffd700 10 client.25627 handle_command_reply: tid=0 2023-07-06T20:27:26.946 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.944+0000 7f1211994b80 1 client.25627 shutdown 2023-07-06T20:27:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.944+0000 7f1211994b80 1 -- 172.21.15.69:0/3637371614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 msgr2=0x557c98d33900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.944+0000 7f1211994b80 1 --2- 172.21.15.69:0/3637371614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 0x557c98d33900 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f11e8099110 tx=0x7f11e80cb040 comp rx=0 tx=0).stop 2023-07-06T20:27:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.945+0000 7f1211994b80 1 -- 172.21.15.69:0/3637371614 shutdown_connections 2023-07-06T20:27:26.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.945+0000 7f1211994b80 1 --2- 172.21.15.69:0/3637371614 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x557c98d8d720 0x557c98d8fb30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.945+0000 7f1211994b80 1 --2- 172.21.15.69:0/3637371614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c98d243a0 0x557c98d30490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.945+0000 7f1211994b80 1 --2- 172.21.15.69:0/3637371614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c98d22f30 0x557c98d33900 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.945+0000 7f1211994b80 1 --2- 172.21.15.69:0/3637371614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c98d23880 0x557c98d33e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.945+0000 7f1211994b80 1 -- 172.21.15.69:0/3637371614 >> 172.21.15.69:0/3637371614 conn(0x557c98d07640 msgr2=0x557c98d2b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:27:26.948 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.945+0000 7f11d6ffd700 0 client.25627 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:27:26.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.945+0000 7f1211994b80 1 -- 172.21.15.69:0/3637371614 shutdown_connections 2023-07-06T20:27:26.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.945+0000 7f1211994b80 1 -- 172.21.15.69:0/3637371614 wait complete. 2023-07-06T20:27:26.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.946+0000 7f1211994b80 20 client.25627 trim_cache size 0 max 16384 2023-07-06T20:27:26.949 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:27:26.950 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:27:26.950 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:27:26.950 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:27:26.950 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:27:26.950 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:27:26.951 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:27:26.951 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:27:26.951 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:27:26.951 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:27:26.952 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.948+0000 7f11fdfff700 1 -- 172.21.15.69:0/78814484 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f11e00a1350 msgr2=0x7f11e00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:26.952 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.948+0000 7f11fdfff700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f11e00a1350 0x7f11e00a3800 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f11f000b8b0 tx=0x7f11f0026000 comp rx=0 tx=0).stop 2023-07-06T20:27:26.952 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.948+0000 7f11fdfff700 1 -- 172.21.15.69:0/78814484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11f8071330 msgr2=0x7f11f8082400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:26.952 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.948+0000 7f11fdfff700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11f8071330 0x7f11f8082400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f11e80078a0 tx=0x7f11e8007c60 comp rx=0 tx=0).stop 2023-07-06T20:27:26.952 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.948+0000 7f11fdfff700 1 -- 172.21.15.69:0/78814484 shutdown_connections 2023-07-06T20:27:26.953 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.948+0000 7f11fdfff700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f11e00a1350 0x7f11e00a3800 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.953 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.948+0000 7f11fdfff700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11f8077340 0x7f11f81b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.953 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.948+0000 7f11fdfff700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11f8071330 0x7f11f8082400 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.953 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.948+0000 7f11fdfff700 1 --2- 172.21.15.69:0/78814484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8074980 0x7f11f8082940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:26.953 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.948+0000 7f11fdfff700 1 -- 172.21.15.69:0/78814484 >> 172.21.15.69:0/78814484 conn(0x7f11f806cbc0 msgr2=0x7f11f80761f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:27:26.953 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.948+0000 7f11fdfff700 1 -- 172.21.15.69:0/78814484 shutdown_connections 2023-07-06T20:27:26.954 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:26.948+0000 7f11fdfff700 1 -- 172.21.15.69:0/78814484 wait complete. 2023-07-06T20:27:26.995 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:27:28.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:27 smithi069 ceph-mon[126399]: pgmap v571: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 37 MiB/s wr, 1.08k op/s 2023-07-06T20:27:28.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:27 smithi156 ceph-mon[129280]: pgmap v571: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 37 MiB/s wr, 1.08k op/s 2023-07-06T20:27:28.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:27 smithi098 ceph-mon[128968]: pgmap v571: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 37 MiB/s wr, 1.08k op/s 2023-07-06T20:27:29.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 340.0 s, 1015.4 tps, lat 98.334 ms stddev 129.108 2023-07-06T20:27:29.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:29 smithi156 ceph-mon[129280]: pgmap v572: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 36 MiB/s wr, 1.44k op/s 2023-07-06T20:27:29.379 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:29 smithi098 ceph-mon[128968]: pgmap v572: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 36 MiB/s wr, 1.44k op/s 2023-07-06T20:27:29.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:29 smithi069 ceph-mon[126399]: pgmap v572: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 36 MiB/s wr, 1.44k op/s 2023-07-06T20:27:31.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:31 smithi156 ceph-mon[129280]: pgmap v573: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 53 MiB/s wr, 3.49k op/s 2023-07-06T20:27:31.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:31 smithi098 ceph-mon[128968]: pgmap v573: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 53 MiB/s wr, 3.49k op/s 2023-07-06T20:27:31.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:31 smithi069 ceph-mon[126399]: pgmap v573: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 53 MiB/s wr, 3.49k op/s 2023-07-06T20:27:33.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:33 smithi098 ceph-mon[128968]: pgmap v574: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 50 MiB/s wr, 3.74k op/s 2023-07-06T20:27:33.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:33 smithi069 ceph-mon[126399]: pgmap v574: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 50 MiB/s wr, 3.74k op/s 2023-07-06T20:27:33.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:33 smithi156 ceph-mon[129280]: pgmap v574: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 50 MiB/s wr, 3.74k op/s 2023-07-06T20:27:34.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 345.0 s, 2825.2 tps, lat 35.441 ms stddev 22.894 2023-07-06T20:27:35.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:35 smithi069 ceph-mon[126399]: pgmap v575: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 59 MiB/s wr, 3.97k op/s 2023-07-06T20:27:35.613 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:35 smithi156 ceph-mon[129280]: pgmap v575: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 59 MiB/s wr, 3.97k op/s 2023-07-06T20:27:35.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:35 smithi098 ceph-mon[128968]: pgmap v575: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 59 MiB/s wr, 3.97k op/s 2023-07-06T20:27:37.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:37 smithi069 ceph-mon[126399]: pgmap v576: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 59 MiB/s wr, 3.90k op/s 2023-07-06T20:27:37.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:37 smithi156 ceph-mon[129280]: pgmap v576: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 59 MiB/s wr, 3.90k op/s 2023-07-06T20:27:37.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:37 smithi098 ceph-mon[128968]: pgmap v576: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 59 MiB/s wr, 3.90k op/s 2023-07-06T20:27:39.234 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 350.0 s, 1607.8 tps, lat 53.127 ms stddev 85.279 2023-07-06T20:27:39.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:39 smithi156 ceph-mon[129280]: pgmap v577: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 MiB/s wr, 3.75k op/s 2023-07-06T20:27:39.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:39 smithi098 ceph-mon[128968]: pgmap v577: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 MiB/s wr, 3.75k op/s 2023-07-06T20:27:39.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:39 smithi069 ceph-mon[126399]: pgmap v577: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 MiB/s wr, 3.75k op/s 2023-07-06T20:27:41.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:41 smithi156 ceph-mon[129280]: pgmap v578: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 57 MiB/s wr, 3.99k op/s 2023-07-06T20:27:41.628 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:41 smithi098 ceph-mon[128968]: pgmap v578: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 57 MiB/s wr, 3.99k op/s 2023-07-06T20:27:41.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:41 smithi069 ceph-mon[126399]: pgmap v578: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 57 MiB/s wr, 3.99k op/s 2023-07-06T20:27:43.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:43 smithi156 ceph-mon[129280]: pgmap v579: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 MiB/s wr, 2.17k op/s 2023-07-06T20:27:43.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:43 smithi098 ceph-mon[128968]: pgmap v579: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 MiB/s wr, 2.17k op/s 2023-07-06T20:27:43.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:43 smithi069 ceph-mon[126399]: pgmap v579: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 MiB/s wr, 2.17k op/s 2023-07-06T20:27:44.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 355.0 s, 2433.2 tps, lat 45.005 ms stddev 111.599 2023-07-06T20:27:45.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:45 smithi069 ceph-mon[126399]: pgmap v580: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 43 MiB/s wr, 2.26k op/s 2023-07-06T20:27:45.866 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:45 smithi156 ceph-mon[129280]: pgmap v580: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 43 MiB/s wr, 2.26k op/s 2023-07-06T20:27:45.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:45 smithi098 ceph-mon[128968]: pgmap v580: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 43 MiB/s wr, 2.26k op/s 2023-07-06T20:27:47.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:47 smithi069 ceph-mon[126399]: pgmap v581: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 MiB/s wr, 2.62k op/s 2023-07-06T20:27:47.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:47 smithi156 ceph-mon[129280]: pgmap v581: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 MiB/s wr, 2.62k op/s 2023-07-06T20:27:47.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:47 smithi098 ceph-mon[128968]: pgmap v581: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 MiB/s wr, 2.62k op/s 2023-07-06T20:27:49.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 360.0 s, 1636.3 tps, lat 64.273 ms stddev 90.004 2023-07-06T20:27:49.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:49 smithi069 ceph-mon[126399]: pgmap v582: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 46 MiB/s wr, 2.97k op/s 2023-07-06T20:27:49.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:49 smithi156 ceph-mon[129280]: pgmap v582: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 46 MiB/s wr, 2.97k op/s 2023-07-06T20:27:49.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:49 smithi098 ceph-mon[128968]: pgmap v582: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 46 MiB/s wr, 2.97k op/s 2023-07-06T20:27:51.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:51 smithi069 ceph-mon[126399]: pgmap v583: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 55 MiB/s wr, 3.86k op/s 2023-07-06T20:27:51.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:51 smithi156 ceph-mon[129280]: pgmap v583: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 55 MiB/s wr, 3.86k op/s 2023-07-06T20:27:51.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:51 smithi098 ceph-mon[128968]: pgmap v583: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 55 MiB/s wr, 3.86k op/s 2023-07-06T20:27:53.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:53 smithi069 ceph-mon[126399]: pgmap v584: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 44 MiB/s wr, 3.06k op/s 2023-07-06T20:27:53.866 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:53 smithi156 ceph-mon[129280]: pgmap v584: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 44 MiB/s wr, 3.06k op/s 2023-07-06T20:27:53.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:53 smithi098 ceph-mon[128968]: pgmap v584: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 44 MiB/s wr, 3.06k op/s 2023-07-06T20:27:54.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 365.0 s, 2796.8 tps, lat 35.657 ms stddev 21.674 2023-07-06T20:27:54.866 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:54 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:27:54.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:54 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:27:55.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:54 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:27:55.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:55 smithi156 ceph-mon[129280]: pgmap v585: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 2.88k op/s 2023-07-06T20:27:55.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:55 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:27:55.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:55 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:27:55.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:55 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:27:55.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:55 smithi098 ceph-mon[128968]: pgmap v585: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 2.88k op/s 2023-07-06T20:27:55.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:55 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:27:55.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:55 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:27:55.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:55 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:27:56.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:55 smithi069 ceph-mon[126399]: pgmap v585: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 2.88k op/s 2023-07-06T20:27:56.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:55 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:27:56.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:55 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:27:56.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:55 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:27:56.997 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:27:57.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.353+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1892966579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c98102fb0 msgr2=0x7f2c98105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:57.366 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.353+0000 7f2ca03ff700 1 --2- 172.21.15.69:0/1892966579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c98102fb0 0x7f2c98105390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009990 tx=0x7f2c88009ca0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.365+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1892966579 shutdown_connections 2023-07-06T20:27:57.367 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.365+0000 7f2ca03ff700 1 --2- 172.21.15.69:0/1892966579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c981058d0 0x7f2c98107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.365+0000 7f2ca03ff700 1 --2- 172.21.15.69:0/1892966579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c98100690 0x7f2c98102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.365+0000 7f2ca03ff700 1 --2- 172.21.15.69:0/1892966579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c98102fb0 0x7f2c98105390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.368 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.365+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1892966579 >> 172.21.15.69:0/1892966579 conn(0x7f2c980fa130 msgr2=0x7f2c980fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:27:57.376 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.374+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1892966579 shutdown_connections 2023-07-06T20:27:57.385 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.381+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1892966579 wait complete. 2023-07-06T20:27:57.386 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.383+0000 7f2ca03ff700 1 Processor -- start 2023-07-06T20:27:57.387 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.385+0000 7f2ca03ff700 1 -- start start 2023-07-06T20:27:57.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.385+0000 7f2ca03ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c98100690 0x7f2c98193760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.385+0000 7f2ca03ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c98102fb0 0x7f2c98193ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.385+0000 7f2ca03ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c981058d0 0x7f2c9819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.385+0000 7f2ca03ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9819b2e0 con 0x7f2c98102fb0 2023-07-06T20:27:57.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.385+0000 7f2ca03ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9819b450 con 0x7f2c98100690 2023-07-06T20:27:57.388 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.385+0000 7f2ca03ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9819b610 con 0x7f2c981058d0 2023-07-06T20:27:57.389 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.386+0000 7f2c9e19b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c98100690 0x7f2c98193760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.389 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.386+0000 7f2c9e19b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c98100690 0x7f2c98193760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:48542/0 (socket says 172.21.15.69:48542) 2023-07-06T20:27:57.390 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.386+0000 7f2c9e19b700 1 -- 172.21.15.69:0/1112538860 learned_addr learned my addr 172.21.15.69:0/1112538860 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:27:57.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.387+0000 7f2c9e99c700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c981058d0 0x7f2c9819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.388+0000 7f2c9e99c700 1 -- 172.21.15.69:0/1112538860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c98100690 msgr2=0x7f2c98193760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:57.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.388+0000 7f2c9e99c700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c98100690 0x7f2c98193760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.388+0000 7f2c9e99c700 1 -- 172.21.15.69:0/1112538860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c98102fb0 msgr2=0x7f2c98193ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:57.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.388+0000 7f2c9e99c700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c98102fb0 0x7f2c98193ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.388+0000 7f2c9e99c700 1 -- 172.21.15.69:0/1112538860 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c94009520 con 0x7f2c981058d0 2023-07-06T20:27:57.392 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.388+0000 7f2c9e99c700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c981058d0 0x7f2c9819ad10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2c9000efc0 tx=0x7f2c9000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:27:57.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.389+0000 7f2c8f7fe700 1 -- 172.21.15.69:0/1112538860 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90018070 con 0x7f2c981058d0 2023-07-06T20:27:57.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.389+0000 7f2c8f7fe700 1 -- 172.21.15.69:0/1112538860 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c9000f040 con 0x7f2c981058d0 2023-07-06T20:27:57.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.389+0000 7f2c8f7fe700 1 -- 172.21.15.69:0/1112538860 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9000ce90 con 0x7f2c981058d0 2023-07-06T20:27:57.394 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.389+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1112538860 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c880095f0 con 0x7f2c981058d0 2023-07-06T20:27:57.395 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.389+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1112538860 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9819bb60 con 0x7f2c981058d0 2023-07-06T20:27:57.397 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.395+0000 7f2cb3c70b80 1 Processor -- start 2023-07-06T20:27:57.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.399+0000 7f2c8f7fe700 1 -- 172.21.15.69:0/1112538860 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2c90004ad0 con 0x7f2c981058d0 2023-07-06T20:27:57.401 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.399+0000 7f2cb3c70b80 1 -- start start 2023-07-06T20:27:57.403 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.399+0000 7f2cb3c70b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce1f20 0x5653a6ce2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.403 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.399+0000 7f2cb3c70b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce2870 0x5653a6ce2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.399+0000 7f2cb3c70b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce3390 0x5653a6ce3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.399+0000 7f2cb3c70b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a6be6cb0 con 0x5653a6ce2870 2023-07-06T20:27:57.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.399+0000 7f2cb3c70b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a6c39a40 con 0x5653a6ce3390 2023-07-06T20:27:57.404 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.399+0000 7f2cb3c70b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a6c32ed0 con 0x5653a6ce1f20 2023-07-06T20:27:57.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.400+0000 7f2c9d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce1f20 0x5653a6ce2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.400+0000 7f2c9d99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce1f20 0x5653a6ce2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:51990/0 (socket says 172.21.15.69:51990) 2023-07-06T20:27:57.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.400+0000 7f2c9d99a700 1 -- 172.21.15.69:0/4221518815 learned_addr learned my addr 172.21.15.69:0/4221518815 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:27:57.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.400+0000 7f2c9d99a700 1 --2- 172.21.15.69:0/4221518815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce3390 0x5653a6ce3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.401+0000 7f2c9e19b700 1 --2- 172.21.15.69:0/4221518815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce2870 0x5653a6ce2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.401+0000 7f2c9d99a700 1 -- 172.21.15.69:0/4221518815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce3390 msgr2=0x5653a6ce3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:57.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.401+0000 7f2c9d99a700 1 --2- 172.21.15.69:0/4221518815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce3390 0x5653a6ce3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.406 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.401+0000 7f2c9d99a700 1 -- 172.21.15.69:0/4221518815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce2870 msgr2=0x5653a6ce2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:57.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.401+0000 7f2c9d99a700 1 --2- 172.21.15.69:0/4221518815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce2870 0x5653a6ce2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.401+0000 7f2c9d99a700 1 -- 172.21.15.69:0/4221518815 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653a6a60de0 con 0x5653a6ce1f20 2023-07-06T20:27:57.407 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.401+0000 7f2c9d99a700 1 --2- 172.21.15.69:0/4221518815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce1f20 0x5653a6ce2330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800b890 tx=0x7f2c88009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=233c75c71fbf87f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:27:57.411 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.401+0000 7f2c8cbfe700 1 -- 172.21.15.69:0/4221518815 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8800bdd0 con 0x5653a6ce1f20 2023-07-06T20:27:57.411 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.401+0000 7f2c8cbfe700 1 -- 172.21.15.69:0/4221518815 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c8800f920 con 0x5653a6ce1f20 2023-07-06T20:27:57.411 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.401+0000 7f2c8cbfe700 1 -- 172.21.15.69:0/4221518815 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88017650 con 0x5653a6ce1f20 2023-07-06T20:27:57.411 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.402+0000 7f2c8f7fe700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f2c840a1320 0x7f2c840a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.412 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.402+0000 7f2c8f7fe700 1 -- 172.21.15.69:0/1112538860 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f2c900cc0e0 con 0x7f2c981058d0 2023-07-06T20:27:57.412 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.402+0000 7f2c9e19b700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f2c840a1320 0x7f2c840a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.412 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.402+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/4221518815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce1f20 msgr2=0x5653a6ce2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:57.412 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.402+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/4221518815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce1f20 0x5653a6ce2330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800b890 tx=0x7f2c88009530 comp rx=0 tx=0).stop 2023-07-06T20:27:57.413 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.402+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/4221518815 shutdown_connections 2023-07-06T20:27:57.413 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.402+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/4221518815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce3390 0x5653a6ce3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.413 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.402+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/4221518815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce1f20 0x5653a6ce2330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.413 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.402+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/4221518815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce2870 0x5653a6ce2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.413 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.402+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/4221518815 >> 172.21.15.69:0/4221518815 conn(0x5653a6cd7640 msgr2=0x5653a6cdc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:27:57.414 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.403+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/4221518815 shutdown_connections 2023-07-06T20:27:57.414 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.403+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/4221518815 wait complete. 2023-07-06T20:27:57.414 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.403+0000 7f2c9e19b700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f2c840a1320 0x7f2c840a37d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2c940124a0 tx=0x7f2c94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:27:57.414 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.403+0000 7f2cb3c70b80 1 -- start start 2023-07-06T20:27:57.414 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.404+0000 7f2cb3c70b80 1 Processor -- start 2023-07-06T20:27:57.415 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.404+0000 7f2cb3c70b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce1f20 0x5653a6ceb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.415 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.404+0000 7f2cb3c70b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce2870 0x5653a6cebd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.415 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.405+0000 7f2cb3c70b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce3390 0x5653a6cf2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.415 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.405+0000 7f2cb3c70b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a6bd2ac0 con 0x5653a6ce1f20 2023-07-06T20:27:57.416 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.405+0000 7f2cb3c70b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a6bd27c0 con 0x5653a6ce2870 2023-07-06T20:27:57.416 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.405+0000 7f2cb3c70b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a6ab6920 con 0x5653a6ce3390 2023-07-06T20:27:57.416 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.405+0000 7f2cb3c70b80 10 client.0 _ll_register_callbacks cb 0x5653a6bd0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:27:57.416 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.405+0000 7f2c9e19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce1f20 0x5653a6ceb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.416 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.405+0000 7f2c9d99a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce2870 0x5653a6cebd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.417 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.405+0000 7f2c9d99a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce2870 0x5653a6cebd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:48562/0 (socket says 172.21.15.69:48562) 2023-07-06T20:27:57.417 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.405+0000 7f2c9d99a700 1 -- 172.21.15.69:0/2728989398 learned_addr learned my addr 172.21.15.69:0/2728989398 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:27:57.417 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.405+0000 7f2c9e99c700 1 --2- 172.21.15.69:0/2728989398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce3390 0x5653a6cf2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.417 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.406+0000 7f2c9d99a700 1 -- 172.21.15.69:0/2728989398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce3390 msgr2=0x5653a6cf2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:57.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.406+0000 7f2c9d99a700 1 --2- 172.21.15.69:0/2728989398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce3390 0x5653a6cf2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.406+0000 7f2c9d99a700 1 -- 172.21.15.69:0/2728989398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce1f20 msgr2=0x5653a6ceb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:57.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.406+0000 7f2c9d99a700 1 --2- 172.21.15.69:0/2728989398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce1f20 0x5653a6ceb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.406+0000 7f2c9d99a700 1 -- 172.21.15.69:0/2728989398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c98108920 con 0x5653a6ce2870 2023-07-06T20:27:57.418 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.406+0000 7f2c9d99a700 1 --2- 172.21.15.69:0/2728989398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce2870 0x5653a6cebd70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009350 tx=0x7f2c8800bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adef710dce1397a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:27:57.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.406+0000 7f2c81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:27:57.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.406+0000 7f2c81ffb700 1 -- 172.21.15.69:0/2728989398 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8802c070 con 0x5653a6ce2870 2023-07-06T20:27:57.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.407+0000 7f2c81ffb700 1 -- 172.21.15.69:0/2728989398 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f2c8800f460 con 0x5653a6ce2870 2023-07-06T20:27:57.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.407+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2728989398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653a6b8c4f0 con 0x5653a6ce2870 2023-07-06T20:27:57.419 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.407+0000 7f2c81ffb700 1 -- 172.21.15.69:0/2728989398 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c8800f620 con 0x5653a6ce2870 2023-07-06T20:27:57.420 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.407+0000 7f2cb3c70b80 10 client.25682 fetch_fsmap learned FSMap version 30 2023-07-06T20:27:57.420 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.407+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2728989398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5653a6a49c60 con 0x5653a6ce2870 2023-07-06T20:27:57.420 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.407+0000 7f2c81ffb700 1 -- 172.21.15.69:0/2728989398 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5653a6b8c4f0 con 0x5653a6ce2870 2023-07-06T20:27:57.420 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.407+0000 7f2c81ffb700 1 -- 172.21.15.69:0/2728989398 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2c88033ba0 con 0x5653a6ce2870 2023-07-06T20:27:57.420 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.408+0000 7f2cb3c70b80 10 client.25682 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:27:57.421 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.408+0000 7f2cb3c70b80 10 client.25682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:27:57.421 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.408+0000 7f2cb3c70b80 20 client.25682 populate_metadata read hostname 'smithi069' 2023-07-06T20:27:57.421 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.408+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2728989398 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5653a6cf7170 0x5653a6cf9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.421 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.408+0000 7f2cb3c70b80 4 client.25682 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:27:57.421 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.408+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2728989398 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653a6cb0d60 con 0x5653a6cf7170 2023-07-06T20:27:57.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.408+0000 7f2c9e19b700 1 --2- 172.21.15.69:0/2728989398 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5653a6cf7170 0x5653a6cf9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.408+0000 7f2c9e19b700 1 --2- 172.21.15.69:0/2728989398 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5653a6cf7170 0x5653a6cf9580 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f701d337562621 server_cookie=ff8ad8f46b2bac6f in_seq=0 out_seq=0 2023-07-06T20:27:57.422 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.408+0000 7f2c81ffb700 10 client.25682 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:27:57.423 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.409+0000 7f2c81ffb700 1 -- 172.21.15.69:0/2728989398 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5653a6cb0d60 con 0x5653a6cf7170 2023-07-06T20:27:57.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.409+0000 7f2c81ffb700 10 client.25682 handle_command_reply: tid=0 2023-07-06T20:27:57.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.409+0000 7f2cb3c70b80 1 client.25682 shutdown 2023-07-06T20:27:57.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.410+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2728989398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce2870 msgr2=0x5653a6cebd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:57.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.410+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2728989398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce2870 0x5653a6cebd70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009350 tx=0x7f2c8800bc60 comp rx=0 tx=0).stop 2023-07-06T20:27:57.424 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.413+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2728989398 shutdown_connections 2023-07-06T20:27:57.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.413+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2728989398 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5653a6cf7170 0x5653a6cf9580 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.413+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2728989398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce3390 0x5653a6cf2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.413+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2728989398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce1f20 0x5653a6ceb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.413+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2728989398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce2870 0x5653a6cebd70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.425 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.413+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2728989398 >> 172.21.15.69:0/2728989398 conn(0x5653a6cd7640 msgr2=0x5653a6cdc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:27:57.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.418+0000 7f2c81ffb700 0 client.25682 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:27:57.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.418+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2728989398 shutdown_connections 2023-07-06T20:27:57.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.419+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2728989398 wait complete. 2023-07-06T20:27:57.426 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.419+0000 7f2cb3c70b80 20 client.25682 trim_cache size 0 max 16384 2023-07-06T20:27:57.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.445+0000 7f2cb3c70b80 1 Processor -- start 2023-07-06T20:27:57.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.445+0000 7f2cb3c70b80 1 -- start start 2023-07-06T20:27:57.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.446+0000 7f2cb3c70b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce1f20 0x5653a6cdc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.446+0000 7f2cb3c70b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce2870 0x5653a6cf2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.446+0000 7f2cb3c70b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce3390 0x5653a6cf9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.446+0000 7f2cb3c70b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a6ceace0 con 0x5653a6ce3390 2023-07-06T20:27:57.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.446+0000 7f2cb3c70b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a6cd71c0 con 0x5653a6ce1f20 2023-07-06T20:27:57.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.446+0000 7f2cb3c70b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a6cde420 con 0x5653a6ce2870 2023-07-06T20:27:57.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.446+0000 7f2c9e19b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce1f20 0x5653a6cdc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.446+0000 7f2c9e19b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce1f20 0x5653a6cdc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:48576/0 (socket says 172.21.15.69:48576) 2023-07-06T20:27:57.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.446+0000 7f2c9e19b700 1 -- 172.21.15.69:0/2786955296 learned_addr learned my addr 172.21.15.69:0/2786955296 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:27:57.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.446+0000 7f2c9d99a700 1 --2- 172.21.15.69:0/2786955296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce2870 0x5653a6cf2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.447+0000 7f2c9e99c700 1 --2- 172.21.15.69:0/2786955296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce3390 0x5653a6cf9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.447+0000 7f2c9e19b700 1 -- 172.21.15.69:0/2786955296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce2870 msgr2=0x5653a6cf2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:57.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.447+0000 7f2c9e19b700 1 --2- 172.21.15.69:0/2786955296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce2870 0x5653a6cf2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.453 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.447+0000 7f2c9e19b700 1 -- 172.21.15.69:0/2786955296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce3390 msgr2=0x5653a6cf9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:57.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.447+0000 7f2c9e19b700 1 --2- 172.21.15.69:0/2786955296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce3390 0x5653a6cf9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.447+0000 7f2c9e19b700 1 -- 172.21.15.69:0/2786955296 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653a6ce3c80 con 0x5653a6ce1f20 2023-07-06T20:27:57.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.447+0000 7f2c9e99c700 1 --2- 172.21.15.69:0/2786955296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce3390 0x5653a6cf9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:27:57.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.447+0000 7f2c9d99a700 1 --2- 172.21.15.69:0/2786955296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce2870 0x5653a6cf2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:27:57.454 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.447+0000 7f2c9e19b700 1 --2- 172.21.15.69:0/2786955296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce1f20 0x5653a6cdc0b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2c94004390 tx=0x7f2c94019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d7001bc137c5e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:27:57.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.447+0000 7f2c8d3ff700 1 -- 172.21.15.69:0/2786955296 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94023030 con 0x5653a6ce1f20 2023-07-06T20:27:57.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.447+0000 7f2c8d3ff700 1 -- 172.21.15.69:0/2786955296 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c94024040 con 0x5653a6ce1f20 2023-07-06T20:27:57.455 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.447+0000 7f2c8d3ff700 1 -- 172.21.15.69:0/2786955296 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94017070 con 0x5653a6ce1f20 2023-07-06T20:27:57.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.448+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2786955296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce1f20 msgr2=0x5653a6cdc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:57.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.448+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2786955296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce1f20 0x5653a6cdc0b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2c94004390 tx=0x7f2c94019000 comp rx=0 tx=0).stop 2023-07-06T20:27:57.457 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.448+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2786955296 shutdown_connections 2023-07-06T20:27:57.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.448+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2786955296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce3390 0x5653a6cf9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.448+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2786955296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce1f20 0x5653a6cdc0b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.448+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2786955296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce2870 0x5653a6cf2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.458 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.448+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2786955296 >> 172.21.15.69:0/2786955296 conn(0x5653a6cd7640 msgr2=0x5653a6cfdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:27:57.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.448+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2786955296 shutdown_connections 2023-07-06T20:27:57.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.448+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2786955296 wait complete. 2023-07-06T20:27:57.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.449+0000 7f2cb3c70b80 1 -- start start 2023-07-06T20:27:57.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.449+0000 7f2cb3c70b80 1 Processor -- start 2023-07-06T20:27:57.460 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.450+0000 7f2cb3c70b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce1f20 0x5653a6ceee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.460 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.450+0000 7f2cb3c70b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce2870 0x5653a6cef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.460 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.450+0000 7f2cb3c70b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce3390 0x5653a6cf2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.460 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.450+0000 7f2cb3c70b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a6cf28b0 con 0x5653a6ce2870 2023-07-06T20:27:57.461 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.450+0000 7f2cb3c70b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a6cf2a20 con 0x5653a6ce1f20 2023-07-06T20:27:57.461 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.450+0000 7f2cb3c70b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a6cf2b90 con 0x5653a6ce3390 2023-07-06T20:27:57.461 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.450+0000 7f2cb3c70b80 10 client.0 _ll_register_callbacks cb 0x7f2c900097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:27:57.461 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.451+0000 7f2c9e99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce3390 0x5653a6cf2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.462 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.451+0000 7f2c9e99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce3390 0x5653a6cf2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:52022/0 (socket says 172.21.15.69:52022) 2023-07-06T20:27:57.462 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.451+0000 7f2c9e99c700 1 -- 172.21.15.69:0/2861289230 learned_addr learned my addr 172.21.15.69:0/2861289230 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:27:57.462 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.451+0000 7f2c9e19b700 1 --2- 172.21.15.69:0/2861289230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce1f20 0x5653a6ceee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.462 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.452+0000 7f2c9e99c700 1 -- 172.21.15.69:0/2861289230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce1f20 msgr2=0x5653a6ceee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:57.462 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.452+0000 7f2c9e99c700 1 --2- 172.21.15.69:0/2861289230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce1f20 0x5653a6ceee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.463 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.452+0000 7f2c9e99c700 1 -- 172.21.15.69:0/2861289230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce2870 msgr2=0x5653a6cef350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:27:57.463 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.452+0000 7f2c9e99c700 1 --2- 172.21.15.69:0/2861289230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce2870 0x5653a6cef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.463 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.452+0000 7f2c9e99c700 1 -- 172.21.15.69:0/2861289230 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c94004040 con 0x5653a6ce3390 2023-07-06T20:27:57.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.457+0000 7f2c9e99c700 1 --2- 172.21.15.69:0/2861289230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce3390 0x5653a6cf2370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2c90099480 tx=0x7f2c90099a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d923cefd9748fcff server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:27:57.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.457+0000 7f2c81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:27:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.458+0000 7f2c81ffb700 1 -- 172.21.15.69:0/2861289230 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9009d360 con 0x5653a6ce3390 2023-07-06T20:27:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.458+0000 7f2c81ffb700 1 -- 172.21.15.69:0/2861289230 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f2c900994f0 con 0x5653a6ce3390 2023-07-06T20:27:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.458+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2861289230 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c8800f920 con 0x5653a6ce3390 2023-07-06T20:27:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.458+0000 7f2c81ffb700 1 -- 172.21.15.69:0/2861289230 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c90098e50 con 0x5653a6ce3390 2023-07-06T20:27:57.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.458+0000 7f2cb3c70b80 10 client.25645 fetch_fsmap learned FSMap version 30 2023-07-06T20:27:57.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.458+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2861289230 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5653a6a87250 con 0x5653a6ce3390 2023-07-06T20:27:57.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.459+0000 7f2c81ffb700 1 -- 172.21.15.69:0/2861289230 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c900b7020 con 0x5653a6ce3390 2023-07-06T20:27:57.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.459+0000 7f2c81ffb700 1 -- 172.21.15.69:0/2861289230 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2c9009f0d0 con 0x5653a6ce3390 2023-07-06T20:27:57.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.459+0000 7f2cb3c70b80 10 client.25645 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:27:57.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.459+0000 7f2cb3c70b80 10 client.25645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:27:57.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.459+0000 7f2cb3c70b80 20 client.25645 populate_metadata read hostname 'smithi069' 2023-07-06T20:27:57.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.459+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2861289230 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5653a6d5d2f0 0x5653a6d5f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:27:57.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.459+0000 7f2cb3c70b80 4 client.25645 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:27:57.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.459+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2861289230 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5653a6c2beb0 con 0x5653a6d5d2f0 2023-07-06T20:27:57.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.460+0000 7f2c9e19b700 1 --2- 172.21.15.69:0/2861289230 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5653a6d5d2f0 0x5653a6d5f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:27:57.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.460+0000 7f2c9e19b700 1 --2- 172.21.15.69:0/2861289230 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5653a6d5d2f0 0x5653a6d5f700 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a7cd02d86ad42a server_cookie=c246b836d8b87b05 in_seq=0 out_seq=0 2023-07-06T20:27:57.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.460+0000 7f2c81ffb700 10 client.25645 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:27:57.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.461+0000 7f2c81ffb700 1 -- 172.21.15.69:0/2861289230 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5653a6c2beb0 con 0x5653a6d5d2f0 2023-07-06T20:27:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.461+0000 7f2c81ffb700 10 client.25645 handle_command_reply: tid=0 2023-07-06T20:27:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.461+0000 7f2cb3c70b80 1 client.25645 shutdown 2023-07-06T20:27:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.461+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2861289230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce3390 msgr2=0x5653a6cf2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.461+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2861289230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce3390 0x5653a6cf2370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2c90099480 tx=0x7f2c90099a30 comp rx=0 tx=0).stop 2023-07-06T20:27:57.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.461+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2861289230 shutdown_connections 2023-07-06T20:27:57.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.461+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2861289230 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5653a6d5d2f0 0x5653a6d5f700 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.462+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2861289230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5653a6ce3390 0x5653a6cf2370 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.462+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2861289230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5653a6ce1f20 0x5653a6ceee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.462+0000 7f2cb3c70b80 1 --2- 172.21.15.69:0/2861289230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5653a6ce2870 0x5653a6cef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.462+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2861289230 >> 172.21.15.69:0/2861289230 conn(0x5653a6cd7640 msgr2=0x5653a6cdc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:27:57.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.462+0000 7f2c81ffb700 0 client.25645 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:27:57.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.462+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2861289230 shutdown_connections 2023-07-06T20:27:57.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.462+0000 7f2cb3c70b80 1 -- 172.21.15.69:0/2861289230 wait complete. 2023-07-06T20:27:57.474 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.462+0000 7f2cb3c70b80 20 client.25645 trim_cache size 0 max 16384 2023-07-06T20:27:57.474 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:27:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:27:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:27:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:27:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:27:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:27:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:27:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:27:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:27:57.476 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:27:57.477 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.463+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1112538860 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f2c840a1320 msgr2=0x7f2c840a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:57.477 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.463+0000 7f2ca03ff700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f2c840a1320 0x7f2c840a37d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2c940124a0 tx=0x7f2c94009260 comp rx=0 tx=0).stop 2023-07-06T20:27:57.478 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.463+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1112538860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c981058d0 msgr2=0x7f2c9819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:27:57.478 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.463+0000 7f2ca03ff700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c981058d0 0x7f2c9819ad10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2c9000efc0 tx=0x7f2c9000c3a0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.478 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.463+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1112538860 shutdown_connections 2023-07-06T20:27:57.478 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.463+0000 7f2ca03ff700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f2c840a1320 0x7f2c840a37d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.463+0000 7f2ca03ff700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c981058d0 0x7f2c9819ad10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.463+0000 7f2ca03ff700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c98100690 0x7f2c98193760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.463+0000 7f2ca03ff700 1 --2- 172.21.15.69:0/1112538860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c98102fb0 0x7f2c98193ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:27:57.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.463+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1112538860 >> 172.21.15.69:0/1112538860 conn(0x7f2c980fa130 msgr2=0x7f2c980fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:27:57.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.463+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1112538860 shutdown_connections 2023-07-06T20:27:57.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:27:57.463+0000 7f2ca03ff700 1 -- 172.21.15.69:0/1112538860 wait complete. 2023-07-06T20:27:57.495 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:27:57.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:57 smithi069 ceph-mon[126399]: pgmap v586: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 MiB/s wr, 2.50k op/s 2023-07-06T20:27:58.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:57 smithi156 ceph-mon[129280]: pgmap v586: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 MiB/s wr, 2.50k op/s 2023-07-06T20:27:58.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:57 smithi098 ceph-mon[128968]: pgmap v586: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 MiB/s wr, 2.50k op/s 2023-07-06T20:27:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 370.0 s, 2054.2 tps, lat 48.596 ms stddev 65.927 2023-07-06T20:28:00.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:27:59 smithi069 ceph-mon[126399]: pgmap v587: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 MiB/s wr, 1.98k op/s 2023-07-06T20:28:00.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:27:59 smithi156 ceph-mon[129280]: pgmap v587: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 MiB/s wr, 1.98k op/s 2023-07-06T20:28:00.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:27:59 smithi098 ceph-mon[128968]: pgmap v587: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 MiB/s wr, 1.98k op/s 2023-07-06T20:28:02.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:01 smithi069 ceph-mon[126399]: pgmap v588: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 1.65k op/s 2023-07-06T20:28:02.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:01 smithi156 ceph-mon[129280]: pgmap v588: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 1.65k op/s 2023-07-06T20:28:02.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:01 smithi098 ceph-mon[128968]: pgmap v588: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 1.65k op/s 2023-07-06T20:28:04.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:03 smithi156 ceph-mon[129280]: pgmap v589: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 767 op/s 2023-07-06T20:28:04.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:03 smithi098 ceph-mon[128968]: pgmap v589: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 767 op/s 2023-07-06T20:28:04.236 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 375.0 s, 1442.8 tps, lat 66.226 ms stddev 94.494 2023-07-06T20:28:04.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:03 smithi069 ceph-mon[126399]: pgmap v589: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 767 op/s 2023-07-06T20:28:05.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:04 smithi156 ceph-mon[129280]: pgmap v590: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 MiB/s wr, 1.64k op/s 2023-07-06T20:28:05.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:04 smithi098 ceph-mon[128968]: pgmap v590: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 MiB/s wr, 1.64k op/s 2023-07-06T20:28:05.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:04 smithi069 ceph-mon[126399]: pgmap v590: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 MiB/s wr, 1.64k op/s 2023-07-06T20:28:07.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:07 smithi156 ceph-mon[129280]: pgmap v591: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 MiB/s wr, 3.12k op/s 2023-07-06T20:28:07.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:07 smithi098 ceph-mon[128968]: pgmap v591: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 MiB/s wr, 3.12k op/s 2023-07-06T20:28:07.520 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:07 smithi069 ceph-mon[126399]: pgmap v591: 97 pgs: 97 active+clean; 8.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 MiB/s wr, 3.12k op/s 2023-07-06T20:28:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 380.0 s, 2146.4 tps, lat 48.708 ms stddev 72.168 2023-07-06T20:28:09.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:09 smithi156 ceph-mon[129280]: pgmap v592: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 47 MiB/s wr, 3.40k op/s 2023-07-06T20:28:09.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:09 smithi098 ceph-mon[128968]: pgmap v592: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 47 MiB/s wr, 3.40k op/s 2023-07-06T20:28:09.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:09 smithi069 ceph-mon[126399]: pgmap v592: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 47 MiB/s wr, 3.40k op/s 2023-07-06T20:28:11.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:11 smithi069 ceph-mon[126399]: pgmap v593: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 50 MiB/s wr, 3.80k op/s 2023-07-06T20:28:11.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:11 smithi156 ceph-mon[129280]: pgmap v593: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 50 MiB/s wr, 3.80k op/s 2023-07-06T20:28:11.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:11 smithi098 ceph-mon[128968]: pgmap v593: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 50 MiB/s wr, 3.80k op/s 2023-07-06T20:28:13.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:13 smithi069 ceph-mon[126399]: pgmap v594: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 52 MiB/s wr, 4.04k op/s 2023-07-06T20:28:13.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:13 smithi156 ceph-mon[129280]: pgmap v594: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 52 MiB/s wr, 4.04k op/s 2023-07-06T20:28:13.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:13 smithi098 ceph-mon[128968]: pgmap v594: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 52 MiB/s wr, 4.04k op/s 2023-07-06T20:28:14.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 385.0 s, 1903.5 tps, lat 52.673 ms stddev 159.859 2023-07-06T20:28:15.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:15 smithi069 ceph-mon[126399]: pgmap v595: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 59 MiB/s wr, 4.61k op/s 2023-07-06T20:28:15.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:15 smithi156 ceph-mon[129280]: pgmap v595: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 59 MiB/s wr, 4.61k op/s 2023-07-06T20:28:15.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:15 smithi098 ceph-mon[128968]: pgmap v595: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 59 MiB/s wr, 4.61k op/s 2023-07-06T20:28:17.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:17 smithi069 ceph-mon[126399]: pgmap v596: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 53 MiB/s wr, 3.92k op/s 2023-07-06T20:28:17.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:17 smithi156 ceph-mon[129280]: pgmap v596: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 53 MiB/s wr, 3.92k op/s 2023-07-06T20:28:17.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:17 smithi098 ceph-mon[128968]: pgmap v596: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 53 MiB/s wr, 3.92k op/s 2023-07-06T20:28:19.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 390.0 s, 1998.6 tps, lat 50.033 ms stddev 75.600 2023-07-06T20:28:19.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:19 smithi156 ceph-mon[129280]: pgmap v597: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 39 MiB/s wr, 2.41k op/s 2023-07-06T20:28:19.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:19 smithi098 ceph-mon[128968]: pgmap v597: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 39 MiB/s wr, 2.41k op/s 2023-07-06T20:28:19.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:19 smithi069 ceph-mon[126399]: pgmap v597: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 39 MiB/s wr, 2.41k op/s 2023-07-06T20:28:21.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:21 smithi156 ceph-mon[129280]: pgmap v598: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 MiB/s wr, 2.15k op/s 2023-07-06T20:28:21.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:21 smithi098 ceph-mon[128968]: pgmap v598: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 MiB/s wr, 2.15k op/s 2023-07-06T20:28:21.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:21 smithi069 ceph-mon[126399]: pgmap v598: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 MiB/s wr, 2.15k op/s 2023-07-06T20:28:23.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:23 smithi069 ceph-mon[126399]: pgmap v599: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 MiB/s wr, 1.72k op/s 2023-07-06T20:28:23.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:23 smithi156 ceph-mon[129280]: pgmap v599: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 MiB/s wr, 1.72k op/s 2023-07-06T20:28:23.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:23 smithi098 ceph-mon[128968]: pgmap v599: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 MiB/s wr, 1.72k op/s 2023-07-06T20:28:24.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 395.0 s, 1532.9 tps, lat 62.489 ms stddev 96.056 2023-07-06T20:28:25.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:25 smithi069 ceph-mon[126399]: pgmap v600: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 40 MiB/s wr, 2.23k op/s 2023-07-06T20:28:25.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:25 smithi156 ceph-mon[129280]: pgmap v600: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 40 MiB/s wr, 2.23k op/s 2023-07-06T20:28:25.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:25 smithi098 ceph-mon[128968]: pgmap v600: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 40 MiB/s wr, 2.23k op/s 2023-07-06T20:28:27.496 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:28:27.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:27 smithi069 ceph-mon[126399]: pgmap v601: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 41 MiB/s wr, 2.57k op/s 2023-07-06T20:28:27.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:27 smithi156 ceph-mon[129280]: pgmap v601: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 41 MiB/s wr, 2.57k op/s 2023-07-06T20:28:27.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:27 smithi098 ceph-mon[128968]: pgmap v601: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 41 MiB/s wr, 2.57k op/s 2023-07-06T20:28:27.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.877+0000 7f71c93ff700 1 -- 172.21.15.69:0/2768482278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c4076bf0 msgr2=0x7f71c4077480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:27.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.877+0000 7f71c93ff700 1 --2- 172.21.15.69:0/2768482278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c4076bf0 0x7f71c4077480 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b0a0 tx=0x7f71bc00b3b0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.877+0000 7f71c93ff700 1 -- 172.21.15.69:0/2768482278 shutdown_connections 2023-07-06T20:28:27.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.877+0000 7f71c93ff700 1 --2- 172.21.15.69:0/2768482278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c4076bf0 0x7f71c4077480 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.877+0000 7f71c93ff700 1 --2- 172.21.15.69:0/2768482278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c4075000 0x7f71c4075410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.878+0000 7f71c93ff700 1 --2- 172.21.15.69:0/2768482278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c4076240 0x7f71c40766b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.878+0000 7f71c93ff700 1 -- 172.21.15.69:0/2768482278 >> 172.21.15.69:0/2768482278 conn(0x7f71c40fd480 msgr2=0x7f71c40ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:28:27.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.878+0000 7f71c93ff700 1 -- 172.21.15.69:0/2768482278 shutdown_connections 2023-07-06T20:28:27.881 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.878+0000 7f71c93ff700 1 -- 172.21.15.69:0/2768482278 wait complete. 2023-07-06T20:28:27.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.879+0000 7f71c93ff700 1 Processor -- start 2023-07-06T20:28:27.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.879+0000 7f71c93ff700 1 -- start start 2023-07-06T20:28:27.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.879+0000 7f71c93ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c4075000 0x7f71c419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.879+0000 7f71c93ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c4076240 0x7f71c419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.882 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.879+0000 7f71c93ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c4076bf0 0x7f71c41a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.879+0000 7f71c93ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c41a3c00 con 0x7f71c4076240 2023-07-06T20:28:27.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.879+0000 7f71c93ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c41a3d40 con 0x7f71c4075000 2023-07-06T20:28:27.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.879+0000 7f71c93ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c41a3ed0 con 0x7f71c4076bf0 2023-07-06T20:28:27.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.880+0000 7f71c27fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c4076240 0x7f71c419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.880+0000 7f71c37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c4076bf0 0x7f71c41a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.880+0000 7f71c37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c4076bf0 0x7f71c41a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:56286/0 (socket says 172.21.15.69:56286) 2023-07-06T20:28:27.883 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.880+0000 7f71c37fe700 1 -- 172.21.15.69:0/843131254 learned_addr learned my addr 172.21.15.69:0/843131254 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:28:27.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.880+0000 7f71c2ffd700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c4075000 0x7f71c419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.880+0000 7f71c37fe700 1 -- 172.21.15.69:0/843131254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c4075000 msgr2=0x7f71c419c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:27.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.880+0000 7f71c37fe700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c4075000 0x7f71c419c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.884 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.880+0000 7f71c37fe700 1 -- 172.21.15.69:0/843131254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c4076240 msgr2=0x7f71c419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:27.885 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.880+0000 7f71c37fe700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c4076240 0x7f71c419c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.885 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.880+0000 7f71c37fe700 1 -- 172.21.15.69:0/843131254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc009d00 con 0x7f71c4076bf0 2023-07-06T20:28:27.885 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.881+0000 7f71c37fe700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c4076bf0 0x7f71c41a3630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009fd0 tx=0x7f71bc00bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:28:27.894 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.882+0000 7f71abfff700 1 -- 172.21.15.69:0/843131254 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00e040 con 0x7f71c4076bf0 2023-07-06T20:28:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.882+0000 7f71abfff700 1 -- 172.21.15.69:0/843131254 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71bc0094a0 con 0x7f71c4076bf0 2023-07-06T20:28:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.882+0000 7f71abfff700 1 -- 172.21.15.69:0/843131254 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc01b6c0 con 0x7f71c4076bf0 2023-07-06T20:28:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.882+0000 7f71c93ff700 1 -- 172.21.15.69:0/843131254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c41a4150 con 0x7f71c4076bf0 2023-07-06T20:28:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.882+0000 7f71c93ff700 1 -- 172.21.15.69:0/843131254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c41a4520 con 0x7f71c4076bf0 2023-07-06T20:28:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.889+0000 7f71dcc5bb80 1 Processor -- start 2023-07-06T20:28:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.890+0000 7f71dcc5bb80 1 -- start start 2023-07-06T20:28:27.896 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.890+0000 7f71dcc5bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88df30 0x560e9f88e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.896 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.890+0000 7f71dcc5bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88e880 0x560e9f88ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.896 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.890+0000 7f71dcc5bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f88f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.896 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.890+0000 7f71dcc5bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9f781cb0 con 0x560e9f88df30 2023-07-06T20:28:27.896 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.890+0000 7f71dcc5bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9f7d4a40 con 0x560e9f88e880 2023-07-06T20:28:27.896 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.890+0000 7f71dcc5bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9f7cded0 con 0x560e9f88f3a0 2023-07-06T20:28:27.896 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.892+0000 7f71c2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88e880 0x560e9f88ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.892+0000 7f71c2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88e880 0x560e9f88ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:56016/0 (socket says 172.21.15.69:56016) 2023-07-06T20:28:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.892+0000 7f71c2ffd700 1 -- 172.21.15.69:0/2722953495 learned_addr learned my addr 172.21.15.69:0/2722953495 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:28:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.893+0000 7f71c27fc700 1 --2- 172.21.15.69:0/2722953495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f88f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.893+0000 7f71c27fc700 1 --2- 172.21.15.69:0/2722953495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88df30 0x560e9f88e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.893+0000 7f71c27fc700 1 -- 172.21.15.69:0/2722953495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88e880 msgr2=0x560e9f88ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.893+0000 7f71c27fc700 1 --2- 172.21.15.69:0/2722953495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88e880 0x560e9f88ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.893+0000 7f71c27fc700 1 -- 172.21.15.69:0/2722953495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88df30 msgr2=0x560e9f88e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.893+0000 7f71c27fc700 1 --2- 172.21.15.69:0/2722953495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88df30 0x560e9f88e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.893+0000 7f71c27fc700 1 -- 172.21.15.69:0/2722953495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e9f5fbde0 con 0x560e9f88f3a0 2023-07-06T20:28:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.894+0000 7f71c2ffd700 1 --2- 172.21.15.69:0/2722953495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88e880 0x560e9f88ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:28:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.894+0000 7f71c27fc700 1 --2- 172.21.15.69:0/2722953495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f88f840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f71b8010970 tx=0x7f71b8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5c8ffb646f6c3da server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:28:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.894+0000 7f71a97fa700 1 -- 172.21.15.69:0/2722953495 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8004a30 con 0x560e9f88f3a0 2023-07-06T20:28:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.894+0000 7f71a97fa700 1 -- 172.21.15.69:0/2722953495 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71b80054e0 con 0x560e9f88f3a0 2023-07-06T20:28:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.894+0000 7f71a97fa700 1 -- 172.21.15.69:0/2722953495 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800f480 con 0x560e9f88f3a0 2023-07-06T20:28:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.894+0000 7f71abfff700 1 -- 172.21.15.69:0/843131254 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f71bc01d470 con 0x7f71c4076bf0 2023-07-06T20:28:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.895+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2722953495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 msgr2=0x560e9f88f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.895+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/2722953495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f88f840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f71b8010970 tx=0x7f71b8004750 comp rx=0 tx=0).stop 2023-07-06T20:28:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.895+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2722953495 shutdown_connections 2023-07-06T20:28:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.895+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/2722953495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f88f840 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.895+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/2722953495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88df30 0x560e9f88e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.895+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/2722953495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88e880 0x560e9f88ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.901 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.895+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2722953495 >> 172.21.15.69:0/2722953495 conn(0x560e9f872640 msgr2=0x560e9f877930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:28:27.901 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.896+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2722953495 shutdown_connections 2023-07-06T20:28:27.901 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.896+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2722953495 wait complete. 2023-07-06T20:28:27.903 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.896+0000 7f71dcc5bb80 1 -- start start 2023-07-06T20:28:27.903 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.897+0000 7f71abfff700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71ac0a1160 0x7f71ac0a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.903 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.897+0000 7f71dcc5bb80 1 Processor -- start 2023-07-06T20:28:27.903 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.897+0000 7f71abfff700 1 -- 172.21.15.69:0/843131254 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f71bc0cda40 con 0x7f71c4076bf0 2023-07-06T20:28:27.903 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.897+0000 7f71c27fc700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71ac0a1160 0x7f71ac0a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.903 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.897+0000 7f71dcc5bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88df30 0x560e9f884640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.897+0000 7f71dcc5bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88e880 0x560e9f884b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.897+0000 7f71dcc5bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f88bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.897+0000 7f71dcc5bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9f76dac0 con 0x560e9f88e880 2023-07-06T20:28:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.897+0000 7f71dcc5bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9f76d7c0 con 0x560e9f88df30 2023-07-06T20:28:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.897+0000 7f71dcc5bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9f651920 con 0x560e9f88f3a0 2023-07-06T20:28:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.897+0000 7f71dcc5bb80 10 client.0 _ll_register_callbacks cb 0x560e9f76b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:28:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.898+0000 7f71c37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f88bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.898+0000 7f71c37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f88bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:56308/0 (socket says 172.21.15.69:56308) 2023-07-06T20:28:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.898+0000 7f71c37fe700 1 -- 172.21.15.69:0/3007241950 learned_addr learned my addr 172.21.15.69:0/3007241950 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:28:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.898+0000 7f71c2ffd700 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88df30 0x560e9f884640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.898+0000 7f71c27fc700 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88e880 0x560e9f884b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.898+0000 7f71c37fe700 1 -- 172.21.15.69:0/3007241950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88df30 msgr2=0x560e9f884640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.898+0000 7f71c37fe700 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88df30 0x560e9f884640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.905 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.898+0000 7f71c37fe700 1 -- 172.21.15.69:0/3007241950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88e880 msgr2=0x560e9f884b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:27.906 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.898+0000 7f71c37fe700 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88e880 0x560e9f884b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.906 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.898+0000 7f71c37fe700 1 -- 172.21.15.69:0/3007241950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b40099e0 con 0x560e9f88f3a0 2023-07-06T20:28:27.906 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.898+0000 7f71c37fe700 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f88bbc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f71bc09ae40 tx=0x7f71bc09ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af78d8710fb324d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:28:27.906 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.898+0000 7f71a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:28:27.906 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71a6ff5700 1 -- 172.21.15.69:0/3007241950 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc09a5a0 con 0x560e9f88f3a0 2023-07-06T20:28:27.906 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71a6ff5700 1 -- 172.21.15.69:0/3007241950 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f71bc09aeb0 con 0x560e9f88f3a0 2023-07-06T20:28:27.906 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/3007241950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e9f7274f0 con 0x560e9f88f3a0 2023-07-06T20:28:27.906 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71a6ff5700 1 -- 172.21.15.69:0/3007241950 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71bc09eee0 con 0x560e9f88f3a0 2023-07-06T20:28:27.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71dcc5bb80 10 client.25657 fetch_fsmap learned FSMap version 30 2023-07-06T20:28:27.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/3007241950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f71b8010620 con 0x560e9f88f3a0 2023-07-06T20:28:27.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71c27fc700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71ac0a1160 0x7f71ac0a3610 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f71b8010940 tx=0x7f71b8009350 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:28:27.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71a6ff5700 1 -- 172.21.15.69:0/3007241950 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e9f7274f0 con 0x560e9f88f3a0 2023-07-06T20:28:27.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71a6ff5700 1 -- 172.21.15.69:0/3007241950 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f71bc01d820 con 0x560e9f88f3a0 2023-07-06T20:28:27.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71dcc5bb80 10 client.25657 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:28:27.907 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71dcc5bb80 10 client.25657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:28:27.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71dcc5bb80 20 client.25657 populate_metadata read hostname 'smithi069' 2023-07-06T20:28:27.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x560e9f8b43e0 0x560e9f8b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71dcc5bb80 4 client.25657 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:28:27.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.899+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/3007241950 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e9f84bd60 con 0x560e9f8b43e0 2023-07-06T20:28:27.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.900+0000 7f71c2ffd700 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x560e9f8b43e0 0x560e9f8b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.900+0000 7f71a6ff5700 10 client.25657 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:28:27.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.900+0000 7f71c2ffd700 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x560e9f8b43e0 0x560e9f8b67f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b194e55c8680ae7 server_cookie=2e809e2f42207995 in_seq=0 out_seq=0 2023-07-06T20:28:27.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.901+0000 7f71a6ff5700 1 -- 172.21.15.69:0/3007241950 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560e9f84bd60 con 0x560e9f8b43e0 2023-07-06T20:28:27.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.901+0000 7f71a6ff5700 10 client.25657 handle_command_reply: tid=0 2023-07-06T20:28:27.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.901+0000 7f71dcc5bb80 1 client.25657 shutdown 2023-07-06T20:28:27.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.901+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/3007241950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 msgr2=0x560e9f88bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:27.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.901+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f88bbc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f71bc09ae40 tx=0x7f71bc09ac80 comp rx=0 tx=0).stop 2023-07-06T20:28:27.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.901+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/3007241950 shutdown_connections 2023-07-06T20:28:27.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.901+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x560e9f8b43e0 0x560e9f8b67f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.901+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f88bbc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.901+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88df30 0x560e9f884640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.901+0000 7f71a6ff5700 0 client.25657 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:28:27.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.901+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/3007241950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88e880 0x560e9f884b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.901+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/3007241950 >> 172.21.15.69:0/3007241950 conn(0x560e9f872640 msgr2=0x560e9f8770b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:28:27.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.902+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/3007241950 shutdown_connections 2023-07-06T20:28:27.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.902+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/3007241950 wait complete. 2023-07-06T20:28:27.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.902+0000 7f71dcc5bb80 20 client.25657 trim_cache size 0 max 16384 2023-07-06T20:28:27.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.928+0000 7f71dcc5bb80 1 Processor -- start 2023-07-06T20:28:27.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.928+0000 7f71dcc5bb80 1 -- start start 2023-07-06T20:28:27.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.928+0000 7f71dcc5bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88df30 0x560e9f8bb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.928+0000 7f71dcc5bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88e880 0x560e9f87dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.929+0000 7f71dcc5bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88f3a0 0x560e9f8770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.929+0000 7f71dcc5bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9f8841b0 con 0x560e9f88df30 2023-07-06T20:28:27.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.929+0000 7f71dcc5bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9f8793b0 con 0x560e9f88f3a0 2023-07-06T20:28:27.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.929+0000 7f71dcc5bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9f88ec50 con 0x560e9f88e880 2023-07-06T20:28:27.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.929+0000 7f71c37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88f3a0 0x560e9f8770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.929+0000 7f71c2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88df30 0x560e9f8bb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.929+0000 7f71c37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88f3a0 0x560e9f8770b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:56032/0 (socket says 172.21.15.69:56032) 2023-07-06T20:28:27.933 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.929+0000 7f71c37fe700 1 -- 172.21.15.69:0/507904498 learned_addr learned my addr 172.21.15.69:0/507904498 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:28:27.933 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.930+0000 7f71c37fe700 1 -- 172.21.15.69:0/507904498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88e880 msgr2=0x560e9f87dc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:28:27.933 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.930+0000 7f71c37fe700 1 --2- 172.21.15.69:0/507904498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88e880 0x560e9f87dc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.933 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.930+0000 7f71c37fe700 1 -- 172.21.15.69:0/507904498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88df30 msgr2=0x560e9f8bb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:27.933 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.930+0000 7f71c37fe700 1 --2- 172.21.15.69:0/507904498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88df30 0x560e9f8bb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.930+0000 7f71c37fe700 1 -- 172.21.15.69:0/507904498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e9f5e4c60 con 0x560e9f88f3a0 2023-07-06T20:28:27.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.930+0000 7f71c37fe700 1 --2- 172.21.15.69:0/507904498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88f3a0 0x560e9f8770b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0cb560 tx=0x7f71bc09b3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83e692539d89597a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:28:27.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.931+0000 7f71a9ffb700 1 -- 172.21.15.69:0/507904498 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0a2fe0 con 0x560e9f88f3a0 2023-07-06T20:28:27.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.931+0000 7f71a9ffb700 1 -- 172.21.15.69:0/507904498 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71bc0a0040 con 0x560e9f88f3a0 2023-07-06T20:28:27.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.931+0000 7f71a9ffb700 1 -- 172.21.15.69:0/507904498 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc09a600 con 0x560e9f88f3a0 2023-07-06T20:28:27.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.932+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/507904498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88f3a0 msgr2=0x560e9f8770b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:27.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.932+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/507904498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88f3a0 0x560e9f8770b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0cb560 tx=0x7f71bc09b3f0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.932+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/507904498 shutdown_connections 2023-07-06T20:28:27.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.932+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/507904498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88f3a0 0x560e9f8770b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.932+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/507904498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88df30 0x560e9f8bb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.932+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/507904498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88e880 0x560e9f87dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.935 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.932+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/507904498 >> 172.21.15.69:0/507904498 conn(0x560e9f872640 msgr2=0x560e9f877900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:28:27.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.932+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/507904498 shutdown_connections 2023-07-06T20:28:27.936 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.934+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/507904498 wait complete. 2023-07-06T20:28:27.937 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.935+0000 7f71dcc5bb80 1 -- start start 2023-07-06T20:28:27.937 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.935+0000 7f71dcc5bb80 1 Processor -- start 2023-07-06T20:28:27.938 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.936+0000 7f71dcc5bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88df30 0x560e9f902410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.938 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.936+0000 7f71dcc5bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88e880 0x560e9f904960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.936+0000 7f71dcc5bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f906f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.936+0000 7f71dcc5bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9f907650 con 0x560e9f88e880 2023-07-06T20:28:27.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.936+0000 7f71dcc5bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9f9077c0 con 0x560e9f88df30 2023-07-06T20:28:27.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.936+0000 7f71dcc5bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9f907980 con 0x560e9f88f3a0 2023-07-06T20:28:27.939 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.936+0000 7f71dcc5bb80 10 client.0 _ll_register_callbacks cb 0x7f71bc009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:28:27.940 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.936+0000 7f71c2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88df30 0x560e9f902410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.940 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.936+0000 7f71c2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88df30 0x560e9f902410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:56036/0 (socket says 172.21.15.69:56036) 2023-07-06T20:28:27.940 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.936+0000 7f71c2ffd700 1 -- 172.21.15.69:0/2265914601 learned_addr learned my addr 172.21.15.69:0/2265914601 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:28:27.940 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.937+0000 7f71c2ffd700 1 -- 172.21.15.69:0/2265914601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 msgr2=0x560e9f906f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:28:27.940 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.937+0000 7f71c2ffd700 1 --2- 172.21.15.69:0/2265914601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f906f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.941 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.937+0000 7f71c2ffd700 1 -- 172.21.15.69:0/2265914601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88e880 msgr2=0x560e9f904960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:27.941 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.937+0000 7f71c2ffd700 1 --2- 172.21.15.69:0/2265914601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88e880 0x560e9f904960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.941 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.937+0000 7f71c2ffd700 1 -- 172.21.15.69:0/2265914601 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc0b2030 con 0x560e9f88df30 2023-07-06T20:28:27.941 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.937+0000 7f71c2ffd700 1 --2- 172.21.15.69:0/2265914601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88df30 0x560e9f902410 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x560e9f8b70c0 tx=0x7f71b40128b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e86bbd525109703e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:28:27.941 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.937+0000 7f71a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:28:27.941 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.938+0000 7f71a6ff5700 1 -- 172.21.15.69:0/2265914601 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4017040 con 0x560e9f88df30 2023-07-06T20:28:27.941 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.938+0000 7f71a6ff5700 1 -- 172.21.15.69:0/2265914601 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f71b40210b0 con 0x560e9f88df30 2023-07-06T20:28:27.941 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.938+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2265914601 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71b80054e0 con 0x560e9f88df30 2023-07-06T20:28:27.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.938+0000 7f71a6ff5700 1 -- 172.21.15.69:0/2265914601 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71b40122d0 con 0x560e9f88df30 2023-07-06T20:28:27.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.938+0000 7f71dcc5bb80 10 client.25706 fetch_fsmap learned FSMap version 30 2023-07-06T20:28:27.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.938+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2265914601 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560e9f622250 con 0x560e9f88df30 2023-07-06T20:28:27.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.938+0000 7f71a6ff5700 1 -- 172.21.15.69:0/2265914601 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71b80054e0 con 0x560e9f88df30 2023-07-06T20:28:27.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.939+0000 7f71a6ff5700 1 -- 172.21.15.69:0/2265914601 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f71b4010430 con 0x560e9f88df30 2023-07-06T20:28:27.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.939+0000 7f71dcc5bb80 10 client.25706 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:28:27.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.939+0000 7f71dcc5bb80 10 client.25706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:28:27.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.939+0000 7f71dcc5bb80 20 client.25706 populate_metadata read hostname 'smithi069' 2023-07-06T20:28:27.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.939+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/2265914601 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x560e9f909310 0x560e9f90b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:27.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.939+0000 7f71dcc5bb80 4 client.25706 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:28:27.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.939+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2265914601 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560e9f7c6eb0 con 0x560e9f909310 2023-07-06T20:28:27.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.940+0000 7f71c27fc700 1 --2- 172.21.15.69:0/2265914601 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x560e9f909310 0x560e9f90b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:27.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.940+0000 7f71c27fc700 1 --2- 172.21.15.69:0/2265914601 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x560e9f909310 0x560e9f90b6f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e6c61b04152f5f9 server_cookie=ccc187fbb88a63b7 in_seq=0 out_seq=0 2023-07-06T20:28:27.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.940+0000 7f71a6ff5700 10 client.25706 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:28:27.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.940+0000 7f71a6ff5700 1 -- 172.21.15.69:0/2265914601 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560e9f7c6eb0 con 0x560e9f909310 2023-07-06T20:28:27.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.940+0000 7f71a6ff5700 10 client.25706 handle_command_reply: tid=0 2023-07-06T20:28:27.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.940+0000 7f71dcc5bb80 1 client.25706 shutdown 2023-07-06T20:28:27.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.941+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2265914601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88df30 msgr2=0x560e9f902410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:27.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.941+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/2265914601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88df30 0x560e9f902410 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x560e9f8b70c0 tx=0x7f71b40128b0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.941+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2265914601 shutdown_connections 2023-07-06T20:28:27.945 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.941+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/2265914601 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x560e9f909310 0x560e9f90b6f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.946 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.941+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/2265914601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560e9f88f3a0 0x560e9f906f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.946 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.941+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/2265914601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560e9f88df30 0x560e9f902410 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.946 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.941+0000 7f71a6ff5700 0 client.25706 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:28:27.946 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.941+0000 7f71dcc5bb80 1 --2- 172.21.15.69:0/2265914601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9f88e880 0x560e9f904960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.946 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.941+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2265914601 >> 172.21.15.69:0/2265914601 conn(0x560e9f872640 msgr2=0x560e9f88c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:28:27.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.941+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2265914601 shutdown_connections 2023-07-06T20:28:27.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.941+0000 7f71dcc5bb80 1 -- 172.21.15.69:0/2265914601 wait complete. 2023-07-06T20:28:27.947 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.941+0000 7f71dcc5bb80 20 client.25706 trim_cache size 0 max 16384 2023-07-06T20:28:27.947 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:28:27.947 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:28:27.948 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:28:27.948 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:28:27.948 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:28:27.948 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:28:27.948 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:28:27.948 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:28:27.948 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:28:27.948 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:28:27.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.942+0000 7f71c93ff700 1 -- 172.21.15.69:0/843131254 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71ac0a1160 msgr2=0x7f71ac0a3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:27.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.942+0000 7f71c93ff700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71ac0a1160 0x7f71ac0a3610 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f71b8010940 tx=0x7f71b8009350 comp rx=0 tx=0).stop 2023-07-06T20:28:27.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.942+0000 7f71c93ff700 1 -- 172.21.15.69:0/843131254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c4076bf0 msgr2=0x7f71c41a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:27.949 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.942+0000 7f71c93ff700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c4076bf0 0x7f71c41a3630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009fd0 tx=0x7f71bc00bce0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.950 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.942+0000 7f71c93ff700 1 -- 172.21.15.69:0/843131254 shutdown_connections 2023-07-06T20:28:27.950 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.942+0000 7f71c93ff700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71ac0a1160 0x7f71ac0a3610 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.950 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.942+0000 7f71c93ff700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71c4076bf0 0x7f71c41a3630 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.950 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.942+0000 7f71c93ff700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c4075000 0x7f71c419c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.950 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.942+0000 7f71c93ff700 1 --2- 172.21.15.69:0/843131254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71c4076240 0x7f71c419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:27.950 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.942+0000 7f71c93ff700 1 -- 172.21.15.69:0/843131254 >> 172.21.15.69:0/843131254 conn(0x7f71c40fd480 msgr2=0x7f71c40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:28:27.950 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.942+0000 7f71c93ff700 1 -- 172.21.15.69:0/843131254 shutdown_connections 2023-07-06T20:28:27.951 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:27.942+0000 7f71c93ff700 1 -- 172.21.15.69:0/843131254 wait complete. 2023-07-06T20:28:27.972 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:28:29.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 400.0 s, 2517.9 tps, lat 41.401 ms stddev 42.088 2023-07-06T20:28:29.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:29 smithi156 ceph-mon[129280]: pgmap v602: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 40 MiB/s wr, 2.59k op/s 2023-07-06T20:28:29.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:29 smithi098 ceph-mon[128968]: pgmap v602: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 40 MiB/s wr, 2.59k op/s 2023-07-06T20:28:30.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:29 smithi069 ceph-mon[126399]: pgmap v602: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 40 MiB/s wr, 2.59k op/s 2023-07-06T20:28:31.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:31 smithi156 ceph-mon[129280]: pgmap v603: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 MiB/s wr, 2.86k op/s 2023-07-06T20:28:31.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:31 smithi098 ceph-mon[128968]: pgmap v603: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 MiB/s wr, 2.86k op/s 2023-07-06T20:28:32.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:31 smithi069 ceph-mon[126399]: pgmap v603: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 MiB/s wr, 2.86k op/s 2023-07-06T20:28:33.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:33 smithi156 ceph-mon[129280]: pgmap v604: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 MiB/s wr, 2.75k op/s 2023-07-06T20:28:33.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:33 smithi098 ceph-mon[128968]: pgmap v604: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 MiB/s wr, 2.75k op/s 2023-07-06T20:28:34.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:33 smithi069 ceph-mon[126399]: pgmap v604: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 MiB/s wr, 2.75k op/s 2023-07-06T20:28:34.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 405.0 s, 2238.2 tps, lat 44.455 ms stddev 61.665 2023-07-06T20:28:35.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:35 smithi156 ceph-mon[129280]: pgmap v605: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 46 MiB/s wr, 2.84k op/s 2023-07-06T20:28:35.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:35 smithi098 ceph-mon[128968]: pgmap v605: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 46 MiB/s wr, 2.84k op/s 2023-07-06T20:28:36.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:35 smithi069 ceph-mon[126399]: pgmap v605: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 46 MiB/s wr, 2.84k op/s 2023-07-06T20:28:38.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:37 smithi069 ceph-mon[126399]: pgmap v606: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 MiB/s wr, 2.10k op/s 2023-07-06T20:28:38.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:37 smithi156 ceph-mon[129280]: pgmap v606: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 MiB/s wr, 2.10k op/s 2023-07-06T20:28:38.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:37 smithi098 ceph-mon[128968]: pgmap v606: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 MiB/s wr, 2.10k op/s 2023-07-06T20:28:39.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 410.0 s, 2616.4 tps, lat 38.425 ms stddev 26.954 2023-07-06T20:28:40.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:39 smithi156 ceph-mon[129280]: pgmap v607: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 34 MiB/s wr, 1.19k op/s 2023-07-06T20:28:40.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:39 smithi098 ceph-mon[128968]: pgmap v607: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 34 MiB/s wr, 1.19k op/s 2023-07-06T20:28:40.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:39 smithi069 ceph-mon[126399]: pgmap v607: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 34 MiB/s wr, 1.19k op/s 2023-07-06T20:28:42.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:41 smithi156 ceph-mon[129280]: pgmap v608: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 1.08k op/s 2023-07-06T20:28:42.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:41 smithi098 ceph-mon[128968]: pgmap v608: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 1.08k op/s 2023-07-06T20:28:42.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:41 smithi069 ceph-mon[126399]: pgmap v608: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 1.08k op/s 2023-07-06T20:28:44.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:43 smithi156 ceph-mon[129280]: pgmap v609: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 MiB/s wr, 1.43k op/s 2023-07-06T20:28:44.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:43 smithi098 ceph-mon[128968]: pgmap v609: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 MiB/s wr, 1.43k op/s 2023-07-06T20:28:44.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 415.0 s, 790.4 tps, lat 126.438 ms stddev 208.323 2023-07-06T20:28:44.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:43 smithi069 ceph-mon[126399]: pgmap v609: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 MiB/s wr, 1.43k op/s 2023-07-06T20:28:46.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:45 smithi156 ceph-mon[129280]: pgmap v610: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 47 MiB/s wr, 2.83k op/s 2023-07-06T20:28:46.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:45 smithi098 ceph-mon[128968]: pgmap v610: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 47 MiB/s wr, 2.83k op/s 2023-07-06T20:28:46.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:45 smithi069 ceph-mon[126399]: pgmap v610: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 47 MiB/s wr, 2.83k op/s 2023-07-06T20:28:47.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:46 smithi098 ceph-mon[128968]: pgmap v611: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 43 MiB/s wr, 3.13k op/s 2023-07-06T20:28:47.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:46 smithi069 ceph-mon[126399]: pgmap v611: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 43 MiB/s wr, 3.13k op/s 2023-07-06T20:28:47.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:46 smithi156 ceph-mon[129280]: pgmap v611: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 43 MiB/s wr, 3.13k op/s 2023-07-06T20:28:49.232 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 420.0 s, 2477.5 tps, lat 40.331 ms stddev 39.885 2023-07-06T20:28:49.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:49 smithi069 ceph-mon[126399]: pgmap v612: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 MiB/s wr, 3.74k op/s 2023-07-06T20:28:49.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:49 smithi156 ceph-mon[129280]: pgmap v612: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 MiB/s wr, 3.74k op/s 2023-07-06T20:28:49.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:49 smithi098 ceph-mon[128968]: pgmap v612: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 MiB/s wr, 3.74k op/s 2023-07-06T20:28:51.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:51 smithi069 ceph-mon[126399]: pgmap v613: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 49 MiB/s wr, 3.75k op/s 2023-07-06T20:28:51.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:51 smithi156 ceph-mon[129280]: pgmap v613: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 49 MiB/s wr, 3.75k op/s 2023-07-06T20:28:51.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:51 smithi098 ceph-mon[128968]: pgmap v613: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 49 MiB/s wr, 3.75k op/s 2023-07-06T20:28:53.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:53 smithi069 ceph-mon[126399]: pgmap v614: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 MiB/s wr, 3.62k op/s 2023-07-06T20:28:53.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:53 smithi156 ceph-mon[129280]: pgmap v614: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 MiB/s wr, 3.62k op/s 2023-07-06T20:28:53.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:53 smithi098 ceph-mon[128968]: pgmap v614: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 MiB/s wr, 3.62k op/s 2023-07-06T20:28:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 425.0 s, 2052.8 tps, lat 48.440 ms stddev 64.167 2023-07-06T20:28:55.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:55 smithi069 ceph-mon[126399]: pgmap v615: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 45 MiB/s wr, 3.07k op/s 2023-07-06T20:28:55.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:55 smithi156 ceph-mon[129280]: pgmap v615: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 45 MiB/s wr, 3.07k op/s 2023-07-06T20:28:55.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:55 smithi098 ceph-mon[128968]: pgmap v615: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 45 MiB/s wr, 3.07k op/s 2023-07-06T20:28:56.513 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:56 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:28:56.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:56 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:28:56.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:56 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:28:57.515 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:57 smithi069 ceph-mon[126399]: pgmap v616: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.67k op/s 2023-07-06T20:28:57.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:28:57.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:28:57.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:28:57.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:57 smithi156 ceph-mon[129280]: pgmap v616: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.67k op/s 2023-07-06T20:28:57.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:28:57.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:28:57.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:28:57.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:57 smithi098 ceph-mon[128968]: pgmap v616: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.67k op/s 2023-07-06T20:28:57.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:28:57.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:28:57.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:28:57.974 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:28:58.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.442+0000 7f99c45ff700 1 -- 172.21.15.69:0/2106251465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99bc1078b0 msgr2=0x7f99bc107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:58.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.442+0000 7f99c45ff700 1 --2- 172.21.15.69:0/2106251465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99bc1078b0 0x7f99bc107cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009990 tx=0x7f99b8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.445 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.442+0000 7f99c45ff700 1 -- 172.21.15.69:0/2106251465 shutdown_connections 2023-07-06T20:28:58.446 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.442+0000 7f99c45ff700 1 --2- 172.21.15.69:0/2106251465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99bc071d50 0x7f99bc10c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.446 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.442+0000 7f99c45ff700 1 --2- 172.21.15.69:0/2106251465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99bc1078b0 0x7f99bc107cc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.446 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.442+0000 7f99c45ff700 1 --2- 172.21.15.69:0/2106251465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99bc108200 0x7f99bc0716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.447 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.442+0000 7f99c45ff700 1 -- 172.21.15.69:0/2106251465 >> 172.21.15.69:0/2106251465 conn(0x7f99bc06d4d0 msgr2=0x7f99bc06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:28:58.447 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.443+0000 7f99c45ff700 1 -- 172.21.15.69:0/2106251465 shutdown_connections 2023-07-06T20:28:58.447 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.443+0000 7f99c45ff700 1 -- 172.21.15.69:0/2106251465 wait complete. 2023-07-06T20:28:58.448 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.444+0000 7f99c45ff700 1 Processor -- start 2023-07-06T20:28:58.448 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.445+0000 7f99c45ff700 1 -- start start 2023-07-06T20:28:58.448 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.445+0000 7f99c45ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99bc071d50 0x7f99bc112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.448 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.445+0000 7f99c45ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99bc1078b0 0x7f99bc112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.449 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.446+0000 7f99c45ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99bc108200 0x7f99bc1b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.449 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.446+0000 7f99c45ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc1160d0 con 0x7f99bc108200 2023-07-06T20:28:58.449 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.446+0000 7f99c45ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc116240 con 0x7f99bc1078b0 2023-07-06T20:28:58.449 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.446+0000 7f99c45ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc1b4ed0 con 0x7f99bc071d50 2023-07-06T20:28:58.449 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.446+0000 7f99c1b9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99bc1078b0 0x7f99bc112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.446+0000 7f99c1b9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99bc1078b0 0x7f99bc112ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:60810/0 (socket says 172.21.15.69:60810) 2023-07-06T20:28:58.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.446+0000 7f99c1b9a700 1 -- 172.21.15.69:0/250085033 learned_addr learned my addr 172.21.15.69:0/250085033 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:28:58.450 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.447+0000 7f99c1b9a700 1 -- 172.21.15.69:0/250085033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99bc071d50 msgr2=0x7f99bc112ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T20:28:58.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.447+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/250085033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99bc071d50 0x7f99bc112ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.447+0000 7f99c1b9a700 1 -- 172.21.15.69:0/250085033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99bc108200 msgr2=0x7f99bc1b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:58.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.447+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/250085033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99bc108200 0x7f99bc1b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.447+0000 7f99c1b9a700 1 -- 172.21.15.69:0/250085033 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b80095f0 con 0x7f99bc1078b0 2023-07-06T20:28:58.451 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.447+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/250085033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99bc1078b0 0x7f99bc112ff0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00d980 tx=0x7f99ac00dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:28:58.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.448+0000 7f99b37fe700 1 -- 172.21.15.69:0/250085033 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac009ae0 con 0x7f99bc1078b0 2023-07-06T20:28:58.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.448+0000 7f99b37fe700 1 -- 172.21.15.69:0/250085033 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ac00de60 con 0x7f99bc1078b0 2023-07-06T20:28:58.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.448+0000 7f99c45ff700 1 -- 172.21.15.69:0/250085033 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99bc1b5130 con 0x7f99bc1078b0 2023-07-06T20:28:58.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.448+0000 7f99b37fe700 1 -- 172.21.15.69:0/250085033 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac00f620 con 0x7f99bc1078b0 2023-07-06T20:28:58.452 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.448+0000 7f99c45ff700 1 -- 172.21.15.69:0/250085033 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99bc1b5660 con 0x7f99bc1078b0 2023-07-06T20:28:58.459 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.456+0000 7f99d7fb0b80 1 Processor -- start 2023-07-06T20:28:58.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.459+0000 7f99b37fe700 1 -- 172.21.15.69:0/250085033 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f99ac010460 con 0x7f99bc1078b0 2023-07-06T20:28:58.466 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.461+0000 7f99b37fe700 1 --2- 172.21.15.69:0/250085033 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f99a80a12a0 0x7f99a80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.461+0000 7f99b37fe700 1 -- 172.21.15.69:0/250085033 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f99ac0cc4d0 con 0x7f99bc1078b0 2023-07-06T20:28:58.467 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.465+0000 7f99c2b9c700 1 --2- 172.21.15.69:0/250085033 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f99a80a12a0 0x7f99a80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.468 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.466+0000 7f99d7fb0b80 1 -- start start 2023-07-06T20:28:58.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.466+0000 7f99d7fb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 0x55aefccc7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.466+0000 7f99d7fb0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc7870 0x55aefccc7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.469 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.466+0000 7f99d7fb0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc8390 0x55aefccc8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.466+0000 7f99d7fb0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefcbcbcb0 con 0x55aefccc7870 2023-07-06T20:28:58.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.466+0000 7f99d7fb0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefcc1ea40 con 0x55aefccc8390 2023-07-06T20:28:58.470 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.466+0000 7f99d7fb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefcc17ed0 con 0x55aefccc6f20 2023-07-06T20:28:58.471 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.466+0000 7f99c239b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 0x55aefccc7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.466+0000 7f99c239b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 0x55aefccc7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:51570/0 (socket says 172.21.15.69:51570) 2023-07-06T20:28:58.472 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.466+0000 7f99c239b700 1 -- 172.21.15.69:0/1724645128 learned_addr learned my addr 172.21.15.69:0/1724645128 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:28:58.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.467+0000 7f99c2b9c700 1 --2- 172.21.15.69:0/250085033 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f99a80a12a0 0x7f99a80a3750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f99b400e350 tx=0x7f99b400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:28:58.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.467+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/1724645128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc7870 0x55aefccc7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.467+0000 7f99c2b9c700 1 --2- 172.21.15.69:0/1724645128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc8390 0x55aefccc8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.473 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.468+0000 7f99c1b9a700 1 -- 172.21.15.69:0/1724645128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 msgr2=0x55aefccc7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:58.474 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.468+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/1724645128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 0x55aefccc7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.474 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.468+0000 7f99c1b9a700 1 -- 172.21.15.69:0/1724645128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc8390 msgr2=0x55aefccc8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:58.474 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.468+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/1724645128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc8390 0x55aefccc8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.474 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.468+0000 7f99c1b9a700 1 -- 172.21.15.69:0/1724645128 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aefca45de0 con 0x55aefccc7870 2023-07-06T20:28:58.474 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.469+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/1724645128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc7870 0x55aefccc7e00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00dfd0 tx=0x7f99ac009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0de53d0a52b5281 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:28:58.475 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.469+0000 7f99b0bfe700 1 -- 172.21.15.69:0/1724645128 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac09d880 con 0x55aefccc7870 2023-07-06T20:28:58.475 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.469+0000 7f99b0bfe700 1 -- 172.21.15.69:0/1724645128 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ac0a1ca0 con 0x55aefccc7870 2023-07-06T20:28:58.476 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.469+0000 7f99b0bfe700 1 -- 172.21.15.69:0/1724645128 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac099d10 con 0x55aefccc7870 2023-07-06T20:28:58.476 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.470+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/1724645128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc7870 msgr2=0x55aefccc7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:58.476 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.470+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/1724645128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc7870 0x55aefccc7e00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00dfd0 tx=0x7f99ac009d60 comp rx=0 tx=0).stop 2023-07-06T20:28:58.476 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.473+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/1724645128 shutdown_connections 2023-07-06T20:28:58.476 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.473+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/1724645128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc8390 0x55aefccc8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.477 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.473+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/1724645128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 0x55aefccc7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.477 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.473+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/1724645128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc7870 0x55aefccc7e00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.477 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.473+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/1724645128 >> 172.21.15.69:0/1724645128 conn(0x55aefccbc640 msgr2=0x55aefccc19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:28:58.477 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.473+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/1724645128 shutdown_connections 2023-07-06T20:28:58.478 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.473+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/1724645128 wait complete. 2023-07-06T20:28:58.478 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.474+0000 7f99d7fb0b80 1 -- start start 2023-07-06T20:28:58.478 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.474+0000 7f99d7fb0b80 1 Processor -- start 2023-07-06T20:28:58.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.475+0000 7f99d7fb0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc6f20 0x55aefccd0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.475+0000 7f99d7fb0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.475+0000 7f99d7fb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc8390 0x55aefccd7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.475+0000 7f99d7fb0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefcbb7ac0 con 0x55aefccc6f20 2023-07-06T20:28:58.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.475+0000 7f99d7fb0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefcbb77c0 con 0x55aefccc7870 2023-07-06T20:28:58.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.475+0000 7f99d7fb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefca9b920 con 0x55aefccc8390 2023-07-06T20:28:58.479 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.475+0000 7f99d7fb0b80 10 client.0 _ll_register_callbacks cb 0x55aefcbb5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:28:58.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.475+0000 7f99c239b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc6f20 0x55aefccd0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.475+0000 7f99c1b9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.475+0000 7f99c239b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc6f20 0x55aefccd0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34700/0 (socket says 172.21.15.69:34700) 2023-07-06T20:28:58.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.475+0000 7f99c239b700 1 -- 172.21.15.69:0/352860666 learned_addr learned my addr 172.21.15.69:0/352860666 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:28:58.480 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.475+0000 7f99c2b9c700 1 --2- 172.21.15.69:0/352860666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc8390 0x55aefccd7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.476+0000 7f99c239b700 1 -- 172.21.15.69:0/352860666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc8390 msgr2=0x55aefccd7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.476+0000 7f99c239b700 1 --2- 172.21.15.69:0/352860666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc8390 0x55aefccd7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.476+0000 7f99c239b700 1 -- 172.21.15.69:0/352860666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 msgr2=0x55aefccd0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.476+0000 7f99c239b700 1 --2- 172.21.15.69:0/352860666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.476+0000 7f99c239b700 1 -- 172.21.15.69:0/352860666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99bc1b5660 con 0x55aefccc6f20 2023-07-06T20:28:58.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.480+0000 7f99c239b700 1 --2- 172.21.15.69:0/352860666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc6f20 0x55aefccd0950 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b680 tx=0x7f99b8005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc043362ff865842 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:28:58.485 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.480+0000 7f99a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:28:58.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.484+0000 7f99a5ffb700 1 -- 172.21.15.69:0/352860666 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b801f040 con 0x55aefccc6f20 2023-07-06T20:28:58.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.484+0000 7f99a5ffb700 1 -- 172.21.15.69:0/352860666 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f99b8004500 con 0x55aefccc6f20 2023-07-06T20:28:58.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.484+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/352860666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aefcb714f0 con 0x55aefccc6f20 2023-07-06T20:28:58.486 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.484+0000 7f99a5ffb700 1 -- 172.21.15.69:0/352860666 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99b80046c0 con 0x55aefccc6f20 2023-07-06T20:28:58.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.484+0000 7f99d7fb0b80 10 client.15573 fetch_fsmap learned FSMap version 30 2023-07-06T20:28:58.487 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.484+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/352860666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55aefca2ec60 con 0x55aefccc6f20 2023-07-06T20:28:58.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.487+0000 7f99a5ffb700 1 -- 172.21.15.69:0/352860666 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aefcb714f0 con 0x55aefccc6f20 2023-07-06T20:28:58.490 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.487+0000 7f99a5ffb700 1 -- 172.21.15.69:0/352860666 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f99b800f8a0 con 0x55aefccc6f20 2023-07-06T20:28:58.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.488+0000 7f99d7fb0b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:28:58.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.488+0000 7f99d7fb0b80 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:28:58.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.488+0000 7f99d7fb0b80 20 client.15573 populate_metadata read hostname 'smithi069' 2023-07-06T20:28:58.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.488+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/352860666 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55aefccdc720 0x55aefccdeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.488+0000 7f99d7fb0b80 4 client.15573 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:28:58.491 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.488+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/352860666 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aefcc95d60 con 0x55aefccdc720 2023-07-06T20:28:58.494 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.491+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/352860666 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55aefccdc720 0x55aefccdeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.495 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.493+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/352860666 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55aefccdc720 0x55aefccdeb30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd21f8878e3cf80 server_cookie=da7f1596b0cbf343 in_seq=0 out_seq=0 2023-07-06T20:28:58.495 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.493+0000 7f99a5ffb700 10 client.15573 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:28:58.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.493+0000 7f99a5ffb700 1 -- 172.21.15.69:0/352860666 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aefcc95d60 con 0x55aefccdc720 2023-07-06T20:28:58.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.493+0000 7f99a5ffb700 10 client.15573 handle_command_reply: tid=0 2023-07-06T20:28:58.496 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.494+0000 7f99d7fb0b80 1 client.15573 shutdown 2023-07-06T20:28:58.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.494+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/352860666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc6f20 msgr2=0x55aefccd0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:58.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.494+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/352860666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc6f20 0x55aefccd0950 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b680 tx=0x7f99b8005a90 comp rx=0 tx=0).stop 2023-07-06T20:28:58.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.494+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/352860666 shutdown_connections 2023-07-06T20:28:58.497 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.494+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/352860666 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55aefccdc720 0x55aefccdeb30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.494+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/352860666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc8390 0x55aefccd7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.494+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/352860666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc6f20 0x55aefccd0950 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.494+0000 7f99a5ffb700 0 client.15573 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:28:58.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.494+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/352860666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.494+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/352860666 >> 172.21.15.69:0/352860666 conn(0x55aefccbc640 msgr2=0x55aefccc10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:28:58.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.494+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/352860666 shutdown_connections 2023-07-06T20:28:58.498 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.494+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/352860666 wait complete. 2023-07-06T20:28:58.499 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.495+0000 7f99d7fb0b80 20 client.15573 trim_cache size 0 max 16384 2023-07-06T20:28:58.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.525+0000 7f99d7fb0b80 1 Processor -- start 2023-07-06T20:28:58.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.525+0000 7f99d7fb0b80 1 -- start start 2023-07-06T20:28:58.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.526+0000 7f99d7fb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 0x55aefccce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.526+0000 7f99d7fb0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.526+0000 7f99d7fb0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc8390 0x55aefcccb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.526+0000 7f99d7fb0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefccd16b0 con 0x55aefccc8390 2023-07-06T20:28:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.526+0000 7f99d7fb0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefccc24e0 con 0x55aefccc7870 2023-07-06T20:28:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.526+0000 7f99d7fb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefccbc210 con 0x55aefccc6f20 2023-07-06T20:28:58.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.526+0000 7f99c1b9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.526+0000 7f99c1b9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:60834/0 (socket says 172.21.15.69:60834) 2023-07-06T20:28:58.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.526+0000 7f99c1b9a700 1 -- 172.21.15.69:0/955323995 learned_addr learned my addr 172.21.15.69:0/955323995 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:28:58.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.527+0000 7f99c1b9a700 1 -- 172.21.15.69:0/955323995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 msgr2=0x55aefccce7d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:28:58.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.527+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/955323995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 0x55aefccce7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.527+0000 7f99c1b9a700 1 -- 172.21.15.69:0/955323995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc8390 msgr2=0x55aefcccb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:58.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.527+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/955323995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc8390 0x55aefcccb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.527+0000 7f99c1b9a700 1 -- 172.21.15.69:0/955323995 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aefccc8c80 con 0x55aefccc7870 2023-07-06T20:28:58.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.528+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/955323995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd7e90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0962c0 tx=0x7f99ac005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c85f3cec27b7be3d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:28:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.528+0000 7f99b13ff700 1 -- 172.21.15.69:0/955323995 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac010e10 con 0x55aefccc7870 2023-07-06T20:28:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.528+0000 7f99b13ff700 1 -- 172.21.15.69:0/955323995 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ac0b1070 con 0x55aefccc7870 2023-07-06T20:28:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.528+0000 7f99b13ff700 1 -- 172.21.15.69:0/955323995 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac0b9320 con 0x55aefccc7870 2023-07-06T20:28:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.530+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/955323995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 msgr2=0x55aefccd7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.530+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/955323995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd7e90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0962c0 tx=0x7f99ac005c80 comp rx=0 tx=0).stop 2023-07-06T20:28:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.531+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/955323995 shutdown_connections 2023-07-06T20:28:58.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.531+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/955323995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc8390 0x55aefcccb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.531+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/955323995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 0x55aefccce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.531+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/955323995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd7e90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.531+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/955323995 >> 172.21.15.69:0/955323995 conn(0x55aefccbc640 msgr2=0x55aefcccf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:28:58.535 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.531+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/955323995 shutdown_connections 2023-07-06T20:28:58.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.533+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/955323995 wait complete. 2023-07-06T20:28:58.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.534+0000 7f99d7fb0b80 1 -- start start 2023-07-06T20:28:58.537 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.534+0000 7f99d7fb0b80 1 Processor -- start 2023-07-06T20:28:58.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.536+0000 7f99d7fb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 0x55aefccd7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.536+0000 7f99d7fb0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.536+0000 7f99d7fb0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc8390 0x55aefccd5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.536+0000 7f99d7fb0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefccd6430 con 0x55aefccc8390 2023-07-06T20:28:58.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.536+0000 7f99d7fb0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefccd65a0 con 0x55aefccc7870 2023-07-06T20:28:58.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.536+0000 7f99d7fb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefccd6760 con 0x55aefccc6f20 2023-07-06T20:28:58.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.536+0000 7f99d7fb0b80 10 client.0 _ll_register_callbacks cb 0x7f99b800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:28:58.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.537+0000 7f99c1b9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.537+0000 7f99c239b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 0x55aefccd7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.537+0000 7f99c1b9a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:60848/0 (socket says 172.21.15.69:60848) 2023-07-06T20:28:58.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.537+0000 7f99c1b9a700 1 -- 172.21.15.69:0/2619397741 learned_addr learned my addr 172.21.15.69:0/2619397741 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:28:58.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.537+0000 7f99c1b9a700 1 -- 172.21.15.69:0/2619397741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 msgr2=0x55aefccd7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:58.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.537+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/2619397741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 0x55aefccd7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.537+0000 7f99c1b9a700 1 -- 172.21.15.69:0/2619397741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc8390 msgr2=0x55aefccd5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:28:58.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.537+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/2619397741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc8390 0x55aefccd5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.537+0000 7f99c1b9a700 1 -- 172.21.15.69:0/2619397741 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aefca45de0 con 0x55aefccc7870 2023-07-06T20:28:58.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.538+0000 7f99c1b9a700 1 --2- 172.21.15.69:0/2619397741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd7d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f99ac097ba0 tx=0x7f99ac0ca270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d26688214cece1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:28:58.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.538+0000 7f99a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:28:58.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.538+0000 7f99a5ffb700 1 -- 172.21.15.69:0/2619397741 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac010e10 con 0x55aefccc7870 2023-07-06T20:28:58.546 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.538+0000 7f99a5ffb700 1 -- 172.21.15.69:0/2619397741 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f99ac097bf0 con 0x55aefccc7870 2023-07-06T20:28:58.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.539+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/2619397741 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99ac0a1ca0 con 0x55aefccc7870 2023-07-06T20:28:58.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.539+0000 7f99a5ffb700 1 -- 172.21.15.69:0/2619397741 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99ac0b90d0 con 0x55aefccc7870 2023-07-06T20:28:58.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.539+0000 7f99d7fb0b80 10 client.25724 fetch_fsmap learned FSMap version 30 2023-07-06T20:28:58.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.539+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/2619397741 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55aefca6c250 con 0x55aefccc7870 2023-07-06T20:28:58.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.539+0000 7f99a5ffb700 1 -- 172.21.15.69:0/2619397741 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ac0a1ca0 con 0x55aefccc7870 2023-07-06T20:28:58.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.539+0000 7f99a5ffb700 1 -- 172.21.15.69:0/2619397741 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f99ac099bb0 con 0x55aefccc7870 2023-07-06T20:28:58.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.540+0000 7f99d7fb0b80 10 client.25724 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:28:58.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.540+0000 7f99d7fb0b80 10 client.25724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:28:58.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.540+0000 7f99d7fb0b80 20 client.25724 populate_metadata read hostname 'smithi069' 2023-07-06T20:28:58.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.540+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/2619397741 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55aefcd31340 0x55aefcd33720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:28:58.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.540+0000 7f99d7fb0b80 4 client.25724 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:28:58.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.540+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/2619397741 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aefcc10eb0 con 0x55aefcd31340 2023-07-06T20:28:58.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.540+0000 7f99c239b700 1 --2- 172.21.15.69:0/2619397741 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55aefcd31340 0x55aefcd33720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:28:58.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99c239b700 1 --2- 172.21.15.69:0/2619397741 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55aefcd31340 0x55aefcd33720 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b090d22d27b2a9 server_cookie=efdafa28e3531305 in_seq=0 out_seq=0 2023-07-06T20:28:58.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99a5ffb700 10 client.25724 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:28:58.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99a5ffb700 1 -- 172.21.15.69:0/2619397741 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55aefcc10eb0 con 0x55aefcd31340 2023-07-06T20:28:58.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99a5ffb700 10 client.25724 handle_command_reply: tid=0 2023-07-06T20:28:58.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99d7fb0b80 1 client.25724 shutdown 2023-07-06T20:28:58.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/2619397741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 msgr2=0x55aefccd7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:58.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/2619397741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd7d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f99ac097ba0 tx=0x7f99ac0ca270 comp rx=0 tx=0).stop 2023-07-06T20:28:58.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/2619397741 shutdown_connections 2023-07-06T20:28:58.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/2619397741 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55aefcd31340 0x55aefcd33720 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/2619397741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aefccc8390 0x55aefccd5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/2619397741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aefccc6f20 0x55aefccd7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99d7fb0b80 1 --2- 172.21.15.69:0/2619397741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aefccc7870 0x55aefccd7d40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99a5ffb700 0 client.25724 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:28:58.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.541+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/2619397741 >> 172.21.15.69:0/2619397741 conn(0x55aefccbc640 msgr2=0x55aefccce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:28:58.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.542+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/2619397741 shutdown_connections 2023-07-06T20:28:58.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.542+0000 7f99d7fb0b80 1 -- 172.21.15.69:0/2619397741 wait complete. 2023-07-06T20:28:58.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.542+0000 7f99d7fb0b80 20 client.25724 trim_cache size 0 max 16384 2023-07-06T20:28:58.562 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:28:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:28:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:28:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:28:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:28:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:28:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:28:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:28:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:28:58.565 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:28:58.590 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.587+0000 7f99c45ff700 1 -- 172.21.15.69:0/250085033 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f99a80a12a0 msgr2=0x7f99a80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:58.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.587+0000 7f99c45ff700 1 --2- 172.21.15.69:0/250085033 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f99a80a12a0 0x7f99a80a3750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f99b400e350 tx=0x7f99b400b040 comp rx=0 tx=0).stop 2023-07-06T20:28:58.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.587+0000 7f99c45ff700 1 -- 172.21.15.69:0/250085033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99bc1078b0 msgr2=0x7f99bc112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:28:58.591 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.587+0000 7f99c45ff700 1 --2- 172.21.15.69:0/250085033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99bc1078b0 0x7f99bc112ff0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00d980 tx=0x7f99ac00dd40 comp rx=0 tx=0).stop 2023-07-06T20:28:58.598 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.595+0000 7f99c45ff700 1 -- 172.21.15.69:0/250085033 shutdown_connections 2023-07-06T20:28:58.598 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.595+0000 7f99c45ff700 1 --2- 172.21.15.69:0/250085033 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f99a80a12a0 0x7f99a80a3750 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.595+0000 7f99c45ff700 1 --2- 172.21.15.69:0/250085033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99bc108200 0x7f99bc1b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.595+0000 7f99c45ff700 1 --2- 172.21.15.69:0/250085033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99bc071d50 0x7f99bc112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.599 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.595+0000 7f99c45ff700 1 --2- 172.21.15.69:0/250085033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99bc1078b0 0x7f99bc112ff0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:28:58.600 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.595+0000 7f99c45ff700 1 -- 172.21.15.69:0/250085033 >> 172.21.15.69:0/250085033 conn(0x7f99bc06d4d0 msgr2=0x7f99bc06e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:28:58.600 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.595+0000 7f99c45ff700 1 -- 172.21.15.69:0/250085033 shutdown_connections 2023-07-06T20:28:58.600 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:28:58.596+0000 7f99c45ff700 1 -- 172.21.15.69:0/250085033 wait complete. 2023-07-06T20:28:58.638 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:28:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 430.0 s, 2450.4 tps, lat 41.019 ms stddev 26.028 2023-07-06T20:28:59.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:28:59 smithi069 ceph-mon[126399]: pgmap v617: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 MiB/s wr, 1.38k op/s 2023-07-06T20:28:59.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:28:59 smithi156 ceph-mon[129280]: pgmap v617: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 MiB/s wr, 1.38k op/s 2023-07-06T20:28:59.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:28:59 smithi098 ceph-mon[128968]: pgmap v617: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 MiB/s wr, 1.38k op/s 2023-07-06T20:29:01.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:01 smithi156 ceph-mon[129280]: pgmap v618: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 702 op/s 2023-07-06T20:29:01.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:01 smithi098 ceph-mon[128968]: pgmap v618: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 702 op/s 2023-07-06T20:29:01.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:01 smithi069 ceph-mon[126399]: pgmap v618: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 MiB/s wr, 702 op/s 2023-07-06T20:29:03.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:03 smithi069 ceph-mon[126399]: pgmap v619: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 MiB/s wr, 1.30k op/s 2023-07-06T20:29:03.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:03 smithi156 ceph-mon[129280]: pgmap v619: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 MiB/s wr, 1.30k op/s 2023-07-06T20:29:03.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:03 smithi098 ceph-mon[128968]: pgmap v619: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 MiB/s wr, 1.30k op/s 2023-07-06T20:29:04.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 435.0 s, 743.2 tps, lat 134.564 ms stddev 165.673 2023-07-06T20:29:05.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:05 smithi069 ceph-mon[126399]: pgmap v620: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 41 MiB/s wr, 2.63k op/s 2023-07-06T20:29:05.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:05 smithi156 ceph-mon[129280]: pgmap v620: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 41 MiB/s wr, 2.63k op/s 2023-07-06T20:29:05.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:05 smithi098 ceph-mon[128968]: pgmap v620: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 41 MiB/s wr, 2.63k op/s 2023-07-06T20:29:07.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:07 smithi098 ceph-mon[128968]: pgmap v621: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 3.19k op/s 2023-07-06T20:29:07.774 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:07 smithi069 ceph-mon[126399]: pgmap v621: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 3.19k op/s 2023-07-06T20:29:07.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:07 smithi156 ceph-mon[129280]: pgmap v621: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 3.19k op/s 2023-07-06T20:29:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 440.0 s, 2567.0 tps, lat 38.760 ms stddev 26.553 2023-07-06T20:29:09.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:09 smithi069 ceph-mon[126399]: pgmap v622: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 47 MiB/s wr, 3.79k op/s 2023-07-06T20:29:09.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:09 smithi156 ceph-mon[129280]: pgmap v622: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 47 MiB/s wr, 3.79k op/s 2023-07-06T20:29:09.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:09 smithi098 ceph-mon[128968]: pgmap v622: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 47 MiB/s wr, 3.79k op/s 2023-07-06T20:29:11.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:11 smithi069 ceph-mon[126399]: pgmap v623: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 47 MiB/s wr, 3.74k op/s 2023-07-06T20:29:11.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:11 smithi098 ceph-mon[128968]: pgmap v623: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 47 MiB/s wr, 3.74k op/s 2023-07-06T20:29:11.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:11 smithi156 ceph-mon[129280]: pgmap v623: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 47 MiB/s wr, 3.74k op/s 2023-07-06T20:29:13.766 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:13 smithi069 ceph-mon[126399]: pgmap v624: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 48 MiB/s wr, 3.74k op/s 2023-07-06T20:29:13.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:13 smithi156 ceph-mon[129280]: pgmap v624: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 48 MiB/s wr, 3.74k op/s 2023-07-06T20:29:13.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:13 smithi098 ceph-mon[128968]: pgmap v624: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 48 MiB/s wr, 3.74k op/s 2023-07-06T20:29:14.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 445.0 s, 1590.4 tps, lat 59.791 ms stddev 158.556 2023-07-06T20:29:15.555 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:15 smithi098 ceph-mon[128968]: pgmap v625: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 47 MiB/s wr, 3.60k op/s 2023-07-06T20:29:15.862 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:15 smithi156 ceph-mon[129280]: pgmap v625: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 47 MiB/s wr, 3.60k op/s 2023-07-06T20:29:16.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:15 smithi069 ceph-mon[126399]: pgmap v625: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 47 MiB/s wr, 3.60k op/s 2023-07-06T20:29:17.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:17 smithi156 ceph-mon[129280]: pgmap v626: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 MiB/s wr, 2.93k op/s 2023-07-06T20:29:17.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:17 smithi098 ceph-mon[128968]: pgmap v626: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 MiB/s wr, 2.93k op/s 2023-07-06T20:29:18.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:17 smithi069 ceph-mon[126399]: pgmap v626: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 MiB/s wr, 2.93k op/s 2023-07-06T20:29:19.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 450.0 s, 2204.1 tps, lat 47.734 ms stddev 48.446 2023-07-06T20:29:19.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:19 smithi098 ceph-mon[128968]: pgmap v627: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 39 MiB/s wr, 2.52k op/s 2023-07-06T20:29:20.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:19 smithi069 ceph-mon[126399]: pgmap v627: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 39 MiB/s wr, 2.52k op/s 2023-07-06T20:29:20.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:19 smithi156 ceph-mon[129280]: pgmap v627: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 39 MiB/s wr, 2.52k op/s 2023-07-06T20:29:22.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:21 smithi069 ceph-mon[126399]: pgmap v628: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 39 MiB/s wr, 2.13k op/s 2023-07-06T20:29:22.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:21 smithi156 ceph-mon[129280]: pgmap v628: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 39 MiB/s wr, 2.13k op/s 2023-07-06T20:29:22.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:21 smithi098 ceph-mon[128968]: pgmap v628: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 39 MiB/s wr, 2.13k op/s 2023-07-06T20:29:24.017 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:23 smithi069 ceph-mon[126399]: pgmap v629: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 MiB/s wr, 2.17k op/s 2023-07-06T20:29:24.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:23 smithi156 ceph-mon[129280]: pgmap v629: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 MiB/s wr, 2.17k op/s 2023-07-06T20:29:24.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:23 smithi098 ceph-mon[128968]: pgmap v629: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 MiB/s wr, 2.17k op/s 2023-07-06T20:29:24.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 455.0 s, 1445.0 tps, lat 68.753 ms stddev 88.488 2023-07-06T20:29:26.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:25 smithi156 ceph-mon[129280]: pgmap v630: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 41 MiB/s wr, 2.63k op/s 2023-07-06T20:29:26.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:25 smithi098 ceph-mon[128968]: pgmap v630: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 41 MiB/s wr, 2.63k op/s 2023-07-06T20:29:26.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:25 smithi069 ceph-mon[126399]: pgmap v630: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 41 MiB/s wr, 2.63k op/s 2023-07-06T20:29:28.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:27 smithi156 ceph-mon[129280]: pgmap v631: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 38 MiB/s wr, 2.58k op/s 2023-07-06T20:29:28.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:27 smithi098 ceph-mon[128968]: pgmap v631: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 38 MiB/s wr, 2.58k op/s 2023-07-06T20:29:28.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:27 smithi069 ceph-mon[126399]: pgmap v631: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 38 MiB/s wr, 2.58k op/s 2023-07-06T20:29:28.639 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:29:28.951 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.949+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/287814089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1e41037a0 msgr2=0x7fa1e4103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:28.952 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.949+0000 7fa1ea1ff700 1 --2- 172.21.15.69:0/287814089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1e41037a0 0x7fa1e4103c10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009f90 tx=0x7fa1dc00b330 comp rx=0 tx=0).stop 2023-07-06T20:29:28.952 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.949+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/287814089 shutdown_connections 2023-07-06T20:29:28.952 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.949+0000 7fa1ea1ff700 1 --2- 172.21.15.69:0/287814089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1e41037a0 0x7fa1e4103c10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:28.952 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.949+0000 7fa1ea1ff700 1 --2- 172.21.15.69:0/287814089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1e40fe480 0x7fa1e41008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:28.952 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.949+0000 7fa1ea1ff700 1 --2- 172.21.15.69:0/287814089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1e4100de0 0x7fa1e4103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:28.952 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.949+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/287814089 >> 172.21.15.69:0/287814089 conn(0x7fa1e40fa130 msgr2=0x7fa1e40fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:29:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.949+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/287814089 shutdown_connections 2023-07-06T20:29:28.953 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.949+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/287814089 wait complete. 2023-07-06T20:29:28.954 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.950+0000 7fa1ea1ff700 1 Processor -- start 2023-07-06T20:29:28.954 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.950+0000 7fa1ea1ff700 1 -- start start 2023-07-06T20:29:28.954 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.950+0000 7fa1ea1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1e40fe480 0x7fa1e419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.950+0000 7fa1ea1ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1e4100de0 0x7fa1e419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.950+0000 7fa1ea1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1e41037a0 0x7fa1e41a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1ea1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e41a3b00 con 0x7fa1e40fe480 2023-07-06T20:29:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1ea1ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e41a3c40 con 0x7fa1e4100de0 2023-07-06T20:29:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1ea1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e41a3dd0 con 0x7fa1e41037a0 2023-07-06T20:29:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1e2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1e4100de0 0x7fa1e419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1e37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1e40fe480 0x7fa1e419bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:28.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1e2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1e4100de0 0x7fa1e419c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:59752/0 (socket says 172.21.15.69:59752) 2023-07-06T20:29:28.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1e2ffd700 1 -- 172.21.15.69:0/4069614122 learned_addr learned my addr 172.21.15.69:0/4069614122 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:29:28.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1e3fff700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1e41037a0 0x7fa1e41a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:28.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1e2ffd700 1 -- 172.21.15.69:0/4069614122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1e41037a0 msgr2=0x7fa1e41a3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:28.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1e2ffd700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1e41037a0 0x7fa1e41a3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:28.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1e2ffd700 1 -- 172.21.15.69:0/4069614122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1e40fe480 msgr2=0x7fa1e419bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:28.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1e2ffd700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1e40fe480 0x7fa1e419bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:28.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.951+0000 7fa1e2ffd700 1 -- 172.21.15.69:0/4069614122 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc009c40 con 0x7fa1e4100de0 2023-07-06T20:29:28.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.952+0000 7fa1e2ffd700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1e4100de0 0x7fa1e419c4c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800b530 tx=0x7fa1d800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:29:28.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.952+0000 7fa1e0ff9700 1 -- 172.21.15.69:0/4069614122 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8011830 con 0x7fa1e4100de0 2023-07-06T20:29:28.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.952+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/4069614122 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e41a40b0 con 0x7fa1e4100de0 2023-07-06T20:29:28.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.952+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/4069614122 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e41a4690 con 0x7fa1e4100de0 2023-07-06T20:29:28.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.952+0000 7fa1e0ff9700 1 -- 172.21.15.69:0/4069614122 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1d8011e50 con 0x7fa1e4100de0 2023-07-06T20:29:28.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.952+0000 7fa1e0ff9700 1 -- 172.21.15.69:0/4069614122 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8010e50 con 0x7fa1e4100de0 2023-07-06T20:29:28.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.958+0000 7fa1fdb65b80 1 Processor -- start 2023-07-06T20:29:28.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.958+0000 7fa1e0ff9700 1 -- 172.21.15.69:0/4069614122 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa1d8011990 con 0x7fa1e4100de0 2023-07-06T20:29:28.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.964+0000 7fa1e0ff9700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa1cc0a1250 0x7fa1cc0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:28.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.964+0000 7fa1e0ff9700 1 -- 172.21.15.69:0/4069614122 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fa1d80cc220 con 0x7fa1e4100de0 2023-07-06T20:29:28.968 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.965+0000 7fa1e3fff700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa1cc0a1250 0x7fa1cc0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:28.968 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.965+0000 7fa1fdb65b80 1 -- start start 2023-07-06T20:29:28.968 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.965+0000 7fa1fdb65b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28007f20 0x555d28008330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:28.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.966+0000 7fa1fdb65b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28008870 0x555d28008e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:28.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.966+0000 7fa1fdb65b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28009390 0x555d28009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:28.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.966+0000 7fa1fdb65b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d27f0ccb0 con 0x555d28009390 2023-07-06T20:29:28.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.966+0000 7fa1fdb65b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d27f5fa40 con 0x555d28008870 2023-07-06T20:29:28.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.966+0000 7fa1fdb65b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d27f58ed0 con 0x555d28007f20 2023-07-06T20:29:28.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.966+0000 7fa1e37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28007f20 0x555d28008330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:28.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.966+0000 7fa1e37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28007f20 0x555d28008330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:58564/0 (socket says 172.21.15.69:58564) 2023-07-06T20:29:28.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.966+0000 7fa1e37fe700 1 -- 172.21.15.69:0/862390825 learned_addr learned my addr 172.21.15.69:0/862390825 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:29:28.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.967+0000 7fa1e3fff700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa1cc0a1250 0x7fa1cc0a3700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009550 tx=0x7fa1dc007720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:29:28.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.967+0000 7fa1e37fe700 1 -- 172.21.15.69:0/862390825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28008870 msgr2=0x555d28008e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:28.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.967+0000 7fa1e37fe700 1 --2- 172.21.15.69:0/862390825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28008870 0x555d28008e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:28.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.967+0000 7fa1e37fe700 1 -- 172.21.15.69:0/862390825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28009390 msgr2=0x555d28009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:28.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.967+0000 7fa1e37fe700 1 --2- 172.21.15.69:0/862390825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28009390 0x555d28009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:28.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.967+0000 7fa1e37fe700 1 -- 172.21.15.69:0/862390825 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d27d86de0 con 0x555d28007f20 2023-07-06T20:29:28.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.967+0000 7fa1e37fe700 1 --2- 172.21.15.69:0/862390825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28007f20 0x555d28008330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4009d60 tx=0x7fa1d4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a070e4db25c0ea11 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:29:28.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.968+0000 7fa1c9ffb700 1 -- 172.21.15.69:0/862390825 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4004a30 con 0x555d28007f20 2023-07-06T20:29:28.971 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.968+0000 7fa1c9ffb700 1 -- 172.21.15.69:0/862390825 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1d40054e0 con 0x555d28007f20 2023-07-06T20:29:28.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.968+0000 7fa1c9ffb700 1 -- 172.21.15.69:0/862390825 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40198d0 con 0x555d28007f20 2023-07-06T20:29:28.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.968+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/862390825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28007f20 msgr2=0x555d28008330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:28.972 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.968+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/862390825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28007f20 0x555d28008330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4009d60 tx=0x7fa1d4011720 comp rx=0 tx=0).stop 2023-07-06T20:29:28.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.969+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/862390825 shutdown_connections 2023-07-06T20:29:28.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.969+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/862390825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28009390 0x555d28009830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:28.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.969+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/862390825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28007f20 0x555d28008330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:28.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.969+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/862390825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28008870 0x555d28008e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:28.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.969+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/862390825 >> 172.21.15.69:0/862390825 conn(0x555d27ffd640 msgr2=0x555d280029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:29:28.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.969+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/862390825 shutdown_connections 2023-07-06T20:29:28.973 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.969+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/862390825 wait complete. 2023-07-06T20:29:28.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.970+0000 7fa1fdb65b80 1 -- start start 2023-07-06T20:29:28.974 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.970+0000 7fa1fdb65b80 1 Processor -- start 2023-07-06T20:29:28.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.971+0000 7fa1fdb65b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28007f20 0x555d280118f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:28.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.971+0000 7fa1fdb65b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28008870 0x555d28011e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:28.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.971+0000 7fa1fdb65b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28009390 0x555d28018e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:28.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.971+0000 7fa1fdb65b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d27ef8ac0 con 0x555d28008870 2023-07-06T20:29:28.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.971+0000 7fa1fdb65b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d27ef87c0 con 0x555d28007f20 2023-07-06T20:29:28.975 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.971+0000 7fa1fdb65b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d27ddc920 con 0x555d28009390 2023-07-06T20:29:28.976 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.971+0000 7fa1fdb65b80 10 client.0 _ll_register_callbacks cb 0x555d27ef6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:29:28.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.978+0000 7fa1e37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28007f20 0x555d280118f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:28.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.978+0000 7fa1e37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28007f20 0x555d280118f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:59762/0 (socket says 172.21.15.69:59762) 2023-07-06T20:29:28.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.978+0000 7fa1e37fe700 1 -- 172.21.15.69:0/4065032495 learned_addr learned my addr 172.21.15.69:0/4065032495 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:29:28.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.980+0000 7fa1e3fff700 1 --2- 172.21.15.69:0/4065032495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28009390 0x555d28018e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:28.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.987+0000 7fa1e37fe700 1 -- 172.21.15.69:0/4065032495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28009390 msgr2=0x555d28018e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:28.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.987+0000 7fa1e37fe700 1 --2- 172.21.15.69:0/4065032495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28009390 0x555d28018e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:28.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.987+0000 7fa1e37fe700 1 -- 172.21.15.69:0/4065032495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28008870 msgr2=0x555d28011e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:28.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.987+0000 7fa1e37fe700 1 --2- 172.21.15.69:0/4065032495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28008870 0x555d28011e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:28.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.987+0000 7fa1e37fe700 1 -- 172.21.15.69:0/4065032495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d40099c0 con 0x555d28007f20 2023-07-06T20:29:28.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.987+0000 7fa1e37fe700 1 --2- 172.21.15.69:0/4065032495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28007f20 0x555d280118f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa1d401b040 tx=0x7fa1d4004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=635adde1b7d035d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:29:28.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.989+0000 7fa1c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:29:28.994 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.992+0000 7fa1c2ffd700 1 -- 172.21.15.69:0/4065032495 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4005650 con 0x555d28007f20 2023-07-06T20:29:28.995 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.992+0000 7fa1c2ffd700 1 -- 172.21.15.69:0/4065032495 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fa1d40195d0 con 0x555d28007f20 2023-07-06T20:29:28.995 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.992+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/4065032495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d27eb24f0 con 0x555d28007f20 2023-07-06T20:29:28.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.996+0000 7fa1c2ffd700 1 -- 172.21.15.69:0/4065032495 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1d4019790 con 0x555d28007f20 2023-07-06T20:29:28.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.996+0000 7fa1fdb65b80 10 client.25733 fetch_fsmap learned FSMap version 30 2023-07-06T20:29:28.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:28.996+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/4065032495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555d27d6fc60 con 0x555d28007f20 2023-07-06T20:29:29.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.000+0000 7fa1c2ffd700 1 -- 172.21.15.69:0/4065032495 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d27eb24f0 con 0x555d28007f20 2023-07-06T20:29:29.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.000+0000 7fa1c2ffd700 1 -- 172.21.15.69:0/4065032495 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa1d402d9f0 con 0x555d28007f20 2023-07-06T20:29:29.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.001+0000 7fa1fdb65b80 10 client.25733 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:29:29.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.001+0000 7fa1fdb65b80 10 client.25733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:29:29.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.001+0000 7fa1fdb65b80 20 client.25733 populate_metadata read hostname 'smithi069' 2023-07-06T20:29:29.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.001+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/4065032495 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x555d2801d670 0x555d2801fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:29.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.001+0000 7fa1fdb65b80 4 client.25733 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:29:29.005 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.001+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/4065032495 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d27fd6d60 con 0x555d2801d670 2023-07-06T20:29:29.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.004+0000 7fa1e2ffd700 1 --2- 172.21.15.69:0/4065032495 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x555d2801d670 0x555d2801fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:29.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.006+0000 7fa1e2ffd700 1 --2- 172.21.15.69:0/4065032495 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x555d2801d670 0x555d2801fa80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=618267c56acd7a35 server_cookie=6d2f3a023a5782d1 in_seq=0 out_seq=0 2023-07-06T20:29:29.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.006+0000 7fa1c2ffd700 10 client.25733 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:29:29.010 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.008+0000 7fa1c2ffd700 1 -- 172.21.15.69:0/4065032495 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555d27fd6d60 con 0x555d2801d670 2023-07-06T20:29:29.010 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.008+0000 7fa1c2ffd700 10 client.25733 handle_command_reply: tid=0 2023-07-06T20:29:29.010 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.008+0000 7fa1fdb65b80 1 client.25733 shutdown 2023-07-06T20:29:29.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.013+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/4065032495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28007f20 msgr2=0x555d280118f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:29.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.013+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/4065032495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28007f20 0x555d280118f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa1d401b040 tx=0x7fa1d4004a30 comp rx=0 tx=0).stop 2023-07-06T20:29:29.015 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.013+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/4065032495 shutdown_connections 2023-07-06T20:29:29.016 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.013+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/4065032495 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x555d2801d670 0x555d2801fa80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.016 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.013+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/4065032495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28009390 0x555d28018e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.016 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.013+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/4065032495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28007f20 0x555d280118f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.016 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.013+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/4065032495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28008870 0x555d28011e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.017 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.013+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/4065032495 >> 172.21.15.69:0/4065032495 conn(0x555d27ffd640 msgr2=0x555d280020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:29:29.017 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.013+0000 7fa1c2ffd700 0 client.25733 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:29:29.018 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.014+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/4065032495 shutdown_connections 2023-07-06T20:29:29.018 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.014+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/4065032495 wait complete. 2023-07-06T20:29:29.018 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.014+0000 7fa1fdb65b80 20 client.25733 trim_cache size 0 max 16384 2023-07-06T20:29:29.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.047+0000 7fa1fdb65b80 1 Processor -- start 2023-07-06T20:29:29.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.048+0000 7fa1fdb65b80 1 -- start start 2023-07-06T20:29:29.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.048+0000 7fa1fdb65b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28007f20 0x555d2801f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:29.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.048+0000 7fa1fdb65b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28008870 0x555d28018d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:29.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.048+0000 7fa1fdb65b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28009390 0x555d2800c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:29.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.048+0000 7fa1fdb65b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d27fd6d60 con 0x555d28007f20 2023-07-06T20:29:29.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.048+0000 7fa1fdb65b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d27eb24f0 con 0x555d28008870 2023-07-06T20:29:29.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.048+0000 7fa1fdb65b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d28012ae0 con 0x555d28009390 2023-07-06T20:29:29.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.048+0000 7fa1e2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28008870 0x555d28018d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:29.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.048+0000 7fa1e2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28008870 0x555d28018d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:59772/0 (socket says 172.21.15.69:59772) 2023-07-06T20:29:29.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.048+0000 7fa1e2ffd700 1 -- 172.21.15.69:0/1731078525 learned_addr learned my addr 172.21.15.69:0/1731078525 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:29:29.053 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.049+0000 7fa1e3fff700 1 --2- 172.21.15.69:0/1731078525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28009390 0x555d2800c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.049+0000 7fa1e37fe700 1 --2- 172.21.15.69:0/1731078525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28007f20 0x555d2801f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.049+0000 7fa1e2ffd700 1 -- 172.21.15.69:0/1731078525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28009390 msgr2=0x555d2800c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.049+0000 7fa1e2ffd700 1 --2- 172.21.15.69:0/1731078525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28009390 0x555d2800c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.049+0000 7fa1e2ffd700 1 -- 172.21.15.69:0/1731078525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28007f20 msgr2=0x555d2801f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.049+0000 7fa1e2ffd700 1 --2- 172.21.15.69:0/1731078525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28007f20 0x555d2801f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.049+0000 7fa1e2ffd700 1 -- 172.21.15.69:0/1731078525 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d27dad250 con 0x555d28008870 2023-07-06T20:29:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.049+0000 7fa1e3fff700 1 --2- 172.21.15.69:0/1731078525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28009390 0x555d2800c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:29:29.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.049+0000 7fa1e2ffd700 1 --2- 172.21.15.69:0/1731078525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28008870 0x555d28018d90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8099530 tx=0x7fa1d8009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1047b65667094adb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:29:29.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.050+0000 7fa1ca7fc700 1 -- 172.21.15.69:0/1731078525 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d809d5d0 con 0x555d28008870 2023-07-06T20:29:29.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.050+0000 7fa1ca7fc700 1 -- 172.21.15.69:0/1731078525 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1d80a19f0 con 0x555d28008870 2023-07-06T20:29:29.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.050+0000 7fa1ca7fc700 1 -- 172.21.15.69:0/1731078525 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80842d0 con 0x555d28008870 2023-07-06T20:29:29.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.050+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/1731078525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28008870 msgr2=0x555d28018d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:29.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.050+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/1731078525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28008870 0x555d28018d90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8099530 tx=0x7fa1d8009380 comp rx=0 tx=0).stop 2023-07-06T20:29:29.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.050+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/1731078525 shutdown_connections 2023-07-06T20:29:29.055 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.051+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/1731078525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28009390 0x555d2800c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.051+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/1731078525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28007f20 0x555d2801f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.051+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/1731078525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28008870 0x555d28018d90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.051+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/1731078525 >> 172.21.15.69:0/1731078525 conn(0x555d27ffd640 msgr2=0x555d280196a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:29:29.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.051+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/1731078525 shutdown_connections 2023-07-06T20:29:29.056 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.051+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/1731078525 wait complete. 2023-07-06T20:29:29.057 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.051+0000 7fa1fdb65b80 1 -- start start 2023-07-06T20:29:29.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.051+0000 7fa1fdb65b80 1 Processor -- start 2023-07-06T20:29:29.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1fdb65b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28007f20 0x555d28014ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:29.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1fdb65b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28008870 0x555d28015430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:29.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1fdb65b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28009390 0x555d28018450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:29.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1fdb65b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d28018990 con 0x555d28007f20 2023-07-06T20:29:29.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1fdb65b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d28018b00 con 0x555d28009390 2023-07-06T20:29:29.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1fdb65b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d28018cc0 con 0x555d28008870 2023-07-06T20:29:29.060 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1fdb65b80 10 client.0 _ll_register_callbacks cb 0x7fa1d4009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:29:29.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1e2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28008870 0x555d28015430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:29.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1e2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28008870 0x555d28015430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:58598/0 (socket says 172.21.15.69:58598) 2023-07-06T20:29:29.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1e2ffd700 1 -- 172.21.15.69:0/2349123674 learned_addr learned my addr 172.21.15.69:0/2349123674 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:29:29.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1e3fff700 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28009390 0x555d28018450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:29.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1e37fe700 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28007f20 0x555d28014ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:29.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1e2ffd700 1 -- 172.21.15.69:0/2349123674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28009390 msgr2=0x555d28018450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:29.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1e2ffd700 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28009390 0x555d28018450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1e2ffd700 1 -- 172.21.15.69:0/2349123674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28007f20 msgr2=0x555d28014ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:29.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1e2ffd700 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28007f20 0x555d28014ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.052+0000 7fa1e2ffd700 1 -- 172.21.15.69:0/2349123674 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e41a4690 con 0x555d28008870 2023-07-06T20:29:29.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.053+0000 7fa1e2ffd700 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28008870 0x555d28015430 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80048a0 tx=0x7fa1d8097790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56e71908609df5b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:29:29.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.053+0000 7fa1c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:29:29.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.053+0000 7fa1c2ffd700 1 -- 172.21.15.69:0/2349123674 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80842c0 con 0x555d28008870 2023-07-06T20:29:29.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.053+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/2349123674 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1d40054e0 con 0x555d28008870 2023-07-06T20:29:29.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.053+0000 7fa1c2ffd700 1 -- 172.21.15.69:0/2349123674 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fa1d809d170 con 0x555d28008870 2023-07-06T20:29:29.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.053+0000 7fa1c2ffd700 1 -- 172.21.15.69:0/2349123674 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1d809d330 con 0x555d28008870 2023-07-06T20:29:29.063 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.053+0000 7fa1fdb65b80 10 client.25684 fetch_fsmap learned FSMap version 30 2023-07-06T20:29:29.064 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.053+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/2349123674 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555d27d852b0 con 0x555d28008870 2023-07-06T20:29:29.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.053+0000 7fa1c2ffd700 1 -- 172.21.15.69:0/2349123674 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1d40054e0 con 0x555d28008870 2023-07-06T20:29:29.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.053+0000 7fa1c2ffd700 1 -- 172.21.15.69:0/2349123674 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa1d80b7cb0 con 0x555d28008870 2023-07-06T20:29:29.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.054+0000 7fa1fdb65b80 10 client.25684 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:29:29.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.054+0000 7fa1fdb65b80 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:29:29.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.054+0000 7fa1fdb65b80 20 client.25684 populate_metadata read hostname 'smithi069' 2023-07-06T20:29:29.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.054+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x555d280723c0 0x555d280747a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:29.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.054+0000 7fa1fdb65b80 4 client.25684 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:29:29.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.054+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/2349123674 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d27f51eb0 con 0x555d280723c0 2023-07-06T20:29:29.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.054+0000 7fa1e37fe700 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x555d280723c0 0x555d280747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:29.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.054+0000 7fa1e37fe700 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x555d280723c0 0x555d280747a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b90ab4b8f795b379 server_cookie=43bacc47aa88ef3d in_seq=0 out_seq=0 2023-07-06T20:29:29.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.054+0000 7fa1c2ffd700 10 client.25684 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:29:29.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1c2ffd700 1 -- 172.21.15.69:0/2349123674 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x555d27f51eb0 con 0x555d280723c0 2023-07-06T20:29:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1c2ffd700 10 client.25684 handle_command_reply: tid=0 2023-07-06T20:29:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1fdb65b80 1 client.25684 shutdown 2023-07-06T20:29:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/2349123674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28008870 msgr2=0x555d28015430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:29.070 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28008870 0x555d28015430 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80048a0 tx=0x7fa1d8097790 comp rx=0 tx=0).stop 2023-07-06T20:29:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/2349123674 shutdown_connections 2023-07-06T20:29:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x555d280723c0 0x555d280747a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d28009390 0x555d28018450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555d28007f20 0x555d28014ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1c2ffd700 0 client.25684 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:29:29.071 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1fdb65b80 1 --2- 172.21.15.69:0/2349123674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d28008870 0x555d28015430 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/2349123674 >> 172.21.15.69:0/2349123674 conn(0x555d27ffd640 msgr2=0x555d280196a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:29:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/2349123674 shutdown_connections 2023-07-06T20:29:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1fdb65b80 1 -- 172.21.15.69:0/2349123674 wait complete. 2023-07-06T20:29:29.072 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.055+0000 7fa1fdb65b80 20 client.25684 trim_cache size 0 max 16384 2023-07-06T20:29:29.072 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:29:29.073 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:29:29.073 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:29:29.073 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:29:29.073 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:29:29.073 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:29:29.073 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:29:29.074 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:29:29.074 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:29:29.074 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:29:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.056+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/4069614122 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa1cc0a1250 msgr2=0x7fa1cc0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.056+0000 7fa1ea1ff700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa1cc0a1250 0x7fa1cc0a3700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009550 tx=0x7fa1dc007720 comp rx=0 tx=0).stop 2023-07-06T20:29:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.056+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/4069614122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1e4100de0 msgr2=0x7fa1e419c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.056+0000 7fa1ea1ff700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1e4100de0 0x7fa1e419c4c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800b530 tx=0x7fa1d800b840 comp rx=0 tx=0).stop 2023-07-06T20:29:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.056+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/4069614122 shutdown_connections 2023-07-06T20:29:29.075 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.056+0000 7fa1ea1ff700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fa1cc0a1250 0x7fa1cc0a3700 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.056+0000 7fa1ea1ff700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1e41037a0 0x7fa1e41a3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.056+0000 7fa1ea1ff700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1e40fe480 0x7fa1e419bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.056+0000 7fa1ea1ff700 1 --2- 172.21.15.69:0/4069614122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1e4100de0 0x7fa1e419c4c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.056+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/4069614122 >> 172.21.15.69:0/4069614122 conn(0x7fa1e40fa130 msgr2=0x7fa1e40ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:29:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.056+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/4069614122 shutdown_connections 2023-07-06T20:29:29.076 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:29.056+0000 7fa1ea1ff700 1 -- 172.21.15.69:0/4069614122 wait complete. 2023-07-06T20:29:29.085 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:29:29.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 460.0 s, 2466.4 tps, lat 40.635 ms stddev 28.775 2023-07-06T20:29:30.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:29 smithi156 ceph-mon[129280]: pgmap v632: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 2.43k op/s 2023-07-06T20:29:30.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:29 smithi098 ceph-mon[128968]: pgmap v632: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 2.43k op/s 2023-07-06T20:29:30.264 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:29 smithi069 ceph-mon[126399]: pgmap v632: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 2.43k op/s 2023-07-06T20:29:31.873 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:31 smithi098 ceph-mon[128968]: pgmap v633: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 MiB/s wr, 2.52k op/s 2023-07-06T20:29:32.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:31 smithi069 ceph-mon[126399]: pgmap v633: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 MiB/s wr, 2.52k op/s 2023-07-06T20:29:32.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:31 smithi156 ceph-mon[129280]: pgmap v633: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 MiB/s wr, 2.52k op/s 2023-07-06T20:29:34.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 465.0 s, 2191.4 tps, lat 45.932 ms stddev 64.191 2023-07-06T20:29:34.266 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:33 smithi069 ceph-mon[126399]: pgmap v634: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 MiB/s wr, 2.31k op/s 2023-07-06T20:29:34.368 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:33 smithi156 ceph-mon[129280]: pgmap v634: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 MiB/s wr, 2.31k op/s 2023-07-06T20:29:34.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:33 smithi098 ceph-mon[128968]: pgmap v634: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 MiB/s wr, 2.31k op/s 2023-07-06T20:29:36.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:35 smithi069 ceph-mon[126399]: pgmap v635: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 MiB/s wr, 2.28k op/s 2023-07-06T20:29:36.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:35 smithi156 ceph-mon[129280]: pgmap v635: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 MiB/s wr, 2.28k op/s 2023-07-06T20:29:36.380 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:35 smithi098 ceph-mon[128968]: pgmap v635: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 MiB/s wr, 2.28k op/s 2023-07-06T20:29:37.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:36 smithi069 ceph-mon[126399]: pgmap v636: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2023-07-06T20:29:37.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:36 smithi156 ceph-mon[129280]: pgmap v636: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2023-07-06T20:29:37.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:36 smithi098 ceph-mon[128968]: pgmap v636: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2023-07-06T20:29:39.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 470.0 s, 2374.0 tps, lat 42.109 ms stddev 28.649 2023-07-06T20:29:39.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:39 smithi069 ceph-mon[126399]: pgmap v637: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 1.43k op/s 2023-07-06T20:29:39.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:39 smithi156 ceph-mon[129280]: pgmap v637: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 1.43k op/s 2023-07-06T20:29:39.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:39 smithi098 ceph-mon[128968]: pgmap v637: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 1.43k op/s 2023-07-06T20:29:41.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:41 smithi156 ceph-mon[129280]: pgmap v638: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 1.07k op/s 2023-07-06T20:29:41.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:41 smithi098 ceph-mon[128968]: pgmap v638: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 1.07k op/s 2023-07-06T20:29:41.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:41 smithi069 ceph-mon[126399]: pgmap v638: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 1.07k op/s 2023-07-06T20:29:43.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:43 smithi156 ceph-mon[129280]: pgmap v639: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 1.01k op/s 2023-07-06T20:29:43.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:43 smithi098 ceph-mon[128968]: pgmap v639: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 1.01k op/s 2023-07-06T20:29:43.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:43 smithi069 ceph-mon[126399]: pgmap v639: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 1.01k op/s 2023-07-06T20:29:44.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 475.0 s, 1003.2 tps, lat 97.142 ms stddev 171.287 2023-07-06T20:29:45.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:45 smithi098 ceph-mon[128968]: pgmap v640: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 39 MiB/s wr, 2.04k op/s 2023-07-06T20:29:45.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:45 smithi069 ceph-mon[126399]: pgmap v640: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 39 MiB/s wr, 2.04k op/s 2023-07-06T20:29:45.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:45 smithi156 ceph-mon[129280]: pgmap v640: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 39 MiB/s wr, 2.04k op/s 2023-07-06T20:29:47.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:47 smithi069 ceph-mon[126399]: pgmap v641: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 2.71k op/s 2023-07-06T20:29:47.867 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:47 smithi156 ceph-mon[129280]: pgmap v641: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 2.71k op/s 2023-07-06T20:29:47.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:47 smithi098 ceph-mon[128968]: pgmap v641: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 2.71k op/s 2023-07-06T20:29:49.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 480.0 s, 2089.0 tps, lat 49.110 ms stddev 62.532 2023-07-06T20:29:49.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:49 smithi069 ceph-mon[126399]: pgmap v642: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 46 MiB/s wr, 3.44k op/s 2023-07-06T20:29:49.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:49 smithi156 ceph-mon[129280]: pgmap v642: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 46 MiB/s wr, 3.44k op/s 2023-07-06T20:29:49.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:49 smithi098 ceph-mon[128968]: pgmap v642: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 46 MiB/s wr, 3.44k op/s 2023-07-06T20:29:51.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:51 smithi069 ceph-mon[126399]: pgmap v643: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 52 MiB/s wr, 3.82k op/s 2023-07-06T20:29:51.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:51 smithi156 ceph-mon[129280]: pgmap v643: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 52 MiB/s wr, 3.82k op/s 2023-07-06T20:29:51.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:51 smithi098 ceph-mon[128968]: pgmap v643: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 52 MiB/s wr, 3.82k op/s 2023-07-06T20:29:53.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:53 smithi069 ceph-mon[126399]: pgmap v644: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 MiB/s wr, 3.61k op/s 2023-07-06T20:29:53.861 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:53 smithi156 ceph-mon[129280]: pgmap v644: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 MiB/s wr, 3.61k op/s 2023-07-06T20:29:53.879 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:53 smithi098 ceph-mon[128968]: pgmap v644: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 MiB/s wr, 3.61k op/s 2023-07-06T20:29:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 485.0 s, 2351.3 tps, lat 39.800 ms stddev 27.942 2023-07-06T20:29:55.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:55 smithi069 ceph-mon[126399]: pgmap v645: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 47 MiB/s wr, 3.39k op/s 2023-07-06T20:29:55.862 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:55 smithi156 ceph-mon[129280]: pgmap v645: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 47 MiB/s wr, 3.39k op/s 2023-07-06T20:29:55.882 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:55 smithi098 ceph-mon[128968]: pgmap v645: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 47 MiB/s wr, 3.39k op/s 2023-07-06T20:29:57.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:57 smithi156 ceph-mon[129280]: pgmap v646: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 2.41k op/s 2023-07-06T20:29:57.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:29:57.879 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:57 smithi098 ceph-mon[128968]: pgmap v646: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 2.41k op/s 2023-07-06T20:29:57.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:29:58.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:57 smithi069 ceph-mon[126399]: pgmap v646: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 2.41k op/s 2023-07-06T20:29:58.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:29:59.087 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:29:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 490.0 s, 2039.9 tps, lat 52.012 ms stddev 63.542 2023-07-06T20:29:59.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.537+0000 7fb8a29ff700 1 -- 172.21.15.69:0/325209583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb89c10be90 msgr2=0x7fb89c10e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:59.540 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.537+0000 7fb8a29ff700 1 --2- 172.21.15.69:0/325209583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb89c10be90 0x7fb89c10e320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb894009990 tx=0x7fb894009ca0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.538+0000 7fb8a29ff700 1 -- 172.21.15.69:0/325209583 shutdown_connections 2023-07-06T20:29:59.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.538+0000 7fb8a29ff700 1 --2- 172.21.15.69:0/325209583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb89c10be90 0x7fb89c10e320 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.538+0000 7fb8a29ff700 1 --2- 172.21.15.69:0/325209583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c071010 0x7fb89c071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.538+0000 7fb8a29ff700 1 --2- 172.21.15.69:0/325209583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c071960 0x7fb89c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.538+0000 7fb8a29ff700 1 -- 172.21.15.69:0/325209583 >> 172.21.15.69:0/325209583 conn(0x7fb89c06cbc0 msgr2=0x7fb89c06eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:29:59.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.538+0000 7fb8a29ff700 1 -- 172.21.15.69:0/325209583 shutdown_connections 2023-07-06T20:29:59.541 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.538+0000 7fb8a29ff700 1 -- 172.21.15.69:0/325209583 wait complete. 2023-07-06T20:29:59.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.540+0000 7fb8a29ff700 1 Processor -- start 2023-07-06T20:29:59.542 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.540+0000 7fb8a29ff700 1 -- start start 2023-07-06T20:29:59.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.540+0000 7fb8a29ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c071010 0x7fb89c113bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.540+0000 7fb8a29ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb89c071960 0x7fb89c114110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.541+0000 7fb8a29ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c10be90 0x7fb89c1b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.541+0000 7fb8a29ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c1171f0 con 0x7fb89c10be90 2023-07-06T20:29:59.543 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.541+0000 7fb8a29ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c117360 con 0x7fb89c071960 2023-07-06T20:29:59.544 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.541+0000 7fb8a29ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c1b4fd0 con 0x7fb89c071010 2023-07-06T20:29:59.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.543+0000 7fb89bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c071010 0x7fb89c113bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.543+0000 7fb89bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c071010 0x7fb89c113bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:48216/0 (socket says 172.21.15.69:48216) 2023-07-06T20:29:59.545 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.543+0000 7fb89bfff700 1 -- 172.21.15.69:0/450961842 learned_addr learned my addr 172.21.15.69:0/450961842 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:29:59.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.544+0000 7fb89bfff700 1 -- 172.21.15.69:0/450961842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb89c071960 msgr2=0x7fb89c114110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:59.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.544+0000 7fb89bfff700 1 --2- 172.21.15.69:0/450961842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb89c071960 0x7fb89c114110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.544+0000 7fb89bfff700 1 -- 172.21.15.69:0/450961842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c10be90 msgr2=0x7fb89c1b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:59.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.544+0000 7fb89bfff700 1 --2- 172.21.15.69:0/450961842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c10be90 0x7fb89c1b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.544+0000 7fb89bfff700 1 -- 172.21.15.69:0/450961842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940095f0 con 0x7fb89c071010 2023-07-06T20:29:59.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.545+0000 7fb89bfff700 1 --2- 172.21.15.69:0/450961842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c071010 0x7fb89c113bd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00ba50 tx=0x7fb88c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:29:59.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.556+0000 7fb8997fa700 1 -- 172.21.15.69:0/450961842 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00cab0 con 0x7fb89c071010 2023-07-06T20:29:59.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.556+0000 7fb8997fa700 1 -- 172.21.15.69:0/450961842 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb88c00cc10 con 0x7fb89c071010 2023-07-06T20:29:59.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.556+0000 7fb8997fa700 1 -- 172.21.15.69:0/450961842 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c0075d0 con 0x7fb89c071010 2023-07-06T20:29:59.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.556+0000 7fb8a29ff700 1 -- 172.21.15.69:0/450961842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89c1b51d0 con 0x7fb89c071010 2023-07-06T20:29:59.560 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.556+0000 7fb8a29ff700 1 -- 172.21.15.69:0/450961842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c1b5730 con 0x7fb89c071010 2023-07-06T20:29:59.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.562+0000 7fb8b63ebb80 1 Processor -- start 2023-07-06T20:29:59.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.563+0000 7fb8b63ebb80 1 -- start start 2023-07-06T20:29:59.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.563+0000 7fb8b63ebb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b6d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.563+0000 7fb8b63ebb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6d870 0x55d4e3b6de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.563+0000 7fb8b63ebb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6e390 0x55d4e3b6e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.563+0000 7fb8b63ebb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3a71cb0 con 0x55d4e3b6d870 2023-07-06T20:29:59.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.563+0000 7fb8b63ebb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3ac4a40 con 0x55d4e3b6cf20 2023-07-06T20:29:59.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.563+0000 7fb8b63ebb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3abded0 con 0x55d4e3b6e390 2023-07-06T20:29:59.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.564+0000 7fb8a0f9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b6d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.570+0000 7fb89b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6e390 0x55d4e3b6e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.579 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.575+0000 7fb8a0f9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b6d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:58056/0 (socket says 172.21.15.69:58056) 2023-07-06T20:29:59.579 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.575+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/1661398171 learned_addr learned my addr 172.21.15.69:0/1661398171 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:29:59.579 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.575+0000 7fb89bfff700 1 --2- 172.21.15.69:0/1661398171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6d870 0x55d4e3b6de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.588+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/1661398171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6e390 msgr2=0x55d4e3b6e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.588+0000 7fb8a0f9c700 1 --2- 172.21.15.69:0/1661398171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6e390 0x55d4e3b6e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.593 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.588+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/1661398171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6d870 msgr2=0x55d4e3b6de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.588+0000 7fb8a0f9c700 1 --2- 172.21.15.69:0/1661398171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6d870 0x55d4e3b6de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.588+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/1661398171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4e38ebde0 con 0x55d4e3b6cf20 2023-07-06T20:29:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.588+0000 7fb8997fa700 1 -- 172.21.15.69:0/450961842 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb88c014440 con 0x7fb89c071010 2023-07-06T20:29:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.590+0000 7fb8997fa700 1 --2- 172.21.15.69:0/450961842 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb8840a1350 0x7fb8840a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.594 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.590+0000 7fb8997fa700 1 -- 172.21.15.69:0/450961842 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fb88c006e40 con 0x7fb89c071010 2023-07-06T20:29:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.593+0000 7fb8a0f9c700 1 --2- 172.21.15.69:0/1661398171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b6d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb89400be40 tx=0x7fb89400be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=990466797bb9bf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:29:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.597+0000 7fb8827fc700 1 -- 172.21.15.69:0/1661398171 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89401d070 con 0x55d4e3b6cf20 2023-07-06T20:29:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.597+0000 7fb8827fc700 1 -- 172.21.15.69:0/1661398171 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8940038a0 con 0x55d4e3b6cf20 2023-07-06T20:29:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.597+0000 7fb8827fc700 1 -- 172.21.15.69:0/1661398171 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894005590 con 0x55d4e3b6cf20 2023-07-06T20:29:59.602 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.598+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/1661398171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 msgr2=0x55d4e3b6d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:59.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.598+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/1661398171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b6d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb89400be40 tx=0x7fb89400be70 comp rx=0 tx=0).stop 2023-07-06T20:29:59.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.598+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/1661398171 shutdown_connections 2023-07-06T20:29:59.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.598+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/1661398171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6e390 0x55d4e3b6e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.598+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/1661398171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b6d330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.598+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/1661398171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6d870 0x55d4e3b6de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.598+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/1661398171 >> 172.21.15.69:0/1661398171 conn(0x55d4e3b62640 msgr2=0x55d4e3b679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:29:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.600+0000 7fb89bfff700 1 --2- 172.21.15.69:0/450961842 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb8840a1350 0x7fb8840a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.600+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/1661398171 shutdown_connections 2023-07-06T20:29:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.600+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/1661398171 wait complete. 2023-07-06T20:29:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.601+0000 7fb8b63ebb80 1 -- start start 2023-07-06T20:29:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.601+0000 7fb8b63ebb80 1 Processor -- start 2023-07-06T20:29:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.602+0000 7fb8b63ebb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b768b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.602+0000 7fb8b63ebb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6d870 0x55d4e3b76df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.602+0000 7fb8b63ebb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6e390 0x55d4e3b7de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.602+0000 7fb8b63ebb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3a5dac0 con 0x55d4e3b6e390 2023-07-06T20:29:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.602+0000 7fb8b63ebb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3a5d7c0 con 0x55d4e3b6cf20 2023-07-06T20:29:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.602+0000 7fb8b63ebb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3941920 con 0x55d4e3b6d870 2023-07-06T20:29:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.602+0000 7fb8b63ebb80 10 client.0 _ll_register_callbacks cb 0x55d4e3a5b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:29:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.603+0000 7fb8a0f9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.603+0000 7fb8a0f9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b768b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:58070/0 (socket says 172.21.15.69:58070) 2023-07-06T20:29:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.603+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/2203056107 learned_addr learned my addr 172.21.15.69:0/2203056107 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:29:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.604+0000 7fb89b7fe700 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6d870 0x55d4e3b76df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.604+0000 7fb8a0f9c700 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6e390 0x55d4e3b7de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.604+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/2203056107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6d870 msgr2=0x55d4e3b76df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:59.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.604+0000 7fb8a0f9c700 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6d870 0x55d4e3b76df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.604+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/2203056107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6e390 msgr2=0x55d4e3b7de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.604+0000 7fb8a0f9c700 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6e390 0x55d4e3b7de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.604+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/2203056107 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c117770 con 0x55d4e3b6cf20 2023-07-06T20:29:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.605+0000 7fb89bfff700 1 --2- 172.21.15.69:0/450961842 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb8840a1350 0x7fb8840a3800 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb88c09efe0 tx=0x7fb88c0cf610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:29:59.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.605+0000 7fb89b7fe700 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6d870 0x55d4e3b76df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:29:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.605+0000 7fb8a0f9c700 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b768b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb894009990 tx=0x7fb89400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15d33a3b65db9fbd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:29:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.606+0000 7fb87b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:29:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.606+0000 7fb87b7fe700 1 -- 172.21.15.69:0/2203056107 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89401d030 con 0x55d4e3b6cf20 2023-07-06T20:29:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.606+0000 7fb87b7fe700 1 -- 172.21.15.69:0/2203056107 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fb894022460 con 0x55d4e3b6cf20 2023-07-06T20:29:59.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.606+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2203056107 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4e3a174f0 con 0x55d4e3b6cf20 2023-07-06T20:29:59.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.607+0000 7fb87b7fe700 1 -- 172.21.15.69:0/2203056107 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb894003e20 con 0x55d4e3b6cf20 2023-07-06T20:29:59.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.607+0000 7fb8b63ebb80 10 client.25748 fetch_fsmap learned FSMap version 30 2023-07-06T20:29:59.610 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.607+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2203056107 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d4e38d4c60 con 0x55d4e3b6cf20 2023-07-06T20:29:59.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.611+0000 7fb87b7fe700 1 -- 172.21.15.69:0/2203056107 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d4e3a174f0 con 0x55d4e3b6cf20 2023-07-06T20:29:59.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.611+0000 7fb87b7fe700 1 -- 172.21.15.69:0/2203056107 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb89400f6c0 con 0x55d4e3b6cf20 2023-07-06T20:29:59.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.612+0000 7fb8b63ebb80 10 client.25748 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:29:59.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.612+0000 7fb8b63ebb80 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:29:59.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.612+0000 7fb8b63ebb80 20 client.25748 populate_metadata read hostname 'smithi069' 2023-07-06T20:29:59.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.612+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d4e3b82030 0x55d4e3b84410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.612+0000 7fb8b63ebb80 4 client.25748 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:29:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.612+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2203056107 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4e3b3bd60 con 0x55d4e3b82030 2023-07-06T20:29:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.612+0000 7fb89b7fe700 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d4e3b82030 0x55d4e3b84410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.613+0000 7fb89b7fe700 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d4e3b82030 0x55d4e3b84410 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3153840ffee3a43 server_cookie=710b52f2f8ba0f71 in_seq=0 out_seq=0 2023-07-06T20:29:59.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.613+0000 7fb87b7fe700 10 client.25748 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:29:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.614+0000 7fb87b7fe700 1 -- 172.21.15.69:0/2203056107 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d4e3b3bd60 con 0x55d4e3b82030 2023-07-06T20:29:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.614+0000 7fb87b7fe700 10 client.25748 handle_command_reply: tid=0 2023-07-06T20:29:59.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.614+0000 7fb8b63ebb80 1 client.25748 shutdown 2023-07-06T20:29:59.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.615+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2203056107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 msgr2=0x55d4e3b768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:59.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.615+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b768b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb894009990 tx=0x7fb89400bf90 comp rx=0 tx=0).stop 2023-07-06T20:29:59.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.615+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2203056107 shutdown_connections 2023-07-06T20:29:59.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.615+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d4e3b82030 0x55d4e3b84410 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.615+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6e390 0x55d4e3b7de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.615+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b768b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.615+0000 7fb87b7fe700 0 client.25748 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:29:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.615+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2203056107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6d870 0x55d4e3b76df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.615+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2203056107 >> 172.21.15.69:0/2203056107 conn(0x55d4e3b62640 msgr2=0x55d4e3b670b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:29:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.615+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2203056107 shutdown_connections 2023-07-06T20:29:59.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.615+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2203056107 wait complete. 2023-07-06T20:29:59.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.615+0000 7fb8b63ebb80 20 client.25748 trim_cache size 0 max 16384 2023-07-06T20:29:59.650 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.647+0000 7fb8b63ebb80 1 Processor -- start 2023-07-06T20:29:59.650 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.648+0000 7fb8b63ebb80 1 -- start start 2023-07-06T20:29:59.650 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.648+0000 7fb8b63ebb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b75570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.651 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.648+0000 7fb8b63ebb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6d870 0x55d4e3b7ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.651 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.648+0000 7fb8b63ebb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6e390 0x55d4e3b74580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.651 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.648+0000 7fb8b63ebb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3b76480 con 0x55d4e3b6d870 2023-07-06T20:29:59.651 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.648+0000 7fb8b63ebb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3b62210 con 0x55d4e3b6e390 2023-07-06T20:29:59.651 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.648+0000 7fb8b63ebb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3b75d60 con 0x55d4e3b6cf20 2023-07-06T20:29:59.652 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb8a0f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b75570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.652 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb89b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6d870 0x55d4e3b7ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.652 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb8a0f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b75570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:48258/0 (socket says 172.21.15.69:48258) 2023-07-06T20:29:59.652 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/2216761368 learned_addr learned my addr 172.21.15.69:0/2216761368 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:29:59.652 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb89b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6d870 0x55d4e3b7ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45944/0 (socket says 172.21.15.69:45944) 2023-07-06T20:29:59.653 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb8a0f9c700 1 --2- 172.21.15.69:0/2216761368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6e390 0x55d4e3b74580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.653 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb89b7fe700 1 -- 172.21.15.69:0/2216761368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 msgr2=0x55d4e3b75570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:59.653 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb89b7fe700 1 --2- 172.21.15.69:0/2216761368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b75570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.653 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb89b7fe700 1 -- 172.21.15.69:0/2216761368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6e390 msgr2=0x55d4e3b74580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:59.653 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb89b7fe700 1 --2- 172.21.15.69:0/2216761368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6e390 0x55d4e3b74580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.653 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb89b7fe700 1 -- 172.21.15.69:0/2216761368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4e3912250 con 0x55d4e3b6d870 2023-07-06T20:29:59.653 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb8a0f9c700 1 --2- 172.21.15.69:0/2216761368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6e390 0x55d4e3b74580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:29:59.654 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.649+0000 7fb89b7fe700 1 --2- 172.21.15.69:0/2216761368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6d870 0x55d4e3b7ddf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55d4e3a5d8a0 tx=0x7fb890009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd217ed4cfd4798 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:29:59.654 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.650+0000 7fb882ffd700 1 -- 172.21.15.69:0/2216761368 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8900192b0 con 0x55d4e3b6d870 2023-07-06T20:29:59.654 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.650+0000 7fb882ffd700 1 -- 172.21.15.69:0/2216761368 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb890019410 con 0x55d4e3b6d870 2023-07-06T20:29:59.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.650+0000 7fb882ffd700 1 -- 172.21.15.69:0/2216761368 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890015430 con 0x55d4e3b6d870 2023-07-06T20:29:59.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.651+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2216761368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6d870 msgr2=0x55d4e3b7ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:59.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.651+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2216761368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6d870 0x55d4e3b7ddf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55d4e3a5d8a0 tx=0x7fb890009190 comp rx=0 tx=0).stop 2023-07-06T20:29:59.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.651+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2216761368 shutdown_connections 2023-07-06T20:29:59.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.651+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2216761368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6e390 0x55d4e3b74580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.651+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2216761368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b75570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.655 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.651+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2216761368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6d870 0x55d4e3b7ddf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.651+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2216761368 >> 172.21.15.69:0/2216761368 conn(0x55d4e3b62640 msgr2=0x55d4e3b7e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:29:59.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.651+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2216761368 shutdown_connections 2023-07-06T20:29:59.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.651+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2216761368 wait complete. 2023-07-06T20:29:59.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.652+0000 7fb8b63ebb80 1 -- start start 2023-07-06T20:29:59.656 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.652+0000 7fb8b63ebb80 1 Processor -- start 2023-07-06T20:29:59.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.653+0000 7fb8b63ebb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b7d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.653+0000 7fb8b63ebb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6d870 0x55d4e3b7dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.653+0000 7fb8b63ebb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6e390 0x55d4e3b7a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.653+0000 7fb8b63ebb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3b7ac60 con 0x55d4e3b6e390 2023-07-06T20:29:59.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.653+0000 7fb8b63ebb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3b7add0 con 0x55d4e3b6d870 2023-07-06T20:29:59.657 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.653+0000 7fb8b63ebb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4e3b7af90 con 0x55d4e3b6cf20 2023-07-06T20:29:59.658 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.653+0000 7fb8b63ebb80 10 client.0 _ll_register_callbacks cb 0x7fb88c0d1e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:29:59.658 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.654+0000 7fb8a0f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b7d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.658 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.654+0000 7fb8a0f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b7d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:48270/0 (socket says 172.21.15.69:48270) 2023-07-06T20:29:59.658 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.654+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/2546769530 learned_addr learned my addr 172.21.15.69:0/2546769530 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:29:59.663 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.660+0000 7fb89b7fe700 1 --2- 172.21.15.69:0/2546769530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6d870 0x55d4e3b7dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.663 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.661+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/2546769530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6d870 msgr2=0x55d4e3b7dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:59.663 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.661+0000 7fb8a0f9c700 1 --2- 172.21.15.69:0/2546769530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6d870 0x55d4e3b7dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.661+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/2546769530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6e390 msgr2=0x55d4e3b7a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:29:59.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.661+0000 7fb8a0f9c700 1 --2- 172.21.15.69:0/2546769530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6e390 0x55d4e3b7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.661+0000 7fb8a0f9c700 1 -- 172.21.15.69:0/2546769530 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89000e040 con 0x55d4e3b6cf20 2023-07-06T20:29:59.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.661+0000 7fb8a0f9c700 1 --2- 172.21.15.69:0/2546769530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b7d8a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb894009990 tx=0x7fb894005300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a584ea38dc4954cd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:29:59.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.661+0000 7fb87b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:29:59.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.663+0000 7fb87b7fe700 1 -- 172.21.15.69:0/2546769530 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89401d040 con 0x55d4e3b6cf20 2023-07-06T20:29:59.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.663+0000 7fb87b7fe700 1 -- 172.21.15.69:0/2546769530 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fb89400fcf0 con 0x55d4e3b6cf20 2023-07-06T20:29:59.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.663+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2546769530 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8940038a0 con 0x55d4e3b6cf20 2023-07-06T20:29:59.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.664+0000 7fb87b7fe700 1 -- 172.21.15.69:0/2546769530 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb89400b6f0 con 0x55d4e3b6cf20 2023-07-06T20:29:59.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.664+0000 7fb8b63ebb80 10 client.25696 fetch_fsmap learned FSMap version 30 2023-07-06T20:29:59.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.664+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2546769530 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d4e38ea2b0 con 0x55d4e3b6cf20 2023-07-06T20:29:59.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.664+0000 7fb87b7fe700 1 -- 172.21.15.69:0/2546769530 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8940038a0 con 0x55d4e3b6cf20 2023-07-06T20:29:59.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.664+0000 7fb87b7fe700 1 -- 172.21.15.69:0/2546769530 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb894040070 con 0x55d4e3b6cf20 2023-07-06T20:29:59.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.665+0000 7fb8b63ebb80 10 client.25696 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:29:59.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.665+0000 7fb8b63ebb80 10 client.25696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:29:59.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.665+0000 7fb8b63ebb80 20 client.25696 populate_metadata read hostname 'smithi069' 2023-07-06T20:29:59.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.665+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2546769530 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d4e3bd7210 0x55d4e3bd9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:29:59.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.665+0000 7fb8b63ebb80 4 client.25696 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:29:59.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.665+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2546769530 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d4e3ab6eb0 con 0x55d4e3bd7210 2023-07-06T20:29:59.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.665+0000 7fb89b7fe700 1 --2- 172.21.15.69:0/2546769530 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d4e3bd7210 0x55d4e3bd9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:29:59.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.666+0000 7fb89b7fe700 1 --2- 172.21.15.69:0/2546769530 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d4e3bd7210 0x55d4e3bd9620 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e9cfffff062c386 server_cookie=53450ba1e6ea9ed7 in_seq=0 out_seq=0 2023-07-06T20:29:59.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.666+0000 7fb87b7fe700 10 client.25696 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:29:59.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.668+0000 7fb87b7fe700 1 -- 172.21.15.69:0/2546769530 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d4e3ab6eb0 con 0x55d4e3bd7210 2023-07-06T20:29:59.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.668+0000 7fb87b7fe700 10 client.25696 handle_command_reply: tid=0 2023-07-06T20:29:59.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.669+0000 7fb8b63ebb80 1 client.25696 shutdown 2023-07-06T20:29:59.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.669+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2546769530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 msgr2=0x55d4e3b7d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:59.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.669+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2546769530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b7d8a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb894009990 tx=0x7fb894005300 comp rx=0 tx=0).stop 2023-07-06T20:29:59.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.672+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2546769530 shutdown_connections 2023-07-06T20:29:59.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.672+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2546769530 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d4e3bd7210 0x55d4e3bd9620 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.674 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.672+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2546769530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4e3b6e390 0x55d4e3b7a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.672+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2546769530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4e3b6cf20 0x55d4e3b7d8a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.672+0000 7fb8b63ebb80 1 --2- 172.21.15.69:0/2546769530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d4e3b6d870 0x55d4e3b7dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.672+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2546769530 >> 172.21.15.69:0/2546769530 conn(0x55d4e3b62640 msgr2=0x55d4e3b75570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:29:59.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.672+0000 7fb87b7fe700 0 client.25696 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:29:59.677 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.675+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2546769530 shutdown_connections 2023-07-06T20:29:59.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.676+0000 7fb8b63ebb80 1 -- 172.21.15.69:0/2546769530 wait complete. 2023-07-06T20:29:59.678 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.676+0000 7fb8b63ebb80 20 client.25696 trim_cache size 0 max 16384 2023-07-06T20:29:59.679 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:29:59.679 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:29:59.679 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:29:59.679 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:29:59.679 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:29:59.679 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:29:59.679 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:29:59.680 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:29:59.680 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:29:59.680 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:29:59.680 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.677+0000 7fb8a29ff700 1 -- 172.21.15.69:0/450961842 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb8840a1350 msgr2=0x7fb8840a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:59.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.677+0000 7fb8a29ff700 1 --2- 172.21.15.69:0/450961842 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb8840a1350 0x7fb8840a3800 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb88c09efe0 tx=0x7fb88c0cf610 comp rx=0 tx=0).stop 2023-07-06T20:29:59.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.677+0000 7fb8a29ff700 1 -- 172.21.15.69:0/450961842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c071010 msgr2=0x7fb89c113bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:29:59.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.677+0000 7fb8a29ff700 1 --2- 172.21.15.69:0/450961842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c071010 0x7fb89c113bd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00ba50 tx=0x7fb88c00be10 comp rx=0 tx=0).stop 2023-07-06T20:29:59.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.677+0000 7fb8a29ff700 1 -- 172.21.15.69:0/450961842 shutdown_connections 2023-07-06T20:29:59.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.677+0000 7fb8a29ff700 1 --2- 172.21.15.69:0/450961842 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fb8840a1350 0x7fb8840a3800 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.681 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.677+0000 7fb8a29ff700 1 --2- 172.21.15.69:0/450961842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c10be90 0x7fb89c1b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.677+0000 7fb8a29ff700 1 --2- 172.21.15.69:0/450961842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c071010 0x7fb89c113bd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.677+0000 7fb8a29ff700 1 --2- 172.21.15.69:0/450961842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb89c071960 0x7fb89c114110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:29:59.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.677+0000 7fb8a29ff700 1 -- 172.21.15.69:0/450961842 >> 172.21.15.69:0/450961842 conn(0x7fb89c06cbc0 msgr2=0x7fb89c06e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:29:59.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.677+0000 7fb8a29ff700 1 -- 172.21.15.69:0/450961842 shutdown_connections 2023-07-06T20:29:59.682 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:29:59.678+0000 7fb8a29ff700 1 -- 172.21.15.69:0/450961842 wait complete. 2023-07-06T20:29:59.759 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:29:59.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:29:59 smithi156 ceph-mon[129280]: pgmap v647: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.70k op/s 2023-07-06T20:29:59.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:29:59 smithi098 ceph-mon[128968]: pgmap v647: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.70k op/s 2023-07-06T20:30:00.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:29:59 smithi069 ceph-mon[126399]: pgmap v647: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.70k op/s 2023-07-06T20:30:00.619 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:00 smithi156 ceph-mon[129280]: overall HEALTH_OK 2023-07-06T20:30:00.619 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:00.620 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:00 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:00.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:00 smithi098 ceph-mon[128968]: overall HEALTH_OK 2023-07-06T20:30:00.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:00.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:00 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:01.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:00 smithi069 ceph-mon[126399]: overall HEALTH_OK 2023-07-06T20:30:01.015 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:01.015 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:00 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:02.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:01 smithi069 ceph-mon[126399]: pgmap v648: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 34 MiB/s wr, 1.11k op/s 2023-07-06T20:30:02.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:02.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:01 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:02.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:01 smithi156 ceph-mon[129280]: pgmap v648: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 34 MiB/s wr, 1.11k op/s 2023-07-06T20:30:02.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:01 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:02.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:01 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:02.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:01 smithi098 ceph-mon[128968]: pgmap v648: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 34 MiB/s wr, 1.11k op/s 2023-07-06T20:30:02.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:01 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:02.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:01 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:04.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:03 smithi156 ceph-mon[129280]: pgmap v649: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 625 op/s 2023-07-06T20:30:04.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:03 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:04.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:03 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:04.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:03 smithi098 ceph-mon[128968]: pgmap v649: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 625 op/s 2023-07-06T20:30:04.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:03 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:04.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:03 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:04.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 495.0 s, 1413.4 tps, lat 67.330 ms stddev 98.160 2023-07-06T20:30:04.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:03 smithi069 ceph-mon[126399]: pgmap v649: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 625 op/s 2023-07-06T20:30:04.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:03 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:04.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:03 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:05.112 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:04 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:30:05.112 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:04 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:30:05.112 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:04 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:05.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:04 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:30:05.130 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:04 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:30:05.130 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:04 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:05.263 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:04 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:30:05.264 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:04 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:30:05.264 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:04 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:30:06.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:05 smithi156 ceph-mon[129280]: pgmap v650: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 990 op/s 2023-07-06T20:30:06.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:05 smithi098 ceph-mon[128968]: pgmap v650: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 990 op/s 2023-07-06T20:30:06.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:05 smithi069 ceph-mon[126399]: pgmap v650: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 990 op/s 2023-07-06T20:30:08.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:07 smithi069 ceph-mon[126399]: pgmap v651: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 1.78k op/s 2023-07-06T20:30:08.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:07 smithi156 ceph-mon[129280]: pgmap v651: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 1.78k op/s 2023-07-06T20:30:08.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:07 smithi098 ceph-mon[128968]: pgmap v651: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 1.78k op/s 2023-07-06T20:30:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 500.0 s, 1308.7 tps, lat 79.406 ms stddev 117.650 2023-07-06T20:30:09.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:09 smithi156 ceph-mon[129280]: pgmap v652: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 MiB/s wr, 2.50k op/s 2023-07-06T20:30:09.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:09 smithi098 ceph-mon[128968]: pgmap v652: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 MiB/s wr, 2.50k op/s 2023-07-06T20:30:09.764 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:09 smithi069 ceph-mon[126399]: pgmap v652: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 MiB/s wr, 2.50k op/s 2023-07-06T20:30:11.615 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:11 smithi156 ceph-mon[129280]: pgmap v653: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 48 MiB/s wr, 3.78k op/s 2023-07-06T20:30:11.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:11 smithi098 ceph-mon[128968]: pgmap v653: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 48 MiB/s wr, 3.78k op/s 2023-07-06T20:30:11.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:11 smithi069 ceph-mon[126399]: pgmap v653: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 48 MiB/s wr, 3.78k op/s 2023-07-06T20:30:13.612 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:13 smithi156 ceph-mon[129280]: pgmap v654: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 44 MiB/s wr, 3.61k op/s 2023-07-06T20:30:13.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:13 smithi098 ceph-mon[128968]: pgmap v654: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 44 MiB/s wr, 3.61k op/s 2023-07-06T20:30:13.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:13 smithi069 ceph-mon[126399]: pgmap v654: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 44 MiB/s wr, 3.61k op/s 2023-07-06T20:30:14.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 505.0 s, 2599.6 tps, lat 38.838 ms stddev 27.130 2023-07-06T20:30:15.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:15 smithi156 ceph-mon[129280]: pgmap v655: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 45 MiB/s wr, 3.63k op/s 2023-07-06T20:30:15.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:15 smithi069 ceph-mon[126399]: pgmap v655: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 45 MiB/s wr, 3.63k op/s 2023-07-06T20:30:15.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:15 smithi098 ceph-mon[128968]: pgmap v655: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 45 MiB/s wr, 3.63k op/s 2023-07-06T20:30:17.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:17 smithi156 ceph-mon[129280]: pgmap v656: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 MiB/s wr, 3.56k op/s 2023-07-06T20:30:17.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:17 smithi098 ceph-mon[128968]: pgmap v656: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 MiB/s wr, 3.56k op/s 2023-07-06T20:30:17.764 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:17 smithi069 ceph-mon[126399]: pgmap v656: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 MiB/s wr, 3.56k op/s 2023-07-06T20:30:19.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 510.0 s, 1103.9 tps, lat 90.930 ms stddev 201.918 2023-07-06T20:30:19.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:19 smithi069 ceph-mon[126399]: pgmap v657: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 3.29k op/s 2023-07-06T20:30:19.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:19 smithi156 ceph-mon[129280]: pgmap v657: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 3.29k op/s 2023-07-06T20:30:19.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:19 smithi098 ceph-mon[128968]: pgmap v657: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 3.29k op/s 2023-07-06T20:30:21.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:21 smithi069 ceph-mon[126399]: pgmap v658: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 46 MiB/s wr, 3.27k op/s 2023-07-06T20:30:21.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:21 smithi156 ceph-mon[129280]: pgmap v658: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 46 MiB/s wr, 3.27k op/s 2023-07-06T20:30:21.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:21 smithi098 ceph-mon[128968]: pgmap v658: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 46 MiB/s wr, 3.27k op/s 2023-07-06T20:30:23.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:23 smithi069 ceph-mon[126399]: pgmap v659: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 2.03k op/s 2023-07-06T20:30:23.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:23 smithi156 ceph-mon[129280]: pgmap v659: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 2.03k op/s 2023-07-06T20:30:23.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:23 smithi098 ceph-mon[128968]: pgmap v659: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 2.03k op/s 2023-07-06T20:30:24.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 515.0 s, 2364.4 tps, lat 42.118 ms stddev 29.589 2023-07-06T20:30:25.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:25 smithi069 ceph-mon[126399]: pgmap v660: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 2.02k op/s 2023-07-06T20:30:25.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:25 smithi156 ceph-mon[129280]: pgmap v660: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 2.02k op/s 2023-07-06T20:30:25.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:25 smithi098 ceph-mon[128968]: pgmap v660: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 2.02k op/s 2023-07-06T20:30:27.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:27 smithi156 ceph-mon[129280]: pgmap v661: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 37 MiB/s wr, 2.09k op/s 2023-07-06T20:30:27.879 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:27 smithi098 ceph-mon[128968]: pgmap v661: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 37 MiB/s wr, 2.09k op/s 2023-07-06T20:30:28.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:27 smithi069 ceph-mon[126399]: pgmap v661: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 37 MiB/s wr, 2.09k op/s 2023-07-06T20:30:29.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 520.0 s, 1259.5 tps, lat 79.493 ms stddev 94.039 2023-07-06T20:30:29.761 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:30:29.861 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:29 smithi156 ceph-mon[129280]: pgmap v662: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 2.18k op/s 2023-07-06T20:30:29.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:29 smithi098 ceph-mon[128968]: pgmap v662: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 2.18k op/s 2023-07-06T20:30:29.959 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:29 smithi069 ceph-mon[126399]: pgmap v662: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 2.18k op/s 2023-07-06T20:30:30.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.204+0000 7f1a26fff700 1 -- 172.21.15.69:0/218102613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a20071960 msgr2=0x7f1a201098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:30:30.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.204+0000 7f1a26fff700 1 --2- 172.21.15.69:0/218102613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a20071960 0x7f1a201098f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0098f0 tx=0x7f1a0c009c00 comp rx=0 tx=0).stop 2023-07-06T20:30:30.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.204+0000 7f1a26fff700 1 -- 172.21.15.69:0/218102613 shutdown_connections 2023-07-06T20:30:30.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.204+0000 7f1a26fff700 1 --2- 172.21.15.69:0/218102613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a20109e80 0x7f1a2010a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.204+0000 7f1a26fff700 1 --2- 172.21.15.69:0/218102613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a20071010 0x7f1a20071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.204+0000 7f1a26fff700 1 --2- 172.21.15.69:0/218102613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a20071960 0x7f1a201098f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.204+0000 7f1a26fff700 1 -- 172.21.15.69:0/218102613 >> 172.21.15.69:0/218102613 conn(0x7f1a2006cbc0 msgr2=0x7f1a2006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:30:30.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.204+0000 7f1a26fff700 1 -- 172.21.15.69:0/218102613 shutdown_connections 2023-07-06T20:30:30.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.205+0000 7f1a26fff700 1 -- 172.21.15.69:0/218102613 wait complete. 2023-07-06T20:30:30.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.206+0000 7f1a26fff700 1 Processor -- start 2023-07-06T20:30:30.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.206+0000 7f1a26fff700 1 -- start start 2023-07-06T20:30:30.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.206+0000 7f1a26fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a20071010 0x7f1a201a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.206+0000 7f1a26fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a20071960 0x7f1a201a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.209 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.206+0000 7f1a26fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a20109e80 0x7f1a201a79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.206+0000 7f1a26fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a201a7f90 con 0x7f1a20071960 2023-07-06T20:30:30.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.206+0000 7f1a26fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a201a80d0 con 0x7f1a20071010 2023-07-06T20:30:30.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.206+0000 7f1a26fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a201a8260 con 0x7f1a20109e80 2023-07-06T20:30:30.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.207+0000 7f1a2559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a20109e80 0x7f1a201a79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.207+0000 7f1a2559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a20109e80 0x7f1a201a79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:59690/0 (socket says 172.21.15.69:59690) 2023-07-06T20:30:30.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.207+0000 7f1a2559c700 1 -- 172.21.15.69:0/3556733291 learned_addr learned my addr 172.21.15.69:0/3556733291 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:30:30.210 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.207+0000 7f1a1ffff700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a20071960 0x7f1a201a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.207+0000 7f1a24d9b700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a20071010 0x7f1a201a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.211 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.207+0000 7f1a2559c700 1 -- 172.21.15.69:0/3556733291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a20071010 msgr2=0x7f1a201a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:30:30.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.207+0000 7f1a2559c700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a20071010 0x7f1a201a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.207+0000 7f1a2559c700 1 -- 172.21.15.69:0/3556733291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a20071960 msgr2=0x7f1a201a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:30:30.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.207+0000 7f1a2559c700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a20071960 0x7f1a201a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.207+0000 7f1a2559c700 1 -- 172.21.15.69:0/3556733291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a18009d00 con 0x7f1a20109e80 2023-07-06T20:30:30.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.207+0000 7f1a2559c700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a20109e80 0x7f1a201a79c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800e3c0 tx=0x7f1a1800e6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:30:30.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.208+0000 7f1a1dffb700 1 -- 172.21.15.69:0/3556733291 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a180090d0 con 0x7f1a20109e80 2023-07-06T20:30:30.212 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.208+0000 7f1a1dffb700 1 -- 172.21.15.69:0/3556733291 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a1800f040 con 0x7f1a20109e80 2023-07-06T20:30:30.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.208+0000 7f1a26fff700 1 -- 172.21.15.69:0/3556733291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c009520 con 0x7f1a20109e80 2023-07-06T20:30:30.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.208+0000 7f1a1dffb700 1 -- 172.21.15.69:0/3556733291 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18007c20 con 0x7f1a20109e80 2023-07-06T20:30:30.213 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.208+0000 7f1a26fff700 1 -- 172.21.15.69:0/3556733291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a201a8910 con 0x7f1a20109e80 2023-07-06T20:30:30.219 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.216+0000 7f1a3a986b80 1 Processor -- start 2023-07-06T20:30:30.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.218+0000 7f1a1dffb700 1 -- 172.21.15.69:0/3556733291 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1a18004bd0 con 0x7f1a20109e80 2023-07-06T20:30:30.220 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.218+0000 7f1a3a986b80 1 -- start start 2023-07-06T20:30:30.222 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.218+0000 7f1a3a986b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.222 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.218+0000 7f1a3a986b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 0x56103c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.222 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.218+0000 7f1a3a986b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cfba0 0x56103c0d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.218+0000 7f1a3a986b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103bfcacb0 con 0x56103c0cfba0 2023-07-06T20:30:30.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.218+0000 7f1a3a986b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103c01da40 con 0x56103c0ce730 2023-07-06T20:30:30.223 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.218+0000 7f1a3a986b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103c016ed0 con 0x56103c0cf080 2023-07-06T20:30:30.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.218+0000 7f1a24d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 0x56103c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.218+0000 7f1a24d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 0x56103c0cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:59698/0 (socket says 172.21.15.69:59698) 2023-07-06T20:30:30.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.218+0000 7f1a24d9b700 1 -- 172.21.15.69:0/3080216140 learned_addr learned my addr 172.21.15.69:0/3080216140 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:30:30.224 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.218+0000 7f1a1ffff700 1 --2- 172.21.15.69:0/3080216140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.225 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.219+0000 7f1a1ffff700 1 --2- 172.21.15.69:0/3080216140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cfba0 0x56103c0d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.225 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.219+0000 7f1a24d9b700 1 -- 172.21.15.69:0/3080216140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 msgr2=0x56103c0ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:30:30.225 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.219+0000 7f1a24d9b700 1 --2- 172.21.15.69:0/3080216140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.225 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.219+0000 7f1a24d9b700 1 -- 172.21.15.69:0/3080216140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cfba0 msgr2=0x56103c0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:30:30.226 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.219+0000 7f1a24d9b700 1 --2- 172.21.15.69:0/3080216140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cfba0 0x56103c0d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.226 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.219+0000 7f1a24d9b700 1 -- 172.21.15.69:0/3080216140 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56103be44de0 con 0x56103c0cf080 2023-07-06T20:30:30.226 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.219+0000 7f1a1ffff700 1 --2- 172.21.15.69:0/3080216140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cfba0 0x56103c0d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:30:30.226 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.219+0000 7f1a24d9b700 1 --2- 172.21.15.69:0/3080216140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 0x56103c0cf610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009d80 tx=0x7f1a14011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4851e0784d899b77 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:30:30.226 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.219+0000 7f1a0affd700 1 -- 172.21.15.69:0/3080216140 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14004230 con 0x56103c0cf080 2023-07-06T20:30:30.227 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.219+0000 7f1a0affd700 1 -- 172.21.15.69:0/3080216140 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a14004390 con 0x56103c0cf080 2023-07-06T20:30:30.227 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.219+0000 7f1a0affd700 1 -- 172.21.15.69:0/3080216140 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14019820 con 0x56103c0cf080 2023-07-06T20:30:30.227 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.220+0000 7f1a1dffb700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1a100a12a0 0x7f1a100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.220+0000 7f1a3a986b80 1 -- 172.21.15.69:0/3080216140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 msgr2=0x56103c0cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:30:30.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.220+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/3080216140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 0x56103c0cf610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009d80 tx=0x7f1a14011720 comp rx=0 tx=0).stop 2023-07-06T20:30:30.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.220+0000 7f1a1dffb700 1 -- 172.21.15.69:0/3556733291 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f1a180cc870 con 0x7f1a20109e80 2023-07-06T20:30:30.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.220+0000 7f1a3a986b80 1 -- 172.21.15.69:0/3080216140 shutdown_connections 2023-07-06T20:30:30.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.220+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/3080216140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cfba0 0x56103c0d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.220+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/3080216140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.220+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/3080216140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 0x56103c0cf610 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.229 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.220+0000 7f1a3a986b80 1 -- 172.21.15.69:0/3080216140 >> 172.21.15.69:0/3080216140 conn(0x56103c0bb640 msgr2=0x56103c0c0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:30:30.229 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.220+0000 7f1a1ffff700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1a100a12a0 0x7f1a100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.229 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.221+0000 7f1a3a986b80 1 -- 172.21.15.69:0/3080216140 shutdown_connections 2023-07-06T20:30:30.229 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.221+0000 7f1a3a986b80 1 -- 172.21.15.69:0/3080216140 wait complete. 2023-07-06T20:30:30.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.222+0000 7f1a3a986b80 1 -- start start 2023-07-06T20:30:30.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.222+0000 7f1a3a986b80 1 Processor -- start 2023-07-06T20:30:30.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.223+0000 7f1a3a986b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0d8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.223+0000 7f1a3a986b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cf080 0x56103c0d8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.223+0000 7f1a3a986b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cfba0 0x56103c0df5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.223+0000 7f1a3a986b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103bfb6ac0 con 0x56103c0cf080 2023-07-06T20:30:30.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.223+0000 7f1a3a986b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103bfb67c0 con 0x56103c0ce730 2023-07-06T20:30:30.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.223+0000 7f1a3a986b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103be9a920 con 0x56103c0cfba0 2023-07-06T20:30:30.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.223+0000 7f1a3a986b80 10 client.0 _ll_register_callbacks cb 0x56103bfb4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:30:30.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.223+0000 7f1a24d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0d8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.223+0000 7f1a24d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0d8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:59668/0 (socket says 172.21.15.69:59668) 2023-07-06T20:30:30.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.223+0000 7f1a24d9b700 1 -- 172.21.15.69:0/1873937796 learned_addr learned my addr 172.21.15.69:0/1873937796 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:30:30.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.223+0000 7f1a1ffff700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1a100a12a0 0x7f1a100a3750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0098c0 tx=0x7f1a0c009f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:30:30.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.226+0000 7f1a24d9b700 1 -- 172.21.15.69:0/1873937796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cfba0 msgr2=0x56103c0df5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:30:30.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.226+0000 7f1a24d9b700 1 --2- 172.21.15.69:0/1873937796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cfba0 0x56103c0df5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.226+0000 7f1a24d9b700 1 -- 172.21.15.69:0/1873937796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cf080 msgr2=0x56103c0d8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:30:30.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.226+0000 7f1a24d9b700 1 --2- 172.21.15.69:0/1873937796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cf080 0x56103c0d8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.226+0000 7f1a24d9b700 1 -- 172.21.15.69:0/1873937796 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a140099e0 con 0x56103c0ce730 2023-07-06T20:30:30.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.226+0000 7f1a24d9b700 1 --2- 172.21.15.69:0/1873937796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0d8020 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a1401b040 tx=0x7f1a140039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d10bdcd37feb46 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:30:30.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.230+0000 7f19fffff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:30:30.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.230+0000 7f19fffff700 1 -- 172.21.15.69:0/1873937796 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14005450 con 0x56103c0ce730 2023-07-06T20:30:30.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.231+0000 7f19fffff700 1 -- 172.21.15.69:0/1873937796 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f1a140055b0 con 0x56103c0ce730 2023-07-06T20:30:30.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.231+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1873937796 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56103bf704f0 con 0x56103c0ce730 2023-07-06T20:30:30.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.231+0000 7f19fffff700 1 -- 172.21.15.69:0/1873937796 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a14009ed0 con 0x56103c0ce730 2023-07-06T20:30:30.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.231+0000 7f1a3a986b80 10 client.25766 fetch_fsmap learned FSMap version 30 2023-07-06T20:30:30.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.231+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1873937796 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56103be2dc60 con 0x56103c0ce730 2023-07-06T20:30:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.232+0000 7f19fffff700 1 -- 172.21.15.69:0/1873937796 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56103bf704f0 con 0x56103c0ce730 2023-07-06T20:30:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.232+0000 7f19fffff700 1 -- 172.21.15.69:0/1873937796 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1a14021040 con 0x56103c0ce730 2023-07-06T20:30:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.232+0000 7f1a3a986b80 10 client.25766 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:30:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.232+0000 7f1a3a986b80 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:30:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.232+0000 7f1a3a986b80 20 client.25766 populate_metadata read hostname 'smithi069' 2023-07-06T20:30:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.232+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/1873937796 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56103c0c6ec0 0x56103c0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.232+0000 7f1a3a986b80 4 client.25766 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:30:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.232+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1873937796 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56103c094d60 con 0x56103c0c6ec0 2023-07-06T20:30:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.233+0000 7f1a1ffff700 1 --2- 172.21.15.69:0/1873937796 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56103c0c6ec0 0x56103c0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.233+0000 7f19fffff700 10 client.25766 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:30:30.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.233+0000 7f1a1ffff700 1 --2- 172.21.15.69:0/1873937796 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56103c0c6ec0 0x56103c0c92d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abbabfc0b41dac7c server_cookie=4a8053e5b40803db in_seq=0 out_seq=0 2023-07-06T20:30:30.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.237+0000 7f19fffff700 1 -- 172.21.15.69:0/1873937796 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56103c094d60 con 0x56103c0c6ec0 2023-07-06T20:30:30.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.237+0000 7f19fffff700 10 client.25766 handle_command_reply: tid=0 2023-07-06T20:30:30.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.237+0000 7f1a3a986b80 1 client.25766 shutdown 2023-07-06T20:30:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.238+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1873937796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 msgr2=0x56103c0d8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:30:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.238+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/1873937796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0d8020 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a1401b040 tx=0x7f1a140039e0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.238+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1873937796 shutdown_connections 2023-07-06T20:30:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.238+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/1873937796 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56103c0c6ec0 0x56103c0c92d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.238+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/1873937796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cfba0 0x56103c0df5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.238+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/1873937796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0d8020 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.238+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/1873937796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cf080 0x56103c0d8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.238+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1873937796 >> 172.21.15.69:0/1873937796 conn(0x56103c0bb640 msgr2=0x56103c0c00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:30:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.238+0000 7f19fffff700 0 client.25766 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:30:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.239+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1873937796 shutdown_connections 2023-07-06T20:30:30.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.239+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1873937796 wait complete. 2023-07-06T20:30:30.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.239+0000 7f1a3a986b80 20 client.25766 trim_cache size 0 max 16384 2023-07-06T20:30:30.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.270+0000 7f1a3a986b80 1 Processor -- start 2023-07-06T20:30:30.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.271+0000 7f1a3a986b80 1 -- start start 2023-07-06T20:30:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.271+0000 7f1a3a986b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0ce730 0x56103c0c0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.271+0000 7f1a3a986b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0cf080 0x56103c0f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.271+0000 7f1a3a986b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cfba0 0x56103c0c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.271+0000 7f1a3a986b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103c0c0e50 con 0x56103c0ce730 2023-07-06T20:30:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.271+0000 7f1a3a986b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103c0f5050 con 0x56103c0cf080 2023-07-06T20:30:30.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.271+0000 7f1a3a986b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103c0c6c40 con 0x56103c0cfba0 2023-07-06T20:30:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.278+0000 7f1a1ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0cf080 0x56103c0f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.278+0000 7f1a1ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0cf080 0x56103c0f4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:59674/0 (socket says 172.21.15.69:59674) 2023-07-06T20:30:30.280 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.278+0000 7f1a1ffff700 1 -- 172.21.15.69:0/1832264156 learned_addr learned my addr 172.21.15.69:0/1832264156 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:30:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.280+0000 7f1a2559c700 1 --2- 172.21.15.69:0/1832264156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cfba0 0x56103c0c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.280+0000 7f1a24d9b700 1 --2- 172.21.15.69:0/1832264156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0ce730 0x56103c0c0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.281+0000 7f1a2559c700 1 -- 172.21.15.69:0/1832264156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0cf080 msgr2=0x56103c0f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:30:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.281+0000 7f1a2559c700 1 --2- 172.21.15.69:0/1832264156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0cf080 0x56103c0f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.281+0000 7f1a2559c700 1 -- 172.21.15.69:0/1832264156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0ce730 msgr2=0x56103c0c0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:30:30.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.281+0000 7f1a2559c700 1 --2- 172.21.15.69:0/1832264156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0ce730 0x56103c0c0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.281+0000 7f1a2559c700 1 -- 172.21.15.69:0/1832264156 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56103be6b250 con 0x56103c0cfba0 2023-07-06T20:30:30.288 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.282+0000 7f1a2559c700 1 --2- 172.21.15.69:0/1832264156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cfba0 0x56103c0c00b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a180095b0 tx=0x7f1a18009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee16c9427b96fb14 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:30:30.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.288+0000 7f1a0b7fe700 1 -- 172.21.15.69:0/1832264156 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18099c80 con 0x56103c0cfba0 2023-07-06T20:30:30.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.288+0000 7f1a0b7fe700 1 -- 172.21.15.69:0/1832264156 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a1809dcb0 con 0x56103c0cfba0 2023-07-06T20:30:30.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.288+0000 7f1a0b7fe700 1 -- 172.21.15.69:0/1832264156 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a180092c0 con 0x56103c0cfba0 2023-07-06T20:30:30.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.289+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1832264156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cfba0 msgr2=0x56103c0c00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:30:30.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.289+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/1832264156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cfba0 0x56103c0c00b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a180095b0 tx=0x7f1a18009230 comp rx=0 tx=0).stop 2023-07-06T20:30:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.293+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1832264156 shutdown_connections 2023-07-06T20:30:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.293+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/1832264156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cfba0 0x56103c0c00b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.293+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/1832264156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0ce730 0x56103c0c0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.293+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/1832264156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0cf080 0x56103c0f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.293+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1832264156 >> 172.21.15.69:0/1832264156 conn(0x56103c0bb640 msgr2=0x56103c0bcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:30:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.294+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1832264156 shutdown_connections 2023-07-06T20:30:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.294+0000 7f1a3a986b80 1 -- 172.21.15.69:0/1832264156 wait complete. 2023-07-06T20:30:30.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.294+0000 7f1a3a986b80 1 -- start start 2023-07-06T20:30:30.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.295+0000 7f1a3a986b80 1 Processor -- start 2023-07-06T20:30:30.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.295+0000 7f1a3a986b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.295+0000 7f1a3a986b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 0x56103c0cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.295+0000 7f1a3a986b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cfba0 0x56103c0e5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.295+0000 7f1a3a986b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103c0e6050 con 0x56103c0cfba0 2023-07-06T20:30:30.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.295+0000 7f1a3a986b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103c0e61c0 con 0x56103c0ce730 2023-07-06T20:30:30.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.295+0000 7f1a3a986b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103c0e6380 con 0x56103c0cf080 2023-07-06T20:30:30.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.295+0000 7f1a3a986b80 10 client.0 _ll_register_callbacks cb 0x7f1a18009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:30:30.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.296+0000 7f1a1ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 0x56103c0cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.296+0000 7f1a24d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.296+0000 7f1a1ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 0x56103c0cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:59734/0 (socket says 172.21.15.69:59734) 2023-07-06T20:30:30.303 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.296+0000 7f1a1ffff700 1 -- 172.21.15.69:0/2973700935 learned_addr learned my addr 172.21.15.69:0/2973700935 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:30:30.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.307+0000 7f1a2559c700 1 --2- 172.21.15.69:0/2973700935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cfba0 0x56103c0e5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.307+0000 7f1a1ffff700 1 -- 172.21.15.69:0/2973700935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 msgr2=0x56103c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:30:30.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.307+0000 7f1a1ffff700 1 --2- 172.21.15.69:0/2973700935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.307+0000 7f1a1ffff700 1 -- 172.21.15.69:0/2973700935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cfba0 msgr2=0x56103c0e5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:30:30.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.307+0000 7f1a1ffff700 1 --2- 172.21.15.69:0/2973700935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cfba0 0x56103c0e5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.307+0000 7f1a1ffff700 1 -- 172.21.15.69:0/2973700935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a201a8910 con 0x56103c0cf080 2023-07-06T20:30:30.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.314+0000 7f1a1ffff700 1 --2- 172.21.15.69:0/2973700935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 0x56103c0cd530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c016000 tx=0x7f1a0c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6636fcd9af7422d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:30:30.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.317+0000 7f19fffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:30:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.322+0000 7f19fffff700 1 -- 172.21.15.69:0/2973700935 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c02e070 con 0x56103c0cf080 2023-07-06T20:30:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.322+0000 7f19fffff700 1 -- 172.21.15.69:0/2973700935 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f1a0c030220 con 0x56103c0cf080 2023-07-06T20:30:30.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.323+0000 7f1a3a986b80 1 -- 172.21.15.69:0/2973700935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a14004390 con 0x56103c0cf080 2023-07-06T20:30:30.327 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.325+0000 7f19fffff700 1 -- 172.21.15.69:0/2973700935 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a0c00f680 con 0x56103c0cf080 2023-07-06T20:30:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.325+0000 7f1a3a986b80 10 client.25711 fetch_fsmap learned FSMap version 30 2023-07-06T20:30:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.325+0000 7f1a3a986b80 1 -- 172.21.15.69:0/2973700935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56103be432b0 con 0x56103c0cf080 2023-07-06T20:30:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.329+0000 7f19fffff700 1 -- 172.21.15.69:0/2973700935 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a0c03f030 con 0x56103c0cf080 2023-07-06T20:30:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.329+0000 7f19fffff700 1 -- 172.21.15.69:0/2973700935 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1a0c005060 con 0x56103c0cf080 2023-07-06T20:30:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.329+0000 7f1a3a986b80 10 client.25711 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:30:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.329+0000 7f1a3a986b80 10 client.25711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:30:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.329+0000 7f1a3a986b80 20 client.25711 populate_metadata read hostname 'smithi069' 2023-07-06T20:30:30.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.329+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/2973700935 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56103c0e7d90 0x56103c0ea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:30:30.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.329+0000 7f1a3a986b80 4 client.25711 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:30:30.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.329+0000 7f1a3a986b80 1 -- 172.21.15.69:0/2973700935 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56103c00feb0 con 0x56103c0e7d90 2023-07-06T20:30:30.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.330+0000 7f1a24d9b700 1 --2- 172.21.15.69:0/2973700935 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56103c0e7d90 0x56103c0ea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:30:30.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.337+0000 7f1a24d9b700 1 --2- 172.21.15.69:0/2973700935 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56103c0e7d90 0x56103c0ea1a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578c58761fd7549 server_cookie=a1ee503741f20830 in_seq=0 out_seq=0 2023-07-06T20:30:30.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.337+0000 7f19fffff700 10 client.25711 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:30:30.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.337+0000 7f19fffff700 1 -- 172.21.15.69:0/2973700935 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56103c00feb0 con 0x56103c0e7d90 2023-07-06T20:30:30.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.337+0000 7f19fffff700 10 client.25711 handle_command_reply: tid=0 2023-07-06T20:30:30.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.341+0000 7f1a3a986b80 1 client.25711 shutdown 2023-07-06T20:30:30.345 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.342+0000 7f1a3a986b80 1 -- 172.21.15.69:0/2973700935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 msgr2=0x56103c0cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:30:30.345 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.342+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/2973700935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 0x56103c0cd530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c016000 tx=0x7f1a0c009c00 comp rx=0 tx=0).stop 2023-07-06T20:30:30.345 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.342+0000 7f1a3a986b80 1 -- 172.21.15.69:0/2973700935 shutdown_connections 2023-07-06T20:30:30.345 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.342+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/2973700935 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x56103c0e7d90 0x56103c0ea1a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.345 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.342+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/2973700935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103c0cfba0 0x56103c0e5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.345 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.343+0000 7f19fffff700 0 client.25711 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:30:30.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.343+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/2973700935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56103c0ce730 0x56103c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.343+0000 7f1a3a986b80 1 --2- 172.21.15.69:0/2973700935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56103c0cf080 0x56103c0cd530 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.343+0000 7f1a3a986b80 1 -- 172.21.15.69:0/2973700935 >> 172.21.15.69:0/2973700935 conn(0x56103c0bb640 msgr2=0x56103c0c2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:30:30.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.343+0000 7f1a3a986b80 1 -- 172.21.15.69:0/2973700935 shutdown_connections 2023-07-06T20:30:30.346 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.343+0000 7f1a3a986b80 1 -- 172.21.15.69:0/2973700935 wait complete. 2023-07-06T20:30:30.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.343+0000 7f1a3a986b80 20 client.25711 trim_cache size 0 max 16384 2023-07-06T20:30:30.347 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:30:30.347 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:30:30.347 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:30:30.347 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:30:30.348 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:30:30.348 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:30:30.348 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:30:30.349 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:30:30.349 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:30:30.349 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:30:30.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.344+0000 7f1a26fff700 1 -- 172.21.15.69:0/3556733291 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1a100a12a0 msgr2=0x7f1a100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:30:30.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.344+0000 7f1a26fff700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1a100a12a0 0x7f1a100a3750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0098c0 tx=0x7f1a0c009f50 comp rx=0 tx=0).stop 2023-07-06T20:30:30.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.344+0000 7f1a26fff700 1 -- 172.21.15.69:0/3556733291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a20109e80 msgr2=0x7f1a201a79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:30:30.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.344+0000 7f1a26fff700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a20109e80 0x7f1a201a79c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800e3c0 tx=0x7f1a1800e6d0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.344+0000 7f1a26fff700 1 -- 172.21.15.69:0/3556733291 shutdown_connections 2023-07-06T20:30:30.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.344+0000 7f1a26fff700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1a100a12a0 0x7f1a100a3750 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.344+0000 7f1a26fff700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a20109e80 0x7f1a201a79c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.344+0000 7f1a26fff700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a20071010 0x7f1a201a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.344+0000 7f1a26fff700 1 --2- 172.21.15.69:0/3556733291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a20071960 0x7f1a201a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:30:30.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.344+0000 7f1a26fff700 1 -- 172.21.15.69:0/3556733291 >> 172.21.15.69:0/3556733291 conn(0x7f1a2006cbc0 msgr2=0x7f1a2010ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:30:30.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.344+0000 7f1a26fff700 1 -- 172.21.15.69:0/3556733291 shutdown_connections 2023-07-06T20:30:30.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:30:30.344+0000 7f1a26fff700 1 -- 172.21.15.69:0/3556733291 wait complete. 2023-07-06T20:30:30.374 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:30:31.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:31 smithi156 ceph-mon[129280]: pgmap v663: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 MiB/s wr, 2.64k op/s 2023-07-06T20:30:31.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:31 smithi098 ceph-mon[128968]: pgmap v663: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 MiB/s wr, 2.64k op/s 2023-07-06T20:30:32.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:31 smithi069 ceph-mon[126399]: pgmap v663: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 MiB/s wr, 2.64k op/s 2023-07-06T20:30:33.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:33 smithi156 ceph-mon[129280]: pgmap v664: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 36 MiB/s wr, 2.33k op/s 2023-07-06T20:30:33.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:33 smithi098 ceph-mon[128968]: pgmap v664: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 36 MiB/s wr, 2.33k op/s 2023-07-06T20:30:34.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:33 smithi069 ceph-mon[126399]: pgmap v664: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 36 MiB/s wr, 2.33k op/s 2023-07-06T20:30:34.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 525.0 s, 2386.2 tps, lat 42.037 ms stddev 29.170 2023-07-06T20:30:35.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:35 smithi156 ceph-mon[129280]: pgmap v665: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 2.28k op/s 2023-07-06T20:30:35.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:35 smithi098 ceph-mon[128968]: pgmap v665: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 2.28k op/s 2023-07-06T20:30:36.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:35 smithi069 ceph-mon[126399]: pgmap v665: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 2.28k op/s 2023-07-06T20:30:38.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:37 smithi069 ceph-mon[126399]: pgmap v666: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 MiB/s wr, 2.34k op/s 2023-07-06T20:30:38.116 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:37 smithi156 ceph-mon[129280]: pgmap v666: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 MiB/s wr, 2.34k op/s 2023-07-06T20:30:38.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:37 smithi098 ceph-mon[128968]: pgmap v666: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 MiB/s wr, 2.34k op/s 2023-07-06T20:30:39.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 530.0 s, 2010.6 tps, lat 49.633 ms stddev 71.358 2023-07-06T20:30:40.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:39 smithi069 ceph-mon[126399]: pgmap v667: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 2.31k op/s 2023-07-06T20:30:40.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:39 smithi156 ceph-mon[129280]: pgmap v667: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 2.31k op/s 2023-07-06T20:30:40.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:39 smithi098 ceph-mon[128968]: pgmap v667: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 2.31k op/s 2023-07-06T20:30:42.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:41 smithi069 ceph-mon[126399]: pgmap v668: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 2.08k op/s 2023-07-06T20:30:42.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:41 smithi156 ceph-mon[129280]: pgmap v668: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 2.08k op/s 2023-07-06T20:30:42.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:41 smithi098 ceph-mon[128968]: pgmap v668: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 2.08k op/s 2023-07-06T20:30:44.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:43 smithi069 ceph-mon[126399]: pgmap v669: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 MiB/s wr, 1.14k op/s 2023-07-06T20:30:44.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:43 smithi156 ceph-mon[129280]: pgmap v669: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 MiB/s wr, 1.14k op/s 2023-07-06T20:30:44.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:43 smithi098 ceph-mon[128968]: pgmap v669: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 MiB/s wr, 1.14k op/s 2023-07-06T20:30:44.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 535.0 s, 2261.0 tps, lat 44.320 ms stddev 32.918 2023-07-06T20:30:46.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:45 smithi069 ceph-mon[126399]: pgmap v670: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 713 op/s 2023-07-06T20:30:46.112 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:45 smithi156 ceph-mon[129280]: pgmap v670: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 713 op/s 2023-07-06T20:30:46.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:45 smithi098 ceph-mon[128968]: pgmap v670: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 713 op/s 2023-07-06T20:30:48.113 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:47 smithi156 ceph-mon[129280]: pgmap v671: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 30 MiB/s wr, 797 op/s 2023-07-06T20:30:48.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:47 smithi098 ceph-mon[128968]: pgmap v671: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 30 MiB/s wr, 797 op/s 2023-07-06T20:30:48.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:47 smithi069 ceph-mon[126399]: pgmap v671: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 30 MiB/s wr, 797 op/s 2023-07-06T20:30:49.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 540.0 s, 1170.0 tps, lat 83.167 ms stddev 166.722 2023-07-06T20:30:50.266 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:49 smithi069 ceph-mon[126399]: pgmap v672: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 30 MiB/s wr, 1.16k op/s 2023-07-06T20:30:50.367 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:49 smithi156 ceph-mon[129280]: pgmap v672: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 30 MiB/s wr, 1.16k op/s 2023-07-06T20:30:50.378 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:49 smithi098 ceph-mon[128968]: pgmap v672: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 30 MiB/s wr, 1.16k op/s 2023-07-06T20:30:52.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:51 smithi069 ceph-mon[126399]: pgmap v673: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 39 MiB/s wr, 2.37k op/s 2023-07-06T20:30:52.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:51 smithi156 ceph-mon[129280]: pgmap v673: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 39 MiB/s wr, 2.37k op/s 2023-07-06T20:30:52.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:51 smithi098 ceph-mon[128968]: pgmap v673: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 39 MiB/s wr, 2.37k op/s 2023-07-06T20:30:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 545.0 s, 1486.0 tps, lat 68.535 ms stddev 86.683 2023-07-06T20:30:54.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:54 smithi156 ceph-mon[129280]: pgmap v674: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 40 MiB/s wr, 3.01k op/s 2023-07-06T20:30:54.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:54 smithi098 ceph-mon[128968]: pgmap v674: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 40 MiB/s wr, 3.01k op/s 2023-07-06T20:30:54.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:54 smithi069 ceph-mon[126399]: pgmap v674: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 40 MiB/s wr, 3.01k op/s 2023-07-06T20:30:56.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:56 smithi156 ceph-mon[129280]: pgmap v675: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 3.11k op/s 2023-07-06T20:30:56.387 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:56 smithi098 ceph-mon[128968]: pgmap v675: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 3.11k op/s 2023-07-06T20:30:56.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:56 smithi069 ceph-mon[126399]: pgmap v675: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 3.11k op/s 2023-07-06T20:30:58.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:30:58 smithi156 ceph-mon[129280]: pgmap v676: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 3.62k op/s 2023-07-06T20:30:58.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:30:58 smithi098 ceph-mon[128968]: pgmap v676: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 3.62k op/s 2023-07-06T20:30:58.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:30:58 smithi069 ceph-mon[126399]: pgmap v676: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 3.62k op/s 2023-07-06T20:30:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 550.0 s, 2512.8 tps, lat 40.043 ms stddev 27.519 2023-07-06T20:31:00.375 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:31:00.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:00 smithi098 ceph-mon[128968]: pgmap v677: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 MiB/s wr, 3.55k op/s 2023-07-06T20:31:00.491 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:00 smithi069 ceph-mon[126399]: pgmap v677: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 MiB/s wr, 3.55k op/s 2023-07-06T20:31:00.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:00 smithi156 ceph-mon[129280]: pgmap v677: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 MiB/s wr, 3.55k op/s 2023-07-06T20:31:00.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.766+0000 7ff2c57ff700 1 -- 172.21.15.69:0/1925798109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c00feb10 msgr2=0x7ff2c00fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:00.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.766+0000 7ff2c57ff700 1 --2- 172.21.15.69:0/1925798109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c00feb10 0x7ff2c00fef20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009990 tx=0x7ff2b0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.768+0000 7ff2c57ff700 1 -- 172.21.15.69:0/1925798109 shutdown_connections 2023-07-06T20:31:00.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.768+0000 7ff2c57ff700 1 --2- 172.21.15.69:0/1925798109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c0100690 0x7ff2c0106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.771 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.768+0000 7ff2c57ff700 1 --2- 172.21.15.69:0/1925798109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c00feb10 0x7ff2c00fef20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.768+0000 7ff2c57ff700 1 --2- 172.21.15.69:0/1925798109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2c00ffd00 0x7ff2c0100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.769+0000 7ff2c57ff700 1 -- 172.21.15.69:0/1925798109 >> 172.21.15.69:0/1925798109 conn(0x7ff2c00fa130 msgr2=0x7ff2c00fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:31:00.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.769+0000 7ff2c57ff700 1 -- 172.21.15.69:0/1925798109 shutdown_connections 2023-07-06T20:31:00.772 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.769+0000 7ff2c57ff700 1 -- 172.21.15.69:0/1925798109 wait complete. 2023-07-06T20:31:00.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.770+0000 7ff2c57ff700 1 Processor -- start 2023-07-06T20:31:00.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.770+0000 7ff2c57ff700 1 -- start start 2023-07-06T20:31:00.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2c57ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c00feb10 0x7ff2c0193710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2c57ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2c00ffd00 0x7ff2c0193c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2c57ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c0100690 0x7ff2c019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2c57ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c019b290 con 0x7ff2c00feb10 2023-07-06T20:31:00.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2c57ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c019b3d0 con 0x7ff2c00ffd00 2023-07-06T20:31:00.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2c57ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c019b560 con 0x7ff2c0100690 2023-07-06T20:31:00.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2b67fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2c00ffd00 0x7ff2c0193c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2beffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c00feb10 0x7ff2c0193710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2bf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c0100690 0x7ff2c019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2b67fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2c00ffd00 0x7ff2c0193c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:44706/0 (socket says 172.21.15.69:44706) 2023-07-06T20:31:00.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2b67fc700 1 -- 172.21.15.69:0/2790549542 learned_addr learned my addr 172.21.15.69:0/2790549542 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:31:00.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2bf7fe700 1 -- 172.21.15.69:0/2790549542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2c00ffd00 msgr2=0x7ff2c0193c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:00.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2bf7fe700 1 --2- 172.21.15.69:0/2790549542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2c00ffd00 0x7ff2c0193c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2bf7fe700 1 -- 172.21.15.69:0/2790549542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c00feb10 msgr2=0x7ff2c0193710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:00.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2bf7fe700 1 --2- 172.21.15.69:0/2790549542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c00feb10 0x7ff2c0193710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.771+0000 7ff2bf7fe700 1 -- 172.21.15.69:0/2790549542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00095f0 con 0x7ff2c0100690 2023-07-06T20:31:00.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.772+0000 7ff2bf7fe700 1 --2- 172.21.15.69:0/2790549542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c0100690 0x7ff2c019acc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8007f10 tx=0x7ff2b800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:31:00.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.772+0000 7ff2bcff9700 1 -- 172.21.15.69:0/2790549542 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8004db0 con 0x7ff2c0100690 2023-07-06T20:31:00.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.772+0000 7ff2bcff9700 1 -- 172.21.15.69:0/2790549542 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b800f040 con 0x7ff2c0100690 2023-07-06T20:31:00.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.772+0000 7ff2bcff9700 1 -- 172.21.15.69:0/2790549542 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8005990 con 0x7ff2c0100690 2023-07-06T20:31:00.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.772+0000 7ff2c57ff700 1 -- 172.21.15.69:0/2790549542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c019b840 con 0x7ff2c0100690 2023-07-06T20:31:00.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.772+0000 7ff2c57ff700 1 -- 172.21.15.69:0/2790549542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c019bd00 con 0x7ff2c0100690 2023-07-06T20:31:00.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.779+0000 7ff2d9215b80 1 Processor -- start 2023-07-06T20:31:00.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.780+0000 7ff2bcff9700 1 -- 172.21.15.69:0/2790549542 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff2b800db20 con 0x7ff2c0100690 2023-07-06T20:31:00.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.780+0000 7ff2d9215b80 1 -- start start 2023-07-06T20:31:00.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.781+0000 7ff2d9215b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627ecca9730 0x5627ecca9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.781+0000 7ff2d9215b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaa080 0x5627eccaa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.783 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.781+0000 7ff2d9215b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccaaba0 0x5627eccab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.781+0000 7ff2d9215b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ecba5cb0 con 0x5627eccaa080 2023-07-06T20:31:00.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.781+0000 7ff2d9215b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ecbf8a40 con 0x5627eccaaba0 2023-07-06T20:31:00.784 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.781+0000 7ff2d9215b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ecbf1ed0 con 0x5627ecca9730 2023-07-06T20:31:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.781+0000 7ff2b67fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccaaba0 0x5627eccab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.781+0000 7ff2b67fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccaaba0 0x5627eccab040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:44722/0 (socket says 172.21.15.69:44722) 2023-07-06T20:31:00.785 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.781+0000 7ff2b67fc700 1 -- 172.21.15.69:0/3671423771 learned_addr learned my addr 172.21.15.69:0/3671423771 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:31:00.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.782+0000 7ff2beffd700 1 --2- 172.21.15.69:0/3671423771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaa080 0x5627eccaa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.782+0000 7ff2b67fc700 1 -- 172.21.15.69:0/3671423771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627ecca9730 msgr2=0x5627ecca9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:00.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.782+0000 7ff2b67fc700 1 --2- 172.21.15.69:0/3671423771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627ecca9730 0x5627ecca9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.782+0000 7ff2b67fc700 1 -- 172.21.15.69:0/3671423771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaa080 msgr2=0x5627eccaa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:00.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.782+0000 7ff2b67fc700 1 --2- 172.21.15.69:0/3671423771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaa080 0x5627eccaa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.782+0000 7ff2bcff9700 1 --2- 172.21.15.69:0/2790549542 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff2ac0a11d0 0x7ff2ac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.782+0000 7ff2b67fc700 1 -- 172.21.15.69:0/3671423771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627eca1fde0 con 0x5627eccaaba0 2023-07-06T20:31:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.782+0000 7ff2bcff9700 1 -- 172.21.15.69:0/2790549542 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7ff2b80cc1e0 con 0x7ff2c0100690 2023-07-06T20:31:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.782+0000 7ff2b67fc700 1 --2- 172.21.15.69:0/3671423771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccaaba0 0x5627eccab040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800aee0 tx=0x7ff2a80116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca156806de6f06cd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:31:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.782+0000 7ff2bf7fe700 1 --2- 172.21.15.69:0/2790549542 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff2ac0a11d0 0x7ff2ac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.783+0000 7ff2b57fa700 1 -- 172.21.15.69:0/3671423771 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8011ee0 con 0x5627eccaaba0 2023-07-06T20:31:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.783+0000 7ff2b57fa700 1 -- 172.21.15.69:0/3671423771 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2a8012910 con 0x5627eccaaba0 2023-07-06T20:31:00.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.783+0000 7ff2b57fa700 1 -- 172.21.15.69:0/3671423771 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a800f780 con 0x5627eccaaba0 2023-07-06T20:31:00.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.783+0000 7ff2d9215b80 1 -- 172.21.15.69:0/3671423771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccaaba0 msgr2=0x5627eccab040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:00.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.783+0000 7ff2bf7fe700 1 --2- 172.21.15.69:0/2790549542 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff2ac0a11d0 0x7ff2ac0a3680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8099260 tx=0x7ff2b80053b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:31:00.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.783+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/3671423771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccaaba0 0x5627eccab040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800aee0 tx=0x7ff2a80116c0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.784+0000 7ff2d9215b80 1 -- 172.21.15.69:0/3671423771 shutdown_connections 2023-07-06T20:31:00.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.784+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/3671423771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccaaba0 0x5627eccab040 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.784+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/3671423771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627ecca9730 0x5627ecca9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.784+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/3671423771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaa080 0x5627eccaa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.784+0000 7ff2d9215b80 1 -- 172.21.15.69:0/3671423771 >> 172.21.15.69:0/3671423771 conn(0x5627ecc96640 msgr2=0x5627ecc9b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:31:00.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.784+0000 7ff2d9215b80 1 -- 172.21.15.69:0/3671423771 shutdown_connections 2023-07-06T20:31:00.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.784+0000 7ff2d9215b80 1 -- 172.21.15.69:0/3671423771 wait complete. 2023-07-06T20:31:00.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.784+0000 7ff2d9215b80 1 -- start start 2023-07-06T20:31:00.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.784+0000 7ff2d9215b80 1 Processor -- start 2023-07-06T20:31:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.785+0000 7ff2d9215b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627ecca9730 0x5627eccb3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.785+0000 7ff2d9215b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 0x5627eccb3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.785+0000 7ff2d9215b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccaaba0 0x5627eccba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.785+0000 7ff2d9215b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ecb91ac0 con 0x5627ecca9730 2023-07-06T20:31:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.785+0000 7ff2d9215b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ecb917c0 con 0x5627eccaaba0 2023-07-06T20:31:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.785+0000 7ff2d9215b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627eca75920 con 0x5627eccaa080 2023-07-06T20:31:00.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.785+0000 7ff2d9215b80 10 client.0 _ll_register_callbacks cb 0x5627ecb8f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:31:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.785+0000 7ff2beffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 0x5627eccb3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.785+0000 7ff2b67fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627ecca9730 0x5627eccb3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.785+0000 7ff2beffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 0x5627eccb3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:42174/0 (socket says 172.21.15.69:42174) 2023-07-06T20:31:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.785+0000 7ff2beffd700 1 -- 172.21.15.69:0/668455253 learned_addr learned my addr 172.21.15.69:0/668455253 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:31:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.786+0000 7ff2b67fc700 1 --2- 172.21.15.69:0/668455253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccaaba0 0x5627eccba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.786+0000 7ff2beffd700 1 -- 172.21.15.69:0/668455253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccaaba0 msgr2=0x5627eccba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:00.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.786+0000 7ff2beffd700 1 --2- 172.21.15.69:0/668455253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccaaba0 0x5627eccba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.786+0000 7ff2beffd700 1 -- 172.21.15.69:0/668455253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627ecca9730 msgr2=0x5627eccb3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:00.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.786+0000 7ff2beffd700 1 --2- 172.21.15.69:0/668455253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627ecca9730 0x5627eccb3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.786+0000 7ff2beffd700 1 -- 172.21.15.69:0/668455253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a800abc0 con 0x5627eccaa080 2023-07-06T20:31:00.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.786+0000 7ff2beffd700 1 --2- 172.21.15.69:0/668455253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 0x5627eccb3660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0015000 tx=0x7ff2b0005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f030e219f1d87207 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:31:00.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.786+0000 7ff2a67fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:31:00.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.786+0000 7ff2a67fc700 1 -- 172.21.15.69:0/668455253 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b001d070 con 0x5627eccaa080 2023-07-06T20:31:00.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.787+0000 7ff2d9215b80 1 -- 172.21.15.69:0/668455253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627ecb4b4f0 con 0x5627eccaa080 2023-07-06T20:31:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.787+0000 7ff2a67fc700 1 -- 172.21.15.69:0/668455253 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7ff2b000bc40 con 0x5627eccaa080 2023-07-06T20:31:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.787+0000 7ff2a67fc700 1 -- 172.21.15.69:0/668455253 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2b000be00 con 0x5627eccaa080 2023-07-06T20:31:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.787+0000 7ff2d9215b80 10 client.25720 fetch_fsmap learned FSMap version 30 2023-07-06T20:31:00.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.787+0000 7ff2d9215b80 1 -- 172.21.15.69:0/668455253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5627eca08c60 con 0x5627eccaa080 2023-07-06T20:31:00.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.796+0000 7ff2a67fc700 1 -- 172.21.15.69:0/668455253 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627ecb4b4f0 con 0x5627eccaa080 2023-07-06T20:31:00.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.797+0000 7ff2a67fc700 1 -- 172.21.15.69:0/668455253 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff2b0004740 con 0x5627eccaa080 2023-07-06T20:31:00.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.797+0000 7ff2d9215b80 10 client.25720 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:31:00.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.797+0000 7ff2d9215b80 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:31:00.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.797+0000 7ff2d9215b80 20 client.25720 populate_metadata read hostname 'smithi069' 2023-07-06T20:31:00.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.797+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/668455253 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5627eccbeec0 0x5627eccc12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.797+0000 7ff2d9215b80 4 client.25720 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:31:00.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.797+0000 7ff2d9215b80 1 -- 172.21.15.69:0/668455253 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627ecc6fd60 con 0x5627eccbeec0 2023-07-06T20:31:00.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.798+0000 7ff2b67fc700 1 --2- 172.21.15.69:0/668455253 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5627eccbeec0 0x5627eccc12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.802 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.800+0000 7ff2b67fc700 1 --2- 172.21.15.69:0/668455253 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5627eccbeec0 0x5627eccc12d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a65ec04913cba7f2 server_cookie=1571bf90e7e8ea90 in_seq=0 out_seq=0 2023-07-06T20:31:00.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.800+0000 7ff2a67fc700 10 client.25720 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:31:00.803 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.801+0000 7ff2a67fc700 1 -- 172.21.15.69:0/668455253 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5627ecc6fd60 con 0x5627eccbeec0 2023-07-06T20:31:00.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.801+0000 7ff2a67fc700 10 client.25720 handle_command_reply: tid=0 2023-07-06T20:31:00.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.801+0000 7ff2d9215b80 1 client.25720 shutdown 2023-07-06T20:31:00.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.802+0000 7ff2d9215b80 1 -- 172.21.15.69:0/668455253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 msgr2=0x5627eccb3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:00.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.802+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/668455253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 0x5627eccb3660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0015000 tx=0x7ff2b0005800 comp rx=0 tx=0).stop 2023-07-06T20:31:00.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.802+0000 7ff2d9215b80 1 -- 172.21.15.69:0/668455253 shutdown_connections 2023-07-06T20:31:00.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.802+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/668455253 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5627eccbeec0 0x5627eccc12d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.802+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/668455253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccaaba0 0x5627eccba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.802+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/668455253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627ecca9730 0x5627eccb3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.802+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/668455253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 0x5627eccb3660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.802+0000 7ff2d9215b80 1 -- 172.21.15.69:0/668455253 >> 172.21.15.69:0/668455253 conn(0x5627ecc96640 msgr2=0x5627ecc9b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:31:00.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.802+0000 7ff2a67fc700 0 client.25720 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:31:00.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.802+0000 7ff2d9215b80 1 -- 172.21.15.69:0/668455253 shutdown_connections 2023-07-06T20:31:00.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.802+0000 7ff2d9215b80 1 -- 172.21.15.69:0/668455253 wait complete. 2023-07-06T20:31:00.807 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.803+0000 7ff2d9215b80 20 client.25720 trim_cache size 0 max 16384 2023-07-06T20:31:00.827 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.825+0000 7ff2d9215b80 1 Processor -- start 2023-07-06T20:31:00.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.825+0000 7ff2d9215b80 1 -- start start 2023-07-06T20:31:00.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.825+0000 7ff2d9215b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627ecca9730 0x5627eccb0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.825+0000 7ff2d9215b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 0x5627eccba660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.825+0000 7ff2d9215b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaaba0 0x5627eccadb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.825+0000 7ff2d9215b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627eccb2c90 con 0x5627eccaaba0 2023-07-06T20:31:00.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.825+0000 7ff2d9215b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ecc9c810 con 0x5627ecca9730 2023-07-06T20:31:00.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.825+0000 7ff2d9215b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627eccb4110 con 0x5627eccaa080 2023-07-06T20:31:00.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.826+0000 7ff2b67fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627ecca9730 0x5627eccb0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.826+0000 7ff2beffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 0x5627eccba660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.826+0000 7ff2beffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 0x5627eccba660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:42186/0 (socket says 172.21.15.69:42186) 2023-07-06T20:31:00.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.826+0000 7ff2beffd700 1 -- 172.21.15.69:0/2022275491 learned_addr learned my addr 172.21.15.69:0/2022275491 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:31:00.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.827+0000 7ff2beffd700 1 -- 172.21.15.69:0/2022275491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627ecca9730 msgr2=0x5627eccb0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:00.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.827+0000 7ff2beffd700 1 --2- 172.21.15.69:0/2022275491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627ecca9730 0x5627eccb0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.827+0000 7ff2beffd700 1 -- 172.21.15.69:0/2022275491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaaba0 msgr2=0x5627eccadb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:00.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.827+0000 7ff2beffd700 1 --2- 172.21.15.69:0/2022275491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaaba0 0x5627eccadb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.827+0000 7ff2beffd700 1 -- 172.21.15.69:0/2022275491 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627eca46250 con 0x5627eccaa080 2023-07-06T20:31:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.827+0000 7ff2beffd700 1 --2- 172.21.15.69:0/2022275491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 0x5627eccba660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005a60 tx=0x7ff2b0004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a1409e50adcd0a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:31:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.828+0000 7ff2b5ffb700 1 -- 172.21.15.69:0/2022275491 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b001d030 con 0x5627eccaa080 2023-07-06T20:31:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.828+0000 7ff2b5ffb700 1 -- 172.21.15.69:0/2022275491 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b001e040 con 0x5627eccaa080 2023-07-06T20:31:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.828+0000 7ff2b5ffb700 1 -- 172.21.15.69:0/2022275491 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0003af0 con 0x5627eccaa080 2023-07-06T20:31:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.829+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2022275491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 msgr2=0x5627eccba660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.829+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/2022275491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 0x5627eccba660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005a60 tx=0x7ff2b0004f80 comp rx=0 tx=0).stop 2023-07-06T20:31:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.829+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2022275491 shutdown_connections 2023-07-06T20:31:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.829+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/2022275491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaaba0 0x5627eccadb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.829+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/2022275491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627ecca9730 0x5627eccb0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.829+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/2022275491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccaa080 0x5627eccba660 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.829+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2022275491 >> 172.21.15.69:0/2022275491 conn(0x5627ecc96640 msgr2=0x5627eccbaf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:31:00.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.830+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2022275491 shutdown_connections 2023-07-06T20:31:00.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.830+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2022275491 wait complete. 2023-07-06T20:31:00.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.830+0000 7ff2d9215b80 1 -- start start 2023-07-06T20:31:00.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.830+0000 7ff2d9215b80 1 Processor -- start 2023-07-06T20:31:00.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2d9215b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaa080 0x5627eccb4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2d9215b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccb4dc0 0x5627eccb9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2d9215b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccb5230 0x5627eccba2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2d9215b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627eccba9c0 con 0x5627eccaa080 2023-07-06T20:31:00.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2d9215b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ecc9fc40 con 0x5627eccb4dc0 2023-07-06T20:31:00.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2d9215b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ecc9fe00 con 0x5627eccb5230 2023-07-06T20:31:00.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2d9215b80 10 client.0 _ll_register_callbacks cb 0x5627ecb8f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:31:00.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2b67fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaa080 0x5627eccb4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2beffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccb4dc0 0x5627eccb9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2beffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccb4dc0 0x5627eccb9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:44748/0 (socket says 172.21.15.69:44748) 2023-07-06T20:31:00.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2b67fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaa080 0x5627eccb4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42224/0 (socket says 172.21.15.69:42224) 2023-07-06T20:31:00.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2beffd700 1 -- 172.21.15.69:0/2980966712 learned_addr learned my addr 172.21.15.69:0/2980966712 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:31:00.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.831+0000 7ff2b67fc700 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccb5230 0x5627eccba2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.832+0000 7ff2beffd700 1 -- 172.21.15.69:0/2980966712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccb5230 msgr2=0x5627eccba2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.832+0000 7ff2beffd700 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccb5230 0x5627eccba2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.832+0000 7ff2beffd700 1 -- 172.21.15.69:0/2980966712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaa080 msgr2=0x5627eccb4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.832+0000 7ff2beffd700 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaa080 0x5627eccb4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.832+0000 7ff2beffd700 1 -- 172.21.15.69:0/2980966712 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627eca08c60 con 0x5627eccb4dc0 2023-07-06T20:31:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.832+0000 7ff2b67fc700 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaa080 0x5627eccb4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:31:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.832+0000 7ff2beffd700 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccb4dc0 0x5627eccb9d30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff2b003b040 tx=0x7ff2b0005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8305f3a1eb67ed1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:31:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.832+0000 7ff2a67fc700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:31:00.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.832+0000 7ff2a67fc700 1 -- 172.21.15.69:0/2980966712 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b001d040 con 0x5627eccb4dc0 2023-07-06T20:31:00.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.832+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2980966712 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2a8012910 con 0x5627eccb4dc0 2023-07-06T20:31:00.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.832+0000 7ff2a67fc700 1 -- 172.21.15.69:0/2980966712 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7ff2b0000e40 con 0x5627eccb4dc0 2023-07-06T20:31:00.839 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.833+0000 7ff2a67fc700 1 -- 172.21.15.69:0/2980966712 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2b000bc40 con 0x5627eccb4dc0 2023-07-06T20:31:00.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.833+0000 7ff2d9215b80 10 client.25784 fetch_fsmap learned FSMap version 30 2023-07-06T20:31:00.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.833+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2980966712 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5627eca1e2b0 con 0x5627eccb4dc0 2023-07-06T20:31:00.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.833+0000 7ff2a67fc700 1 -- 172.21.15.69:0/2980966712 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2a8012910 con 0x5627eccb4dc0 2023-07-06T20:31:00.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.833+0000 7ff2a67fc700 1 -- 172.21.15.69:0/2980966712 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff2b003f070 con 0x5627eccb4dc0 2023-07-06T20:31:00.840 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.833+0000 7ff2d9215b80 10 client.25784 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:31:00.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.833+0000 7ff2d9215b80 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:31:00.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.833+0000 7ff2d9215b80 20 client.25784 populate_metadata read hostname 'smithi069' 2023-07-06T20:31:00.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.833+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5627ecca1220 0x5627ecca3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:00.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.833+0000 7ff2d9215b80 4 client.25784 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:31:00.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.833+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2980966712 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5627ecbeaeb0 con 0x5627ecca1220 2023-07-06T20:31:00.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.833+0000 7ff2b67fc700 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5627ecca1220 0x5627ecca3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:00.841 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.834+0000 7ff2a67fc700 10 client.25784 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:31:00.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.834+0000 7ff2b67fc700 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5627ecca1220 0x5627ecca3630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ddb89f0590e4e13 server_cookie=ffa8c71544aff5a in_seq=0 out_seq=0 2023-07-06T20:31:00.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.834+0000 7ff2a67fc700 1 -- 172.21.15.69:0/2980966712 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5627ecbeaeb0 con 0x5627ecca1220 2023-07-06T20:31:00.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.834+0000 7ff2a67fc700 10 client.25784 handle_command_reply: tid=0 2023-07-06T20:31:00.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.834+0000 7ff2d9215b80 1 client.25784 shutdown 2023-07-06T20:31:00.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.834+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2980966712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccb4dc0 msgr2=0x5627eccb9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:00.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.834+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccb4dc0 0x5627eccb9d30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff2b003b040 tx=0x7ff2b0005ef0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.842 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.835+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2980966712 shutdown_connections 2023-07-06T20:31:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.835+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5627ecca1220 0x5627ecca3630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.835+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627eccb5230 0x5627eccba2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.835+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627eccaa080 0x5627eccb4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.835+0000 7ff2d9215b80 1 --2- 172.21.15.69:0/2980966712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627eccb4dc0 0x5627eccb9d30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.835+0000 7ff2a67fc700 0 client.25784 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:31:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.835+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2980966712 >> 172.21.15.69:0/2980966712 conn(0x5627ecc96640 msgr2=0x5627eccb0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:31:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.835+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2980966712 shutdown_connections 2023-07-06T20:31:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.835+0000 7ff2d9215b80 1 -- 172.21.15.69:0/2980966712 wait complete. 2023-07-06T20:31:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.835+0000 7ff2d9215b80 20 client.25784 trim_cache size 0 max 16384 2023-07-06T20:31:00.844 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:31:00.845 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:31:00.845 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:31:00.845 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:31:00.845 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:31:00.845 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:31:00.845 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:31:00.845 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:31:00.845 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:31:00.846 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:31:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.837+0000 7ff2c57ff700 1 -- 172.21.15.69:0/2790549542 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff2ac0a11d0 msgr2=0x7ff2ac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.837+0000 7ff2c57ff700 1 --2- 172.21.15.69:0/2790549542 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff2ac0a11d0 0x7ff2ac0a3680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8099260 tx=0x7ff2b80053b0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.837+0000 7ff2c57ff700 1 -- 172.21.15.69:0/2790549542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c0100690 msgr2=0x7ff2c019acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:00.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.837+0000 7ff2c57ff700 1 --2- 172.21.15.69:0/2790549542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c0100690 0x7ff2c019acc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8007f10 tx=0x7ff2b800d3b0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.837+0000 7ff2c57ff700 1 -- 172.21.15.69:0/2790549542 shutdown_connections 2023-07-06T20:31:00.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.837+0000 7ff2c57ff700 1 --2- 172.21.15.69:0/2790549542 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ff2ac0a11d0 0x7ff2ac0a3680 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.837+0000 7ff2c57ff700 1 --2- 172.21.15.69:0/2790549542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c0100690 0x7ff2c019acc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.847 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.837+0000 7ff2c57ff700 1 --2- 172.21.15.69:0/2790549542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c00feb10 0x7ff2c0193710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.837+0000 7ff2c57ff700 1 --2- 172.21.15.69:0/2790549542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2c00ffd00 0x7ff2c0193c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:00.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.837+0000 7ff2c57ff700 1 -- 172.21.15.69:0/2790549542 >> 172.21.15.69:0/2790549542 conn(0x7ff2c00fa130 msgr2=0x7ff2c00fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:31:00.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.839+0000 7ff2c57ff700 1 -- 172.21.15.69:0/2790549542 shutdown_connections 2023-07-06T20:31:00.848 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:00.839+0000 7ff2c57ff700 1 -- 172.21.15.69:0/2790549542 wait complete. 2023-07-06T20:31:00.884 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:31:01.241 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:01 smithi069 ceph-mon[126399]: pgmap v678: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 46 MiB/s wr, 3.21k op/s 2023-07-06T20:31:01.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:01 smithi156 ceph-mon[129280]: pgmap v678: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 46 MiB/s wr, 3.21k op/s 2023-07-06T20:31:01.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:01 smithi098 ceph-mon[128968]: pgmap v678: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 46 MiB/s wr, 3.21k op/s 2023-07-06T20:31:03.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:03 smithi069 ceph-mon[126399]: pgmap v679: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 2.00k op/s 2023-07-06T20:31:03.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:03 smithi156 ceph-mon[129280]: pgmap v679: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 2.00k op/s 2023-07-06T20:31:03.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:03 smithi098 ceph-mon[128968]: pgmap v679: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 2.00k op/s 2023-07-06T20:31:04.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 555.0 s, 1674.0 tps, lat 59.548 ms stddev 86.381 2023-07-06T20:31:05.619 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:05 smithi156 ceph-mon[129280]: pgmap v680: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 MiB/s wr, 1.20k op/s 2023-07-06T20:31:05.619 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:05 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:31:05.632 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:05 smithi098 ceph-mon[128968]: pgmap v680: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 MiB/s wr, 1.20k op/s 2023-07-06T20:31:05.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:05 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:31:05.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:05 smithi069 ceph-mon[126399]: pgmap v680: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 MiB/s wr, 1.20k op/s 2023-07-06T20:31:05.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:05 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:31:07.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:07 smithi069 ceph-mon[126399]: pgmap v681: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 1.16k op/s 2023-07-06T20:31:07.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:31:07.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:31:07.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:31:07.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:07 smithi156 ceph-mon[129280]: pgmap v681: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 1.16k op/s 2023-07-06T20:31:07.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:31:07.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:31:07.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:31:07.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:07 smithi098 ceph-mon[128968]: pgmap v681: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 1.16k op/s 2023-07-06T20:31:07.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:31:07.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:31:07.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:31:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 560.0 s, 2291.2 tps, lat 43.704 ms stddev 31.159 2023-07-06T20:31:09.766 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:09 smithi069 ceph-mon[126399]: pgmap v682: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 675 op/s 2023-07-06T20:31:09.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:09 smithi156 ceph-mon[129280]: pgmap v682: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 675 op/s 2023-07-06T20:31:09.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:09 smithi098 ceph-mon[128968]: pgmap v682: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 675 op/s 2023-07-06T20:31:11.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:11 smithi069 ceph-mon[126399]: pgmap v683: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 729 op/s 2023-07-06T20:31:11.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:11 smithi156 ceph-mon[129280]: pgmap v683: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 729 op/s 2023-07-06T20:31:11.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:11 smithi098 ceph-mon[128968]: pgmap v683: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 729 op/s 2023-07-06T20:31:13.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:13 smithi156 ceph-mon[129280]: pgmap v684: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 27 MiB/s wr, 1.21k op/s 2023-07-06T20:31:13.879 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:13 smithi098 ceph-mon[128968]: pgmap v684: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 27 MiB/s wr, 1.21k op/s 2023-07-06T20:31:14.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:13 smithi069 ceph-mon[126399]: pgmap v684: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 27 MiB/s wr, 1.21k op/s 2023-07-06T20:31:14.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 565.0 s, 677.0 tps, lat 144.826 ms stddev 189.166 2023-07-06T20:31:15.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:15 smithi156 ceph-mon[129280]: pgmap v685: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 28 MiB/s wr, 1.40k op/s 2023-07-06T20:31:15.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:15 smithi098 ceph-mon[128968]: pgmap v685: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 28 MiB/s wr, 1.40k op/s 2023-07-06T20:31:16.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:15 smithi069 ceph-mon[126399]: pgmap v685: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 28 MiB/s wr, 1.40k op/s 2023-07-06T20:31:18.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:17 smithi069 ceph-mon[126399]: pgmap v686: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 43 MiB/s wr, 3.14k op/s 2023-07-06T20:31:18.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:17 smithi156 ceph-mon[129280]: pgmap v686: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 43 MiB/s wr, 3.14k op/s 2023-07-06T20:31:18.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:17 smithi098 ceph-mon[128968]: pgmap v686: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 43 MiB/s wr, 3.14k op/s 2023-07-06T20:31:19.234 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 570.0 s, 1294.0 tps, lat 78.676 ms stddev 213.359 2023-07-06T20:31:20.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:19 smithi069 ceph-mon[126399]: pgmap v687: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 MiB/s wr, 3.74k op/s 2023-07-06T20:31:20.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:19 smithi156 ceph-mon[129280]: pgmap v687: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 MiB/s wr, 3.74k op/s 2023-07-06T20:31:20.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:19 smithi098 ceph-mon[128968]: pgmap v687: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 MiB/s wr, 3.74k op/s 2023-07-06T20:31:22.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:21 smithi069 ceph-mon[126399]: pgmap v688: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 MiB/s wr, 3.93k op/s 2023-07-06T20:31:22.111 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:21 smithi156 ceph-mon[129280]: pgmap v688: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 MiB/s wr, 3.93k op/s 2023-07-06T20:31:22.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:21 smithi098 ceph-mon[128968]: pgmap v688: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 MiB/s wr, 3.93k op/s 2023-07-06T20:31:24.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:23 smithi069 ceph-mon[126399]: pgmap v689: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 44 MiB/s wr, 3.90k op/s 2023-07-06T20:31:24.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:23 smithi156 ceph-mon[129280]: pgmap v689: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 44 MiB/s wr, 3.90k op/s 2023-07-06T20:31:24.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:23 smithi098 ceph-mon[128968]: pgmap v689: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 44 MiB/s wr, 3.90k op/s 2023-07-06T20:31:24.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 575.0 s, 2492.5 tps, lat 40.325 ms stddev 28.591 2023-07-06T20:31:26.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:25 smithi098 ceph-mon[128968]: pgmap v690: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 3.43k op/s 2023-07-06T20:31:26.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:25 smithi069 ceph-mon[126399]: pgmap v690: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 3.43k op/s 2023-07-06T20:31:26.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:25 smithi156 ceph-mon[129280]: pgmap v690: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 3.43k op/s 2023-07-06T20:31:28.265 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:27 smithi069 ceph-mon[126399]: pgmap v691: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 MiB/s wr, 3.29k op/s 2023-07-06T20:31:28.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:27 smithi156 ceph-mon[129280]: pgmap v691: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 MiB/s wr, 3.29k op/s 2023-07-06T20:31:28.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:27 smithi098 ceph-mon[128968]: pgmap v691: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 MiB/s wr, 3.29k op/s 2023-07-06T20:31:29.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 580.0 s, 1806.0 tps, lat 55.256 ms stddev 61.862 2023-07-06T20:31:30.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:29 smithi069 ceph-mon[126399]: pgmap v692: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 31 MiB/s wr, 1.61k op/s 2023-07-06T20:31:30.369 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:29 smithi156 ceph-mon[129280]: pgmap v692: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 31 MiB/s wr, 1.61k op/s 2023-07-06T20:31:30.383 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:29 smithi098 ceph-mon[128968]: pgmap v692: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 31 MiB/s wr, 1.61k op/s 2023-07-06T20:31:30.886 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:31:31.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.265+0000 7fefdddff700 1 -- 172.21.15.69:0/3937020664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd8075000 msgr2=0x7fefd8075410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:31.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.265+0000 7fefdddff700 1 --2- 172.21.15.69:0/3937020664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd8075000 0x7fefd8075410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009990 tx=0x7fefc8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.265+0000 7fefdddff700 1 -- 172.21.15.69:0/3937020664 shutdown_connections 2023-07-06T20:31:31.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.265+0000 7fefdddff700 1 --2- 172.21.15.69:0/3937020664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefd8076bf0 0x7fefd8077480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.265+0000 7fefdddff700 1 --2- 172.21.15.69:0/3937020664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd8075000 0x7fefd8075410 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.265+0000 7fefdddff700 1 --2- 172.21.15.69:0/3937020664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefd8076240 0x7fefd80766b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.265+0000 7fefdddff700 1 -- 172.21.15.69:0/3937020664 >> 172.21.15.69:0/3937020664 conn(0x7fefd80fd480 msgr2=0x7fefd80ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:31:31.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.265+0000 7fefdddff700 1 -- 172.21.15.69:0/3937020664 shutdown_connections 2023-07-06T20:31:31.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.265+0000 7fefdddff700 1 -- 172.21.15.69:0/3937020664 wait complete. 2023-07-06T20:31:31.269 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.267+0000 7fefdddff700 1 Processor -- start 2023-07-06T20:31:31.269 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.267+0000 7fefdddff700 1 -- start start 2023-07-06T20:31:31.269 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.267+0000 7fefdddff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd8075000 0x7fefd819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.270 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.267+0000 7fefdddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefd8076240 0x7fefd819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.270 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.267+0000 7fefdddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefd8076bf0 0x7fefd81a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.270 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.267+0000 7fefdddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd81a3b10 con 0x7fefd8076240 2023-07-06T20:31:31.270 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.267+0000 7fefdddff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd81a3c50 con 0x7fefd8075000 2023-07-06T20:31:31.270 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.267+0000 7fefdddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd81a3de0 con 0x7fefd8076bf0 2023-07-06T20:31:31.271 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.267+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd8075000 0x7fefd819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.271 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.267+0000 7fefd7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefd8076bf0 0x7fefd81a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.271 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.267+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd8075000 0x7fefd819bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:34354/0 (socket says 172.21.15.69:34354) 2023-07-06T20:31:31.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.267+0000 7fefd77fe700 1 -- 172.21.15.69:0/618050882 learned_addr learned my addr 172.21.15.69:0/618050882 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:31:31.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.268+0000 7fefd6ffd700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefd8076240 0x7fefd819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.268+0000 7fefd7fff700 1 -- 172.21.15.69:0/618050882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd8075000 msgr2=0x7fefd819bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:31.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.268+0000 7fefd7fff700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd8075000 0x7fefd819bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.268+0000 7fefd7fff700 1 -- 172.21.15.69:0/618050882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefd8076240 msgr2=0x7fefd819c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:31.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.268+0000 7fefd7fff700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefd8076240 0x7fefd819c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.268+0000 7fefd7fff700 1 -- 172.21.15.69:0/618050882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc009520 con 0x7fefd8076bf0 2023-07-06T20:31:31.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.268+0000 7fefd7fff700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefd8076bf0 0x7fefd81a3540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fefd0007f10 tx=0x7fefd000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:31:31.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.268+0000 7fefd4ff9700 1 -- 172.21.15.69:0/618050882 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0004db0 con 0x7fefd8076bf0 2023-07-06T20:31:31.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.268+0000 7fefd4ff9700 1 -- 172.21.15.69:0/618050882 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefd000f040 con 0x7fefd8076bf0 2023-07-06T20:31:31.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.269+0000 7fefdddff700 1 -- 172.21.15.69:0/618050882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc80095f0 con 0x7fefd8076bf0 2023-07-06T20:31:31.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.269+0000 7fefd4ff9700 1 -- 172.21.15.69:0/618050882 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0005990 con 0x7fefd8076bf0 2023-07-06T20:31:31.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.269+0000 7fefdddff700 1 -- 172.21.15.69:0/618050882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd81a4440 con 0x7fefd8076bf0 2023-07-06T20:31:31.279 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.277+0000 7feff1802b80 1 Processor -- start 2023-07-06T20:31:31.282 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.280+0000 7fefd4ff9700 1 -- 172.21.15.69:0/618050882 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fefd000db20 con 0x7fefd8076bf0 2023-07-06T20:31:31.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.281+0000 7feff1802b80 1 -- start start 2023-07-06T20:31:31.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.281+0000 7feff1802b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0715f20 0x558ba0716330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.281+0000 7feff1802b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 0x558ba0716e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.283 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.281+0000 7feff1802b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0717390 0x558ba0717830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.281+0000 7feff1802b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba061acb0 con 0x558ba0716870 2023-07-06T20:31:31.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.281+0000 7feff1802b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba066da40 con 0x558ba0717390 2023-07-06T20:31:31.284 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.281+0000 7feff1802b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba0666ed0 con 0x558ba0715f20 2023-07-06T20:31:31.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.281+0000 7fefd6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0717390 0x558ba0717830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.281+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 0x558ba0716e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.285 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.282+0000 7fefd6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0717390 0x558ba0717830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:34362/0 (socket says 172.21.15.69:34362) 2023-07-06T20:31:31.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.282+0000 7fefd6ffd700 1 -- 172.21.15.69:0/1443655121 learned_addr learned my addr 172.21.15.69:0/1443655121 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:31:31.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.282+0000 7fefd6ffd700 1 --2- 172.21.15.69:0/1443655121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0715f20 0x558ba0716330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.282+0000 7fefd6ffd700 1 -- 172.21.15.69:0/1443655121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0715f20 msgr2=0x558ba0716330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:31.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.282+0000 7fefd6ffd700 1 --2- 172.21.15.69:0/1443655121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0715f20 0x558ba0716330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.282+0000 7fefd6ffd700 1 -- 172.21.15.69:0/1443655121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 msgr2=0x558ba0716e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:31.286 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.282+0000 7fefd6ffd700 1 --2- 172.21.15.69:0/1443655121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 0x558ba0716e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.282+0000 7fefd6ffd700 1 -- 172.21.15.69:0/1443655121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ba0494de0 con 0x558ba0717390 2023-07-06T20:31:31.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.282+0000 7fefd6ffd700 1 --2- 172.21.15.69:0/1443655121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0717390 0x558ba0717830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fefcc011a70 tx=0x7fefcc009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=159602038064b74e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:31:31.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.283+0000 7fefd4ff9700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fefc00a11d0 0x7fefc00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.283+0000 7fefbdffb700 1 -- 172.21.15.69:0/1443655121 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc011110 con 0x558ba0717390 2023-07-06T20:31:31.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.283+0000 7fefbdffb700 1 -- 172.21.15.69:0/1443655121 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefcc0044f0 con 0x558ba0717390 2023-07-06T20:31:31.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.283+0000 7fefbdffb700 1 -- 172.21.15.69:0/1443655121 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc00fcc0 con 0x558ba0717390 2023-07-06T20:31:31.287 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.283+0000 7fefd4ff9700 1 -- 172.21.15.69:0/618050882 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fefd00cc180 con 0x7fefd8076bf0 2023-07-06T20:31:31.288 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.283+0000 7fefd77fe700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fefc00a11d0 0x7fefc00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.288 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.284+0000 7feff1802b80 1 -- 172.21.15.69:0/1443655121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0717390 msgr2=0x558ba0717830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:31.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.284+0000 7feff1802b80 1 --2- 172.21.15.69:0/1443655121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0717390 0x558ba0717830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fefcc011a70 tx=0x7fefcc009a10 comp rx=0 tx=0).stop 2023-07-06T20:31:31.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.284+0000 7fefd77fe700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fefc00a11d0 0x7fefc00a3680 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fefc8006010 tx=0x7fefc800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:31:31.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.284+0000 7feff1802b80 1 -- 172.21.15.69:0/1443655121 shutdown_connections 2023-07-06T20:31:31.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.284+0000 7feff1802b80 1 --2- 172.21.15.69:0/1443655121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0717390 0x558ba0717830 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.284+0000 7feff1802b80 1 --2- 172.21.15.69:0/1443655121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0715f20 0x558ba0716330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.289 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.284+0000 7feff1802b80 1 --2- 172.21.15.69:0/1443655121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 0x558ba0716e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.290 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.284+0000 7feff1802b80 1 -- 172.21.15.69:0/1443655121 >> 172.21.15.69:0/1443655121 conn(0x558ba070b640 msgr2=0x558ba07109f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:31:31.290 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.284+0000 7feff1802b80 1 -- 172.21.15.69:0/1443655121 shutdown_connections 2023-07-06T20:31:31.290 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.284+0000 7feff1802b80 1 -- 172.21.15.69:0/1443655121 wait complete. 2023-07-06T20:31:31.290 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.285+0000 7feff1802b80 1 -- start start 2023-07-06T20:31:31.290 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.285+0000 7feff1802b80 1 Processor -- start 2023-07-06T20:31:31.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.286+0000 7feff1802b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0715f20 0x558ba071f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.286+0000 7feff1802b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0716870 0x558ba071fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.286+0000 7feff1802b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0717390 0x558ba0726e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.286+0000 7feff1802b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba0606ac0 con 0x558ba0717390 2023-07-06T20:31:31.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.286+0000 7feff1802b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba06067c0 con 0x558ba0716870 2023-07-06T20:31:31.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.286+0000 7feff1802b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba04ea920 con 0x558ba0715f20 2023-07-06T20:31:31.293 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.286+0000 7feff1802b80 10 client.0 _ll_register_callbacks cb 0x558ba0604760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:31:31.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.287+0000 7fefd6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0716870 0x558ba071fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.287+0000 7fefd6ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0716870 0x558ba071fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:34372/0 (socket says 172.21.15.69:34372) 2023-07-06T20:31:31.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.287+0000 7fefd6ffd700 1 -- 172.21.15.69:0/661602672 learned_addr learned my addr 172.21.15.69:0/661602672 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:31:31.294 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.287+0000 7fefd77fe700 1 --2- 172.21.15.69:0/661602672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0715f20 0x558ba071f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.287+0000 7fefd6ffd700 1 -- 172.21.15.69:0/661602672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0715f20 msgr2=0x558ba071f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:31.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.287+0000 7fefd6ffd700 1 --2- 172.21.15.69:0/661602672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0715f20 0x558ba071f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.287+0000 7fefd6ffd700 1 -- 172.21.15.69:0/661602672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0717390 msgr2=0x558ba0726e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:31:31.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.287+0000 7fefd6ffd700 1 --2- 172.21.15.69:0/661602672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0717390 0x558ba0726e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.287+0000 7fefd6ffd700 1 -- 172.21.15.69:0/661602672 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc019040 con 0x558ba0716870 2023-07-06T20:31:31.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.288+0000 7fefd6ffd700 1 --2- 172.21.15.69:0/661602672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0716870 0x558ba071fdd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00a010 tx=0x7fefcc0092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbbaf3772ea3f8ec server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:31:31.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.288+0000 7fefb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:31:31.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.288+0000 7fefb6ffd700 1 -- 172.21.15.69:0/661602672 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc025070 con 0x558ba0716870 2023-07-06T20:31:31.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.288+0000 7fefb6ffd700 1 -- 172.21.15.69:0/661602672 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fefcc00fc50 con 0x558ba0716870 2023-07-06T20:31:31.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.288+0000 7feff1802b80 1 -- 172.21.15.69:0/661602672 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ba05c04f0 con 0x558ba0716870 2023-07-06T20:31:31.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.289+0000 7fefb6ffd700 1 -- 172.21.15.69:0/661602672 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefcc00fe10 con 0x558ba0716870 2023-07-06T20:31:31.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.289+0000 7feff1802b80 10 client.25796 fetch_fsmap learned FSMap version 30 2023-07-06T20:31:31.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.289+0000 7feff1802b80 1 -- 172.21.15.69:0/661602672 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558ba047dc60 con 0x558ba0716870 2023-07-06T20:31:31.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.289+0000 7fefb6ffd700 1 -- 172.21.15.69:0/661602672 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558ba05c04f0 con 0x558ba0716870 2023-07-06T20:31:31.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.289+0000 7fefb6ffd700 1 -- 172.21.15.69:0/661602672 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fefcc01d070 con 0x558ba0716870 2023-07-06T20:31:31.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.289+0000 7feff1802b80 10 client.25796 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:31:31.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.289+0000 7feff1802b80 10 client.25796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:31:31.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.289+0000 7feff1802b80 20 client.25796 populate_metadata read hostname 'smithi069' 2023-07-06T20:31:31.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.290+0000 7feff1802b80 1 --2- 172.21.15.69:0/661602672 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558ba072b4a0 0x558ba072d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.290+0000 7feff1802b80 4 client.25796 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:31:31.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.290+0000 7feff1802b80 1 -- 172.21.15.69:0/661602672 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ba06e4d60 con 0x558ba072b4a0 2023-07-06T20:31:31.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.290+0000 7fefd77fe700 1 --2- 172.21.15.69:0/661602672 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558ba072b4a0 0x558ba072d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.291+0000 7fefd77fe700 1 --2- 172.21.15.69:0/661602672 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558ba072b4a0 0x558ba072d8b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670d9e3f41ebad29 server_cookie=534e7bbd3e031954 in_seq=0 out_seq=0 2023-07-06T20:31:31.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.291+0000 7fefb6ffd700 10 client.25796 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:31:31.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.292+0000 7fefb6ffd700 1 -- 172.21.15.69:0/661602672 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558ba06e4d60 con 0x558ba072b4a0 2023-07-06T20:31:31.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.292+0000 7fefb6ffd700 10 client.25796 handle_command_reply: tid=0 2023-07-06T20:31:31.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.292+0000 7feff1802b80 1 client.25796 shutdown 2023-07-06T20:31:31.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.293+0000 7feff1802b80 1 -- 172.21.15.69:0/661602672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0716870 msgr2=0x558ba071fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:31.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.293+0000 7feff1802b80 1 --2- 172.21.15.69:0/661602672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0716870 0x558ba071fdd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00a010 tx=0x7fefcc0092f0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.294+0000 7feff1802b80 1 -- 172.21.15.69:0/661602672 shutdown_connections 2023-07-06T20:31:31.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.294+0000 7feff1802b80 1 --2- 172.21.15.69:0/661602672 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558ba072b4a0 0x558ba072d8b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.294+0000 7feff1802b80 1 --2- 172.21.15.69:0/661602672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0717390 0x558ba0726e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.294+0000 7feff1802b80 1 --2- 172.21.15.69:0/661602672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0715f20 0x558ba071f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.294+0000 7feff1802b80 1 --2- 172.21.15.69:0/661602672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0716870 0x558ba071fdd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.294+0000 7feff1802b80 1 -- 172.21.15.69:0/661602672 >> 172.21.15.69:0/661602672 conn(0x558ba070b640 msgr2=0x558ba07100b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:31:31.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.294+0000 7fefb6ffd700 0 client.25796 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:31:31.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.294+0000 7feff1802b80 1 -- 172.21.15.69:0/661602672 shutdown_connections 2023-07-06T20:31:31.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.294+0000 7feff1802b80 1 -- 172.21.15.69:0/661602672 wait complete. 2023-07-06T20:31:31.301 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.294+0000 7feff1802b80 20 client.25796 trim_cache size 0 max 16384 2023-07-06T20:31:31.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.322+0000 7feff1802b80 1 Processor -- start 2023-07-06T20:31:31.325 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.323+0000 7feff1802b80 1 -- start start 2023-07-06T20:31:31.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.323+0000 7feff1802b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 0x558ba071a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.323+0000 7feff1802b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 0x558ba071a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.323+0000 7feff1802b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0717390 0x558ba0726dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.323+0000 7feff1802b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba071ed40 con 0x558ba0716870 2023-07-06T20:31:31.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.323+0000 7feff1802b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba072b2c0 con 0x558ba0715f20 2023-07-06T20:31:31.326 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.323+0000 7feff1802b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba071f400 con 0x558ba0717390 2023-07-06T20:31:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.323+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 0x558ba071a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.323+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 0x558ba071a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:34376/0 (socket says 172.21.15.69:34376) 2023-07-06T20:31:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.323+0000 7fefd77fe700 1 -- 172.21.15.69:0/2832070734 learned_addr learned my addr 172.21.15.69:0/2832070734 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:31:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.324+0000 7fefd6ffd700 1 --2- 172.21.15.69:0/2832070734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 0x558ba071a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.324+0000 7fefd7fff700 1 --2- 172.21.15.69:0/2832070734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0717390 0x558ba0726dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.324+0000 7fefd77fe700 1 -- 172.21.15.69:0/2832070734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0717390 msgr2=0x558ba0726dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.324+0000 7fefd77fe700 1 --2- 172.21.15.69:0/2832070734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0717390 0x558ba0726dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.324+0000 7fefd77fe700 1 -- 172.21.15.69:0/2832070734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 msgr2=0x558ba071a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.324+0000 7fefd77fe700 1 --2- 172.21.15.69:0/2832070734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 0x558ba071a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.324+0000 7fefd77fe700 1 -- 172.21.15.69:0/2832070734 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ba04bb250 con 0x558ba0715f20 2023-07-06T20:31:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.324+0000 7fefd77fe700 1 --2- 172.21.15.69:0/2832070734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 0x558ba071a2e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fefc8000c00 tx=0x7fefc8023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1a05f33a91646e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:31:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.324+0000 7fefbe7fc700 1 -- 172.21.15.69:0/2832070734 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8035af0 con 0x558ba0715f20 2023-07-06T20:31:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.325+0000 7fefbe7fc700 1 -- 172.21.15.69:0/2832070734 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefc801e040 con 0x558ba0715f20 2023-07-06T20:31:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.325+0000 7fefbe7fc700 1 -- 172.21.15.69:0/2832070734 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc800f460 con 0x558ba0715f20 2023-07-06T20:31:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.325+0000 7feff1802b80 1 -- 172.21.15.69:0/2832070734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 msgr2=0x558ba071a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.325+0000 7feff1802b80 1 --2- 172.21.15.69:0/2832070734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 0x558ba071a2e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fefc8000c00 tx=0x7fefc8023000 comp rx=0 tx=0).stop 2023-07-06T20:31:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.325+0000 7feff1802b80 1 -- 172.21.15.69:0/2832070734 shutdown_connections 2023-07-06T20:31:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.325+0000 7feff1802b80 1 --2- 172.21.15.69:0/2832070734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0717390 0x558ba0726dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.325+0000 7feff1802b80 1 --2- 172.21.15.69:0/2832070734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 0x558ba071a2e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.325+0000 7feff1802b80 1 --2- 172.21.15.69:0/2832070734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 0x558ba071a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.325+0000 7feff1802b80 1 -- 172.21.15.69:0/2832070734 >> 172.21.15.69:0/2832070734 conn(0x558ba070b640 msgr2=0x558ba0727700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:31:31.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.325+0000 7feff1802b80 1 -- 172.21.15.69:0/2832070734 shutdown_connections 2023-07-06T20:31:31.331 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.326+0000 7feff1802b80 1 -- 172.21.15.69:0/2832070734 wait complete. 2023-07-06T20:31:31.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.326+0000 7feff1802b80 1 -- start start 2023-07-06T20:31:31.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.326+0000 7feff1802b80 1 Processor -- start 2023-07-06T20:31:31.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.327+0000 7feff1802b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 0x558ba07266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.327+0000 7feff1802b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 0x558ba0726c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.327+0000 7feff1802b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0717390 0x558ba0723170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.332 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.327+0000 7feff1802b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba07237c0 con 0x558ba0716870 2023-07-06T20:31:31.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.327+0000 7feff1802b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba0723930 con 0x558ba0715f20 2023-07-06T20:31:31.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.327+0000 7feff1802b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ba0723af0 con 0x558ba0717390 2023-07-06T20:31:31.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.327+0000 7feff1802b80 10 client.0 _ll_register_callbacks cb 0x7fefcc005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:31:31.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.327+0000 7fefd6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 0x558ba0726c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.327+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 0x558ba07266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.327+0000 7fefd77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 0x558ba07266d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:34384/0 (socket says 172.21.15.69:34384) 2023-07-06T20:31:31.333 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.327+0000 7fefd77fe700 1 -- 172.21.15.69:0/1905120846 learned_addr learned my addr 172.21.15.69:0/1905120846 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:31:31.334 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.328+0000 7fefd77fe700 1 -- 172.21.15.69:0/1905120846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0717390 msgr2=0x558ba0723170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:31:31.334 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.328+0000 7fefd77fe700 1 --2- 172.21.15.69:0/1905120846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0717390 0x558ba0723170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.334 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.328+0000 7fefd77fe700 1 -- 172.21.15.69:0/1905120846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 msgr2=0x558ba0726c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:31:31.334 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.328+0000 7fefd77fe700 1 --2- 172.21.15.69:0/1905120846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 0x558ba0726c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.335 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.328+0000 7fefd77fe700 1 -- 172.21.15.69:0/1905120846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd8077d70 con 0x558ba0715f20 2023-07-06T20:31:31.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.330+0000 7fefd77fe700 1 --2- 172.21.15.69:0/1905120846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 0x558ba07266d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fefc8005910 tx=0x7fefc8005980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d99cfc863ca5b88 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:31:31.336 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.330+0000 7fefb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:31:31.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.334+0000 7fefb6ffd700 1 -- 172.21.15.69:0/1905120846 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8004e40 con 0x558ba0715f20 2023-07-06T20:31:31.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.335+0000 7fefb6ffd700 1 -- 172.21.15.69:0/1905120846 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fefc8035bb0 con 0x558ba0715f20 2023-07-06T20:31:31.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.335+0000 7feff1802b80 1 -- 172.21.15.69:0/1905120846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefcc0044f0 con 0x558ba0715f20 2023-07-06T20:31:31.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.335+0000 7fefb6ffd700 1 -- 172.21.15.69:0/1905120846 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefc8035170 con 0x558ba0715f20 2023-07-06T20:31:31.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.335+0000 7feff1802b80 10 client.25802 fetch_fsmap learned FSMap version 30 2023-07-06T20:31:31.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.335+0000 7feff1802b80 1 -- 172.21.15.69:0/1905120846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558ba04932b0 con 0x558ba0715f20 2023-07-06T20:31:31.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.336+0000 7fefb6ffd700 1 -- 172.21.15.69:0/1905120846 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefcc0044f0 con 0x558ba0715f20 2023-07-06T20:31:31.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.336+0000 7fefb6ffd700 1 -- 172.21.15.69:0/1905120846 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fefc803c030 con 0x558ba0715f20 2023-07-06T20:31:31.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.336+0000 7feff1802b80 10 client.25802 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:31:31.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.336+0000 7feff1802b80 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:31:31.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.336+0000 7feff1802b80 20 client.25802 populate_metadata read hostname 'smithi069' 2023-07-06T20:31:31.340 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.336+0000 7feff1802b80 1 --2- 172.21.15.69:0/1905120846 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558ba0780460 0x558ba0782840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:31:31.340 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.336+0000 7feff1802b80 4 client.25802 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:31:31.340 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.336+0000 7feff1802b80 1 -- 172.21.15.69:0/1905120846 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558ba065feb0 con 0x558ba0780460 2023-07-06T20:31:31.340 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.336+0000 7fefd6ffd700 1 --2- 172.21.15.69:0/1905120846 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558ba0780460 0x558ba0782840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:31:31.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.337+0000 7fefd6ffd700 1 --2- 172.21.15.69:0/1905120846 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558ba0780460 0x558ba0782840 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a06de7d79d6afc8b server_cookie=dd8276d04a867d79 in_seq=0 out_seq=0 2023-07-06T20:31:31.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.337+0000 7fefb6ffd700 10 client.25802 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:31:31.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.337+0000 7fefb6ffd700 1 -- 172.21.15.69:0/1905120846 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558ba065feb0 con 0x558ba0780460 2023-07-06T20:31:31.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.337+0000 7fefb6ffd700 10 client.25802 handle_command_reply: tid=0 2023-07-06T20:31:31.341 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.337+0000 7feff1802b80 1 client.25802 shutdown 2023-07-06T20:31:31.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.339+0000 7feff1802b80 1 -- 172.21.15.69:0/1905120846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 msgr2=0x558ba07266d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:31.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.339+0000 7feff1802b80 1 --2- 172.21.15.69:0/1905120846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 0x558ba07266d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fefc8005910 tx=0x7fefc8005980 comp rx=0 tx=0).stop 2023-07-06T20:31:31.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.340+0000 7feff1802b80 1 -- 172.21.15.69:0/1905120846 shutdown_connections 2023-07-06T20:31:31.342 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.340+0000 7feff1802b80 1 --2- 172.21.15.69:0/1905120846 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x558ba0780460 0x558ba0782840 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.340+0000 7feff1802b80 1 --2- 172.21.15.69:0/1905120846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ba0717390 0x558ba0723170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.340+0000 7feff1802b80 1 --2- 172.21.15.69:0/1905120846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ba0715f20 0x558ba07266d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.340+0000 7feff1802b80 1 --2- 172.21.15.69:0/1905120846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ba0716870 0x558ba0726c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.340+0000 7feff1802b80 1 -- 172.21.15.69:0/1905120846 >> 172.21.15.69:0/1905120846 conn(0x558ba070b640 msgr2=0x558ba071a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:31:31.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.340+0000 7fefb6ffd700 0 client.25802 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:31:31.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.340+0000 7feff1802b80 1 -- 172.21.15.69:0/1905120846 shutdown_connections 2023-07-06T20:31:31.343 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.340+0000 7feff1802b80 1 -- 172.21.15.69:0/1905120846 wait complete. 2023-07-06T20:31:31.344 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.340+0000 7feff1802b80 20 client.25802 trim_cache size 0 max 16384 2023-07-06T20:31:31.344 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:31:31.344 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:31:31.344 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:31:31.345 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:31:31.345 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:31:31.345 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:31:31.346 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:31:31.346 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:31:31.346 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:31:31.346 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:31:31.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.343+0000 7fefdddff700 1 -- 172.21.15.69:0/618050882 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fefc00a11d0 msgr2=0x7fefc00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:31.347 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.343+0000 7fefdddff700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fefc00a11d0 0x7fefc00a3680 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fefc8006010 tx=0x7fefc800b260 comp rx=0 tx=0).stop 2023-07-06T20:31:31.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.343+0000 7fefdddff700 1 -- 172.21.15.69:0/618050882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefd8076bf0 msgr2=0x7fefd81a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:31:31.348 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.343+0000 7fefdddff700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefd8076bf0 0x7fefd81a3540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fefd0007f10 tx=0x7fefd000d3b0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.349 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.346+0000 7fefdddff700 1 -- 172.21.15.69:0/618050882 shutdown_connections 2023-07-06T20:31:31.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.346+0000 7fefdddff700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fefc00a11d0 0x7fefc00a3680 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.346+0000 7fefdddff700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefd8076bf0 0x7fefd81a3540 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.350 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.346+0000 7fefdddff700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd8075000 0x7fefd819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.346+0000 7fefdddff700 1 --2- 172.21.15.69:0/618050882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefd8076240 0x7fefd819c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:31:31.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.346+0000 7fefdddff700 1 -- 172.21.15.69:0/618050882 >> 172.21.15.69:0/618050882 conn(0x7fefd80fd480 msgr2=0x7fefd80ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:31:31.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.346+0000 7fefdddff700 1 -- 172.21.15.69:0/618050882 shutdown_connections 2023-07-06T20:31:31.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:31:31.346+0000 7fefdddff700 1 -- 172.21.15.69:0/618050882 wait complete. 2023-07-06T20:31:31.376 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:31:32.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:31 smithi069 ceph-mon[126399]: pgmap v693: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 MiB/s wr, 945 op/s 2023-07-06T20:31:32.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:31 smithi156 ceph-mon[129280]: pgmap v693: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 MiB/s wr, 945 op/s 2023-07-06T20:31:32.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:31 smithi098 ceph-mon[128968]: pgmap v693: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 MiB/s wr, 945 op/s 2023-07-06T20:31:34.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 585.0 s, 2287.3 tps, lat 42.782 ms stddev 29.148 2023-07-06T20:31:34.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:33 smithi069 ceph-mon[126399]: pgmap v694: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 735 op/s 2023-07-06T20:31:34.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:33 smithi156 ceph-mon[129280]: pgmap v694: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 735 op/s 2023-07-06T20:31:34.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:33 smithi098 ceph-mon[128968]: pgmap v694: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 735 op/s 2023-07-06T20:31:36.264 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:35 smithi069 ceph-mon[126399]: pgmap v695: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 27 MiB/s wr, 698 op/s 2023-07-06T20:31:36.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:35 smithi156 ceph-mon[129280]: pgmap v695: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 27 MiB/s wr, 698 op/s 2023-07-06T20:31:36.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:35 smithi098 ceph-mon[128968]: pgmap v695: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 27 MiB/s wr, 698 op/s 2023-07-06T20:31:38.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:38 smithi069 ceph-mon[126399]: pgmap v696: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 30 MiB/s wr, 1.09k op/s 2023-07-06T20:31:38.361 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:38 smithi156 ceph-mon[129280]: pgmap v696: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 30 MiB/s wr, 1.09k op/s 2023-07-06T20:31:38.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:38 smithi098 ceph-mon[128968]: pgmap v696: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 30 MiB/s wr, 1.09k op/s 2023-07-06T20:31:39.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 590.0 s, 443.3 tps, lat 229.789 ms stddev 217.669 2023-07-06T20:31:40.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:40 smithi156 ceph-mon[129280]: pgmap v697: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 30 MiB/s wr, 1.50k op/s 2023-07-06T20:31:40.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:40 smithi098 ceph-mon[128968]: pgmap v697: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 30 MiB/s wr, 1.50k op/s 2023-07-06T20:31:40.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:40 smithi069 ceph-mon[126399]: pgmap v697: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 30 MiB/s wr, 1.50k op/s 2023-07-06T20:31:42.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:42 smithi156 ceph-mon[129280]: pgmap v698: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 36 MiB/s wr, 2.37k op/s 2023-07-06T20:31:42.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:42 smithi098 ceph-mon[128968]: pgmap v698: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 36 MiB/s wr, 2.37k op/s 2023-07-06T20:31:42.515 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:42 smithi069 ceph-mon[126399]: pgmap v698: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 36 MiB/s wr, 2.37k op/s 2023-07-06T20:31:44.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 595.0 s, 2327.5 tps, lat 42.456 ms stddev 30.355 2023-07-06T20:31:44.361 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:44 smithi156 ceph-mon[129280]: pgmap v699: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 39 MiB/s wr, 3.07k op/s 2023-07-06T20:31:44.378 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:44 smithi098 ceph-mon[128968]: pgmap v699: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 39 MiB/s wr, 3.07k op/s 2023-07-06T20:31:44.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:44 smithi069 ceph-mon[126399]: pgmap v699: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 39 MiB/s wr, 3.07k op/s 2023-07-06T20:31:46.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:46 smithi156 ceph-mon[129280]: pgmap v700: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 MiB/s wr, 3.18k op/s 2023-07-06T20:31:46.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:46 smithi098 ceph-mon[128968]: pgmap v700: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 MiB/s wr, 3.18k op/s 2023-07-06T20:31:46.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:46 smithi069 ceph-mon[126399]: pgmap v700: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 MiB/s wr, 3.18k op/s 2023-07-06T20:31:48.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:48 smithi069 ceph-mon[126399]: pgmap v701: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 44 MiB/s wr, 3.27k op/s 2023-07-06T20:31:48.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:48 smithi156 ceph-mon[129280]: pgmap v701: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 44 MiB/s wr, 3.27k op/s 2023-07-06T20:31:48.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:48 smithi098 ceph-mon[128968]: pgmap v701: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 44 MiB/s wr, 3.27k op/s 2023-07-06T20:31:49.233 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 600.0 s, 1781.2 tps, lat 55.740 ms stddev 82.320 2023-07-06T20:31:49.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:49 smithi069 ceph-mon[126399]: pgmap v702: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 MiB/s wr, 2.79k op/s 2023-07-06T20:31:49.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:49 smithi156 ceph-mon[129280]: pgmap v702: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 MiB/s wr, 2.79k op/s 2023-07-06T20:31:49.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:49 smithi098 ceph-mon[128968]: pgmap v702: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 MiB/s wr, 2.79k op/s 2023-07-06T20:31:51.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:51 smithi069 ceph-mon[126399]: pgmap v703: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 MiB/s wr, 2.80k op/s 2023-07-06T20:31:51.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:51 smithi156 ceph-mon[129280]: pgmap v703: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 MiB/s wr, 2.80k op/s 2023-07-06T20:31:51.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:51 smithi098 ceph-mon[128968]: pgmap v703: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 MiB/s wr, 2.80k op/s 2023-07-06T20:31:53.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:53 smithi156 ceph-mon[129280]: pgmap v704: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 2.35k op/s 2023-07-06T20:31:53.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:53 smithi098 ceph-mon[128968]: pgmap v704: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 2.35k op/s 2023-07-06T20:31:54.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:53 smithi069 ceph-mon[126399]: pgmap v704: 97 pgs: 97 active+clean; 8.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 2.35k op/s 2023-07-06T20:31:54.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 605.0 s, 1664.5 tps, lat 61.541 ms stddev 87.961 2023-07-06T20:31:55.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:55 smithi156 ceph-mon[129280]: pgmap v705: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 1.88k op/s 2023-07-06T20:31:55.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:55 smithi098 ceph-mon[128968]: pgmap v705: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 1.88k op/s 2023-07-06T20:31:56.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:55 smithi069 ceph-mon[126399]: pgmap v705: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 1.88k op/s 2023-07-06T20:31:57.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:57 smithi098 ceph-mon[128968]: pgmap v706: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 35 MiB/s wr, 1.86k op/s 2023-07-06T20:31:58.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:57 smithi069 ceph-mon[126399]: pgmap v706: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 35 MiB/s wr, 1.86k op/s 2023-07-06T20:31:58.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:57 smithi156 ceph-mon[129280]: pgmap v706: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 35 MiB/s wr, 1.86k op/s 2023-07-06T20:31:59.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 610.0 s, 2207.2 tps, lat 43.277 ms stddev 28.905 2023-07-06T20:32:00.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:31:59 smithi069 ceph-mon[126399]: pgmap v707: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2023-07-06T20:32:00.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:31:59 smithi156 ceph-mon[129280]: pgmap v707: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2023-07-06T20:32:00.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:31:59 smithi098 ceph-mon[128968]: pgmap v707: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2023-07-06T20:32:01.378 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:32:01.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.886+0000 7f29029ff700 1 -- 172.21.15.69:0/226515135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28fc071010 msgr2=0x7f28fc071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:01.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.886+0000 7f29029ff700 1 --2- 172.21.15.69:0/226515135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28fc071010 0x7f28fc071420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f28f00098f0 tx=0x7f28f0009c00 comp rx=0 tx=0).stop 2023-07-06T20:32:01.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.887+0000 7f29029ff700 1 -- 172.21.15.69:0/226515135 shutdown_connections 2023-07-06T20:32:01.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.887+0000 7f29029ff700 1 --2- 172.21.15.69:0/226515135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28fc10be90 0x7f28fc10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.887+0000 7f29029ff700 1 --2- 172.21.15.69:0/226515135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28fc071010 0x7f28fc071420 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.887+0000 7f29029ff700 1 --2- 172.21.15.69:0/226515135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28fc071960 0x7f28fc10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.887+0000 7f29029ff700 1 -- 172.21.15.69:0/226515135 >> 172.21.15.69:0/226515135 conn(0x7f28fc06cbc0 msgr2=0x7f28fc06eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:32:01.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.888+0000 7f29029ff700 1 -- 172.21.15.69:0/226515135 shutdown_connections 2023-07-06T20:32:01.891 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.888+0000 7f29029ff700 1 -- 172.21.15.69:0/226515135 wait complete. 2023-07-06T20:32:01.892 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.889+0000 7f29029ff700 1 Processor -- start 2023-07-06T20:32:01.892 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.890+0000 7f29029ff700 1 -- start start 2023-07-06T20:32:01.892 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.890+0000 7f29029ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28fc071010 0x7f28fc113c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:01.893 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.890+0000 7f29029ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28fc071960 0x7f28fc1141b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:01.893 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.890+0000 7f29029ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28fc10be90 0x7f28fc1b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:01.893 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.890+0000 7f29029ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc117290 con 0x7f28fc071010 2023-07-06T20:32:01.893 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.890+0000 7f29029ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc117400 con 0x7f28fc071960 2023-07-06T20:32:01.893 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.890+0000 7f29029ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc1b4f00 con 0x7f28fc10be90 2023-07-06T20:32:01.896 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.894+0000 7f28fb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28fc071960 0x7f28fc1141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:01.896 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.894+0000 7f28fb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28fc071960 0x7f28fc1141b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:49802/0 (socket says 172.21.15.69:49802) 2023-07-06T20:32:01.896 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.894+0000 7f28fb7fe700 1 -- 172.21.15.69:0/1735426912 learned_addr learned my addr 172.21.15.69:0/1735426912 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:32:01.900 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.898+0000 7f2900f9c700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28fc10be90 0x7f28fc1b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:01.900 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.898+0000 7f28fbfff700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28fc071010 0x7f28fc113c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.899+0000 7f28fbfff700 1 -- 172.21.15.69:0/1735426912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28fc10be90 msgr2=0x7f28fc1b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.899+0000 7f28fbfff700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28fc10be90 0x7f28fc1b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.899+0000 7f28fbfff700 1 -- 172.21.15.69:0/1735426912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28fc071960 msgr2=0x7f28fc1141b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:01.901 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.899+0000 7f28fbfff700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28fc071960 0x7f28fc1141b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.902 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.899+0000 7f28fbfff700 1 -- 172.21.15.69:0/1735426912 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f0009520 con 0x7f28fc071010 2023-07-06T20:32:01.902 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.899+0000 7f28fbfff700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28fc071010 0x7f28fc113c70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f28f00098f0 tx=0x7f28f000bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:32:01.908 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.900+0000 7f28f97fa700 1 -- 172.21.15.69:0/1735426912 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0005520 con 0x7f28fc071010 2023-07-06T20:32:01.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.900+0000 7f28f97fa700 1 -- 172.21.15.69:0/1735426912 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28f0005680 con 0x7f28fc071010 2023-07-06T20:32:01.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.900+0000 7f28f97fa700 1 -- 172.21.15.69:0/1735426912 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0017980 con 0x7f28fc071010 2023-07-06T20:32:01.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.900+0000 7f29029ff700 1 -- 172.21.15.69:0/1735426912 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28fc1b5180 con 0x7f28fc071010 2023-07-06T20:32:01.909 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.900+0000 7f29029ff700 1 -- 172.21.15.69:0/1735426912 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc1b54f0 con 0x7f28fc071010 2023-07-06T20:32:01.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.906+0000 7f2916347b80 1 Processor -- start 2023-07-06T20:32:01.910 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.907+0000 7f2916347b80 1 -- start start 2023-07-06T20:32:01.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.907+0000 7f2916347b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98fe3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:01.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.907+0000 7f2916347b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 0x55da98fe3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:01.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.907+0000 7f2916347b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe4390 0x55da98fe4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:01.911 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.908+0000 7f2916347b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da98ee7cb0 con 0x55da98fe2f20 2023-07-06T20:32:01.912 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.908+0000 7f2916347b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da98f3aa40 con 0x55da98fe3870 2023-07-06T20:32:01.912 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.908+0000 7f2916347b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da98f33ed0 con 0x55da98fe4390 2023-07-06T20:32:01.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.909+0000 7f28fb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe4390 0x55da98fe4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:01.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.909+0000 7f28fb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe4390 0x55da98fe4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:35306/0 (socket says 172.21.15.69:35306) 2023-07-06T20:32:01.913 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.909+0000 7f28fb7fe700 1 -- 172.21.15.69:0/120511841 learned_addr learned my addr 172.21.15.69:0/120511841 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:32:01.914 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.910+0000 7f28fbfff700 1 --2- 172.21.15.69:0/120511841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 0x55da98fe3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:01.914 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.911+0000 7f2900f9c700 1 --2- 172.21.15.69:0/120511841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98fe3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:01.915 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.911+0000 7f2900f9c700 1 -- 172.21.15.69:0/120511841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe4390 msgr2=0x55da98fe4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:01.915 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.911+0000 7f2900f9c700 1 --2- 172.21.15.69:0/120511841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe4390 0x55da98fe4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.915 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.911+0000 7f2900f9c700 1 -- 172.21.15.69:0/120511841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 msgr2=0x55da98fe3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:01.916 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.911+0000 7f2900f9c700 1 --2- 172.21.15.69:0/120511841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 0x55da98fe3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.916 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.911+0000 7f2900f9c700 1 -- 172.21.15.69:0/120511841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da98d61de0 con 0x55da98fe2f20 2023-07-06T20:32:01.916 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.912+0000 7f2900f9c700 1 --2- 172.21.15.69:0/120511841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98fe3330 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009c90 tx=0x7f28f4007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9496c51f1a763beb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:32:01.916 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.914+0000 7f28e27fc700 1 -- 172.21.15.69:0/120511841 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4018070 con 0x55da98fe2f20 2023-07-06T20:32:01.917 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.914+0000 7f28e27fc700 1 -- 172.21.15.69:0/120511841 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28f400f050 con 0x55da98fe2f20 2023-07-06T20:32:01.917 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.914+0000 7f28e27fc700 1 -- 172.21.15.69:0/120511841 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4005970 con 0x55da98fe2f20 2023-07-06T20:32:01.917 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.915+0000 7f2916347b80 1 -- 172.21.15.69:0/120511841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 msgr2=0x55da98fe3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:01.917 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.915+0000 7f2916347b80 1 --2- 172.21.15.69:0/120511841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98fe3330 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009c90 tx=0x7f28f4007500 comp rx=0 tx=0).stop 2023-07-06T20:32:01.918 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.915+0000 7f28f97fa700 1 -- 172.21.15.69:0/1735426912 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f28f0017ae0 con 0x7f28fc071010 2023-07-06T20:32:01.918 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.915+0000 7f2916347b80 1 -- 172.21.15.69:0/120511841 shutdown_connections 2023-07-06T20:32:01.918 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.916+0000 7f2916347b80 1 --2- 172.21.15.69:0/120511841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe4390 0x55da98fe4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.918 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.916+0000 7f2916347b80 1 --2- 172.21.15.69:0/120511841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98fe3330 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.918 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.916+0000 7f2916347b80 1 --2- 172.21.15.69:0/120511841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 0x55da98fe3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.919 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.916+0000 7f2916347b80 1 -- 172.21.15.69:0/120511841 >> 172.21.15.69:0/120511841 conn(0x55da98fd8640 msgr2=0x55da98fdd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:32:01.929 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.925+0000 7f28f97fa700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f28e40a1260 0x7f28e40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:01.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.925+0000 7f28f97fa700 1 -- 172.21.15.69:0/1735426912 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f28f00d3280 con 0x7f28fc071010 2023-07-06T20:32:01.930 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.927+0000 7f2916347b80 1 -- 172.21.15.69:0/120511841 shutdown_connections 2023-07-06T20:32:01.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.927+0000 7f2916347b80 1 -- 172.21.15.69:0/120511841 wait complete. 2023-07-06T20:32:01.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.928+0000 7f2916347b80 1 -- start start 2023-07-06T20:32:01.931 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.928+0000 7f2916347b80 1 Processor -- start 2023-07-06T20:32:01.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.929+0000 7f2916347b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98fec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:01.932 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.929+0000 7f2916347b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 0x55da98fece20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.929+0000 7f2916347b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe4390 0x55da98ff3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.929+0000 7f2916347b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da98ed3ac0 con 0x55da98fe2f20 2023-07-06T20:32:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.929+0000 7f2916347b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da98ed37c0 con 0x55da98fe3870 2023-07-06T20:32:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.929+0000 7f2916347b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da98db7920 con 0x55da98fe4390 2023-07-06T20:32:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.929+0000 7f2916347b80 10 client.0 _ll_register_callbacks cb 0x55da98ed1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:32:01.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.940+0000 7f28fb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 0x55da98fece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:01.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.940+0000 7f28fb7fe700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f28e40a1260 0x7f28e40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:01.943 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.940+0000 7f2900f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe4390 0x55da98ff3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:01.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.941+0000 7f28fbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98fec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:01.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.941+0000 7f28fbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98fec8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35762/0 (socket says 172.21.15.69:35762) 2023-07-06T20:32:01.944 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.941+0000 7f28fbfff700 1 -- 172.21.15.69:0/1913554817 learned_addr learned my addr 172.21.15.69:0/1913554817 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:32:01.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.953+0000 7f28fbfff700 1 -- 172.21.15.69:0/1913554817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe4390 msgr2=0x55da98ff3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:01.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.953+0000 7f28fbfff700 1 --2- 172.21.15.69:0/1913554817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe4390 0x55da98ff3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.953+0000 7f28fbfff700 1 -- 172.21.15.69:0/1913554817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 msgr2=0x55da98fece20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:01.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.953+0000 7f28fbfff700 1 --2- 172.21.15.69:0/1913554817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 0x55da98fece20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.953+0000 7f28fbfff700 1 -- 172.21.15.69:0/1913554817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f40098c0 con 0x55da98fe2f20 2023-07-06T20:32:01.965 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.962+0000 7f28fbfff700 1 --2- 172.21.15.69:0/1913554817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98fec8e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009810 tx=0x7f28f009ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f55b287e93d058 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:32:01.965 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.962+0000 7f28db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:32:01.965 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.963+0000 7f28fb7fe700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f28e40a1260 0x7f28e40a3710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f28ec005b40 tx=0x7f28ec009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:32:01.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.963+0000 7f28db7fe700 1 -- 172.21.15.69:0/1913554817 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f009ff60 con 0x55da98fe2f20 2023-07-06T20:32:01.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.963+0000 7f28db7fe700 1 -- 172.21.15.69:0/1913554817 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f28f00c00b0 con 0x55da98fe2f20 2023-07-06T20:32:01.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.964+0000 7f2916347b80 1 -- 172.21.15.69:0/1913554817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da98e8d4f0 con 0x55da98fe2f20 2023-07-06T20:32:01.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.964+0000 7f28db7fe700 1 -- 172.21.15.69:0/1913554817 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28f00a0600 con 0x55da98fe2f20 2023-07-06T20:32:01.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.965+0000 7f2916347b80 10 client.15648 fetch_fsmap learned FSMap version 30 2023-07-06T20:32:01.967 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.965+0000 7f2916347b80 1 -- 172.21.15.69:0/1913554817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55da98d4ac60 con 0x55da98fe2f20 2023-07-06T20:32:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.966+0000 7f28db7fe700 1 -- 172.21.15.69:0/1913554817 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55da98e8d4f0 con 0x55da98fe2f20 2023-07-06T20:32:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.966+0000 7f28db7fe700 1 -- 172.21.15.69:0/1913554817 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f28f009faf0 con 0x55da98fe2f20 2023-07-06T20:32:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.966+0000 7f2916347b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:32:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.966+0000 7f2916347b80 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:32:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.966+0000 7f2916347b80 20 client.15648 populate_metadata read hostname 'smithi069' 2023-07-06T20:32:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.966+0000 7f2916347b80 1 --2- 172.21.15.69:0/1913554817 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55da98ff8350 0x55da98ffa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.966+0000 7f2916347b80 4 client.15648 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:32:01.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.966+0000 7f2916347b80 1 -- 172.21.15.69:0/1913554817 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da98fb1d60 con 0x55da98ff8350 2023-07-06T20:32:01.970 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.967+0000 7f28fb7fe700 1 --2- 172.21.15.69:0/1913554817 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55da98ff8350 0x55da98ffa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:01.980 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.978+0000 7f28fb7fe700 1 --2- 172.21.15.69:0/1913554817 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55da98ff8350 0x55da98ffa760 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dba9fdd17c0eead server_cookie=956c61bb275955de in_seq=0 out_seq=0 2023-07-06T20:32:01.980 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.978+0000 7f28db7fe700 10 client.15648 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:32:01.981 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.979+0000 7f28db7fe700 1 -- 172.21.15.69:0/1913554817 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55da98fb1d60 con 0x55da98ff8350 2023-07-06T20:32:01.982 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.979+0000 7f28db7fe700 10 client.15648 handle_command_reply: tid=0 2023-07-06T20:32:01.982 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.979+0000 7f2916347b80 1 client.15648 shutdown 2023-07-06T20:32:01.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.988+0000 7f2916347b80 1 -- 172.21.15.69:0/1913554817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 msgr2=0x55da98fec8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:01.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.988+0000 7f2916347b80 1 --2- 172.21.15.69:0/1913554817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98fec8e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009810 tx=0x7f28f009ca70 comp rx=0 tx=0).stop 2023-07-06T20:32:01.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.988+0000 7f2916347b80 1 -- 172.21.15.69:0/1913554817 shutdown_connections 2023-07-06T20:32:01.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.988+0000 7f2916347b80 1 --2- 172.21.15.69:0/1913554817 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55da98ff8350 0x55da98ffa760 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.988+0000 7f2916347b80 1 --2- 172.21.15.69:0/1913554817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe4390 0x55da98ff3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.988+0000 7f2916347b80 1 --2- 172.21.15.69:0/1913554817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98fec8e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.988+0000 7f2916347b80 1 --2- 172.21.15.69:0/1913554817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 0x55da98fece20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:01.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.988+0000 7f2916347b80 1 -- 172.21.15.69:0/1913554817 >> 172.21.15.69:0/1913554817 conn(0x55da98fd8640 msgr2=0x55da98fdd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:32:01.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.989+0000 7f28db7fe700 0 client.15648 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:32:01.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.989+0000 7f2916347b80 1 -- 172.21.15.69:0/1913554817 shutdown_connections 2023-07-06T20:32:01.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.989+0000 7f2916347b80 1 -- 172.21.15.69:0/1913554817 wait complete. 2023-07-06T20:32:01.993 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:01.989+0000 7f2916347b80 20 client.15648 trim_cache size 0 max 16384 2023-07-06T20:32:02.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:01 smithi069 ceph-mon[126399]: pgmap v708: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 MiB/s wr, 1.97k op/s 2023-07-06T20:32:02.024 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.021+0000 7f2916347b80 1 Processor -- start 2023-07-06T20:32:02.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.022+0000 7f2916347b80 1 -- start start 2023-07-06T20:32:02.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.022+0000 7f2916347b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fd8640 0x55da98fdd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:02.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.022+0000 7f2916347b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98ff3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:02.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.022+0000 7f2916347b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 0x55da98ffa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:02.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.022+0000 7f2916347b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da98fd8450 con 0x55da98fe2f20 2023-07-06T20:32:02.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.022+0000 7f2916347b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da98febd90 con 0x55da98fe3870 2023-07-06T20:32:02.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.022+0000 7f2916347b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da98fec450 con 0x55da98fd8640 2023-07-06T20:32:02.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.023+0000 7f28fbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fd8640 0x55da98fdd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:02.036 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.024+0000 7f28fbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fd8640 0x55da98fdd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:35322/0 (socket says 172.21.15.69:35322) 2023-07-06T20:32:02.036 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.024+0000 7f28fbfff700 1 -- 172.21.15.69:0/2214265469 learned_addr learned my addr 172.21.15.69:0/2214265469 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:32:02.036 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.031+0000 7f2900f9c700 1 --2- 172.21.15.69:0/2214265469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 0x55da98ffa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:02.036 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.033+0000 7f28fbfff700 1 -- 172.21.15.69:0/2214265469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 msgr2=0x55da98ffa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:02.036 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.033+0000 7f28fbfff700 1 --2- 172.21.15.69:0/2214265469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 0x55da98ffa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.037 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.033+0000 7f28fbfff700 1 -- 172.21.15.69:0/2214265469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 msgr2=0x55da98ff3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:02.037 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.033+0000 7f28fbfff700 1 --2- 172.21.15.69:0/2214265469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98ff3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.037 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.033+0000 7f28fbfff700 1 -- 172.21.15.69:0/2214265469 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da98fe4c80 con 0x55da98fd8640 2023-07-06T20:32:02.037 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.035+0000 7f28fbfff700 1 --2- 172.21.15.69:0/2214265469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fd8640 0x55da98fdd0b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f28f009cb00 tx=0x7f28f009ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7479dcc8075fb64 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:32:02.040 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.038+0000 7f28e2ffd700 1 -- 172.21.15.69:0/2214265469 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f009ff60 con 0x55da98fd8640 2023-07-06T20:32:02.040 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.038+0000 7f28e2ffd700 1 -- 172.21.15.69:0/2214265469 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28f00c00b0 con 0x55da98fd8640 2023-07-06T20:32:02.040 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.038+0000 7f28e2ffd700 1 -- 172.21.15.69:0/2214265469 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f00d0d70 con 0x55da98fd8640 2023-07-06T20:32:02.041 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.039+0000 7f2916347b80 1 -- 172.21.15.69:0/2214265469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fd8640 msgr2=0x55da98fdd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:02.041 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.039+0000 7f2916347b80 1 --2- 172.21.15.69:0/2214265469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fd8640 0x55da98fdd0b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f28f009cb00 tx=0x7f28f009ebb0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.043+0000 7f2916347b80 1 -- 172.21.15.69:0/2214265469 shutdown_connections 2023-07-06T20:32:02.045 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.043+0000 7f2916347b80 1 --2- 172.21.15.69:0/2214265469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe3870 0x55da98ffa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.043+0000 7f2916347b80 1 --2- 172.21.15.69:0/2214265469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fd8640 0x55da98fdd0b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.043+0000 7f2916347b80 1 --2- 172.21.15.69:0/2214265469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fe2f20 0x55da98ff3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.043+0000 7f2916347b80 1 -- 172.21.15.69:0/2214265469 >> 172.21.15.69:0/2214265469 conn(0x55da98feb990 msgr2=0x55da98fff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:32:02.046 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.044+0000 7f2916347b80 1 -- 172.21.15.69:0/2214265469 shutdown_connections 2023-07-06T20:32:02.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.044+0000 7f2916347b80 1 -- 172.21.15.69:0/2214265469 wait complete. 2023-07-06T20:32:02.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.044+0000 7f2916347b80 1 -- start start 2023-07-06T20:32:02.047 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.045+0000 7f2916347b80 1 Processor -- start 2023-07-06T20:32:02.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.045+0000 7f2916347b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fd8640 0x55da98ff3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:02.048 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.045+0000 7f2916347b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe2f20 0x55da98ff3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:02.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.045+0000 7f2916347b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe3870 0x55da98ff0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:02.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.045+0000 7f2916347b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da98ff0570 con 0x55da98fd8640 2023-07-06T20:32:02.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.045+0000 7f2916347b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da98ff06e0 con 0x55da98fe2f20 2023-07-06T20:32:02.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.045+0000 7f2916347b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da98ff08a0 con 0x55da98fe3870 2023-07-06T20:32:02.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.046+0000 7f2916347b80 10 client.0 _ll_register_callbacks cb 0x55da98ed1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:32:02.049 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.046+0000 7f28fb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe2f20 0x55da98ff3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:02.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.046+0000 7f2900f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe3870 0x55da98ff0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:02.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.046+0000 7f2900f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe3870 0x55da98ff0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:35326/0 (socket says 172.21.15.69:35326) 2023-07-06T20:32:02.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.047+0000 7f2900f9c700 1 -- 172.21.15.69:0/767832568 learned_addr learned my addr 172.21.15.69:0/767832568 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:32:02.050 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.048+0000 7f28fbfff700 1 --2- 172.21.15.69:0/767832568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fd8640 0x55da98ff3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:02.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.048+0000 7f28fb7fe700 1 -- 172.21.15.69:0/767832568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe3870 msgr2=0x55da98ff0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:02.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.048+0000 7f28fb7fe700 1 --2- 172.21.15.69:0/767832568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe3870 0x55da98ff0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.048+0000 7f28fb7fe700 1 -- 172.21.15.69:0/767832568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fd8640 msgr2=0x55da98ff3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:02.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.048+0000 7f28fb7fe700 1 --2- 172.21.15.69:0/767832568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fd8640 0x55da98ff3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.051 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.048+0000 7f28fb7fe700 1 -- 172.21.15.69:0/767832568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da98d4ac60 con 0x55da98fe2f20 2023-07-06T20:32:02.052 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.049+0000 7f28fb7fe700 1 --2- 172.21.15.69:0/767832568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe2f20 0x55da98ff3ad0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55da98feda50 tx=0x7f28ec0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3a65bf30f26b2b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:32:02.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.052+0000 7f28db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:32:02.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.052+0000 7f28db7fe700 1 -- 172.21.15.69:0/767832568 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec023040 con 0x55da98fe2f20 2023-07-06T20:32:02.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.052+0000 7f28db7fe700 1 -- 172.21.15.69:0/767832568 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f28ec028da0 con 0x55da98fe2f20 2023-07-06T20:32:02.054 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.052+0000 7f2916347b80 1 -- 172.21.15.69:0/767832568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28f400f050 con 0x55da98fe2f20 2023-07-06T20:32:02.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.059+0000 7f28db7fe700 1 -- 172.21.15.69:0/767832568 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28ec0283b0 con 0x55da98fe2f20 2023-07-06T20:32:02.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.059+0000 7f2916347b80 10 client.25817 fetch_fsmap learned FSMap version 30 2023-07-06T20:32:02.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.059+0000 7f2916347b80 1 -- 172.21.15.69:0/767832568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55da98d88250 con 0x55da98fe2f20 2023-07-06T20:32:02.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.077+0000 7f28db7fe700 1 -- 172.21.15.69:0/767832568 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28f400f050 con 0x55da98fe2f20 2023-07-06T20:32:02.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.077+0000 7f28db7fe700 1 -- 172.21.15.69:0/767832568 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f28ec031470 con 0x55da98fe2f20 2023-07-06T20:32:02.079 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.077+0000 7f2916347b80 10 client.25817 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:32:02.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.077+0000 7f2916347b80 10 client.25817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:32:02.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.077+0000 7f2916347b80 20 client.25817 populate_metadata read hostname 'smithi069' 2023-07-06T20:32:02.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.077+0000 7f2916347b80 1 --2- 172.21.15.69:0/767832568 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55da98fe4390 0x55da9904f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:02.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.077+0000 7f2916347b80 4 client.25817 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:32:02.080 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.077+0000 7f2916347b80 1 -- 172.21.15.69:0/767832568 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55da98f2ceb0 con 0x55da98fe4390 2023-07-06T20:32:02.084 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.082+0000 7f28fbfff700 1 --2- 172.21.15.69:0/767832568 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55da98fe4390 0x55da9904f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:02.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.083+0000 7f28fbfff700 1 --2- 172.21.15.69:0/767832568 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55da98fe4390 0x55da9904f7c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d435b46e52de1d9b server_cookie=faa00c7d58aba7fe in_seq=0 out_seq=0 2023-07-06T20:32:02.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.084+0000 7f28db7fe700 10 client.25817 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:32:02.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.093+0000 7f28db7fe700 1 -- 172.21.15.69:0/767832568 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55da98f2ceb0 con 0x55da98fe4390 2023-07-06T20:32:02.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.093+0000 7f28db7fe700 10 client.25817 handle_command_reply: tid=0 2023-07-06T20:32:02.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.093+0000 7f2916347b80 1 client.25817 shutdown 2023-07-06T20:32:02.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.093+0000 7f2916347b80 1 -- 172.21.15.69:0/767832568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe2f20 msgr2=0x55da98ff3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:02.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.093+0000 7f2916347b80 1 --2- 172.21.15.69:0/767832568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe2f20 0x55da98ff3ad0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55da98feda50 tx=0x7f28ec0047c0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.094+0000 7f2916347b80 1 -- 172.21.15.69:0/767832568 shutdown_connections 2023-07-06T20:32:02.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.094+0000 7f2916347b80 1 --2- 172.21.15.69:0/767832568 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55da98fe4390 0x55da9904f7c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.094+0000 7f2916347b80 1 --2- 172.21.15.69:0/767832568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da98fe3870 0x55da98ff0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.094+0000 7f2916347b80 1 --2- 172.21.15.69:0/767832568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55da98fd8640 0x55da98ff3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.094+0000 7f2916347b80 1 --2- 172.21.15.69:0/767832568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55da98fe2f20 0x55da98ff3ad0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.094+0000 7f2916347b80 1 -- 172.21.15.69:0/767832568 >> 172.21.15.69:0/767832568 conn(0x55da98feb990 msgr2=0x55da98fdd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:32:02.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.094+0000 7f28db7fe700 0 client.25817 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:32:02.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.094+0000 7f2916347b80 1 -- 172.21.15.69:0/767832568 shutdown_connections 2023-07-06T20:32:02.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.094+0000 7f2916347b80 1 -- 172.21.15.69:0/767832568 wait complete. 2023-07-06T20:32:02.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.094+0000 7f2916347b80 20 client.25817 trim_cache size 0 max 16384 2023-07-06T20:32:02.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:01 smithi156 ceph-mon[129280]: pgmap v708: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 MiB/s wr, 1.97k op/s 2023-07-06T20:32:02.109 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:32:02.109 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:32:02.109 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:32:02.109 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:32:02.109 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:32:02.110 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:32:02.110 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:32:02.110 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:32:02.110 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:32:02.110 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:32:02.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:01 smithi098 ceph-mon[128968]: pgmap v708: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 MiB/s wr, 1.97k op/s 2023-07-06T20:32:02.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.124+0000 7f29029ff700 1 -- 172.21.15.69:0/1735426912 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f28e40a1260 msgr2=0x7f28e40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:02.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.124+0000 7f29029ff700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f28e40a1260 0x7f28e40a3710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f28ec005b40 tx=0x7f28ec009580 comp rx=0 tx=0).stop 2023-07-06T20:32:02.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.124+0000 7f29029ff700 1 -- 172.21.15.69:0/1735426912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28fc071010 msgr2=0x7f28fc113c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:02.127 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.124+0000 7f29029ff700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28fc071010 0x7f28fc113c70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f28f00098f0 tx=0x7f28f000bb00 comp rx=0 tx=0).stop 2023-07-06T20:32:02.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.131+0000 7f29029ff700 1 -- 172.21.15.69:0/1735426912 shutdown_connections 2023-07-06T20:32:02.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.131+0000 7f29029ff700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f28e40a1260 0x7f28e40a3710 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.133 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.131+0000 7f29029ff700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28fc10be90 0x7f28fc1b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.134 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.131+0000 7f29029ff700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28fc071010 0x7f28fc113c70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.134 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.131+0000 7f29029ff700 1 --2- 172.21.15.69:0/1735426912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28fc071960 0x7f28fc1141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:02.134 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.131+0000 7f29029ff700 1 -- 172.21.15.69:0/1735426912 >> 172.21.15.69:0/1735426912 conn(0x7f28fc06cbc0 msgr2=0x7f28fc06ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:32:02.142 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.140+0000 7f29029ff700 1 -- 172.21.15.69:0/1735426912 shutdown_connections 2023-07-06T20:32:02.142 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:02.140+0000 7f29029ff700 1 -- 172.21.15.69:0/1735426912 wait complete. 2023-07-06T20:32:02.172 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:32:04.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:03 smithi069 ceph-mon[126399]: pgmap v709: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 MiB/s wr, 1.82k op/s 2023-07-06T20:32:04.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:03 smithi156 ceph-mon[129280]: pgmap v709: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 MiB/s wr, 1.82k op/s 2023-07-06T20:32:04.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:03 smithi098 ceph-mon[128968]: pgmap v709: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 MiB/s wr, 1.82k op/s 2023-07-06T20:32:04.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 615.0 s, 920.6 tps, lat 113.453 ms stddev 148.997 2023-07-06T20:32:06.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:05 smithi069 ceph-mon[126399]: pgmap v710: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 1.78k op/s 2023-07-06T20:32:06.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:05 smithi156 ceph-mon[129280]: pgmap v710: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 1.78k op/s 2023-07-06T20:32:06.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:05 smithi098 ceph-mon[128968]: pgmap v710: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 1.78k op/s 2023-07-06T20:32:08.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:07 smithi069 ceph-mon[126399]: pgmap v711: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 2.57k op/s 2023-07-06T20:32:08.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:07 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:32:08.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:07 smithi156 ceph-mon[129280]: pgmap v711: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 2.57k op/s 2023-07-06T20:32:08.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:07 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:32:08.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:07 smithi098 ceph-mon[128968]: pgmap v711: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 38 MiB/s wr, 2.57k op/s 2023-07-06T20:32:08.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:07 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:32:09.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 620.0 s, 2252.0 tps, lat 44.419 ms stddev 31.611 2023-07-06T20:32:09.265 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.265 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.266 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.266 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.266 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.266 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:08 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:08 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:09.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:08 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:10.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:10 smithi069 ceph-mon[126399]: pgmap v712: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 MiB/s wr, 2.41k op/s 2023-07-06T20:32:10.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:32:10.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:32:10.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:10 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:10.614 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:10 smithi156 ceph-mon[129280]: pgmap v712: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 MiB/s wr, 2.41k op/s 2023-07-06T20:32:10.614 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:10 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:32:10.614 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:10 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:32:10.614 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:10 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:10.629 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:10 smithi098 ceph-mon[128968]: pgmap v712: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 MiB/s wr, 2.41k op/s 2023-07-06T20:32:10.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:10 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:32:10.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:10 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:32:10.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:10 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:32:12.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:12 smithi069 ceph-mon[126399]: pgmap v713: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 2.49k op/s 2023-07-06T20:32:12.613 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:12 smithi156 ceph-mon[129280]: pgmap v713: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 2.49k op/s 2023-07-06T20:32:12.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:12 smithi098 ceph-mon[128968]: pgmap v713: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 2.49k op/s 2023-07-06T20:32:13.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:13 smithi098 ceph-mon[128968]: pgmap v714: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 2.30k op/s 2023-07-06T20:32:13.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:13 smithi069 ceph-mon[126399]: pgmap v714: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 2.30k op/s 2023-07-06T20:32:13.859 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:13 smithi156 ceph-mon[129280]: pgmap v714: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 2.30k op/s 2023-07-06T20:32:14.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 625.0 s, 1792.5 tps, lat 54.844 ms stddev 80.249 2023-07-06T20:32:15.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:15 smithi156 ceph-mon[129280]: pgmap v715: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 MiB/s wr, 1.93k op/s 2023-07-06T20:32:15.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:15 smithi098 ceph-mon[128968]: pgmap v715: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 MiB/s wr, 1.93k op/s 2023-07-06T20:32:16.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:15 smithi069 ceph-mon[126399]: pgmap v715: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 MiB/s wr, 1.93k op/s 2023-07-06T20:32:17.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:17 smithi156 ceph-mon[129280]: pgmap v716: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 1.78k op/s 2023-07-06T20:32:17.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:17 smithi098 ceph-mon[128968]: pgmap v716: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 1.78k op/s 2023-07-06T20:32:18.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:17 smithi069 ceph-mon[126399]: pgmap v716: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 1.78k op/s 2023-07-06T20:32:19.232 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 630.0 s, 2111.3 tps, lat 47.870 ms stddev 37.446 2023-07-06T20:32:20.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:19 smithi069 ceph-mon[126399]: pgmap v717: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 684 op/s 2023-07-06T20:32:20.119 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:19 smithi156 ceph-mon[129280]: pgmap v717: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 684 op/s 2023-07-06T20:32:20.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:19 smithi098 ceph-mon[128968]: pgmap v717: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 684 op/s 2023-07-06T20:32:22.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:21 smithi069 ceph-mon[126399]: pgmap v718: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 MiB/s wr, 890 op/s 2023-07-06T20:32:22.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:21 smithi156 ceph-mon[129280]: pgmap v718: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 MiB/s wr, 890 op/s 2023-07-06T20:32:22.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:21 smithi098 ceph-mon[128968]: pgmap v718: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 MiB/s wr, 890 op/s 2023-07-06T20:32:24.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:23 smithi069 ceph-mon[126399]: pgmap v719: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 MiB/s wr, 1.51k op/s 2023-07-06T20:32:24.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:23 smithi156 ceph-mon[129280]: pgmap v719: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 MiB/s wr, 1.51k op/s 2023-07-06T20:32:24.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:23 smithi098 ceph-mon[128968]: pgmap v719: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 MiB/s wr, 1.51k op/s 2023-07-06T20:32:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 635.0 s, 1424.5 tps, lat 68.703 ms stddev 164.410 2023-07-06T20:32:26.113 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:25 smithi156 ceph-mon[129280]: pgmap v720: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2023-07-06T20:32:26.130 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:25 smithi098 ceph-mon[128968]: pgmap v720: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2023-07-06T20:32:26.269 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:25 smithi069 ceph-mon[126399]: pgmap v720: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2023-07-06T20:32:28.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:27 smithi156 ceph-mon[129280]: pgmap v721: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 38 MiB/s wr, 2.48k op/s 2023-07-06T20:32:28.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:27 smithi098 ceph-mon[128968]: pgmap v721: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 38 MiB/s wr, 2.48k op/s 2023-07-06T20:32:28.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:27 smithi069 ceph-mon[126399]: pgmap v721: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 38 MiB/s wr, 2.48k op/s 2023-07-06T20:32:29.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 640.0 s, 806.9 tps, lat 126.561 ms stddev 161.305 2023-07-06T20:32:30.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:29 smithi156 ceph-mon[129280]: pgmap v722: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 2.41k op/s 2023-07-06T20:32:30.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:29 smithi098 ceph-mon[128968]: pgmap v722: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 2.41k op/s 2023-07-06T20:32:30.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:29 smithi069 ceph-mon[126399]: pgmap v722: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 34 MiB/s wr, 2.41k op/s 2023-07-06T20:32:32.173 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:32:32.268 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:31 smithi069 ceph-mon[126399]: pgmap v723: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 2.82k op/s 2023-07-06T20:32:32.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:31 smithi156 ceph-mon[129280]: pgmap v723: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 2.82k op/s 2023-07-06T20:32:32.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:31 smithi098 ceph-mon[128968]: pgmap v723: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 2.82k op/s 2023-07-06T20:32:32.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.544+0000 7fbd609ff700 1 -- 172.21.15.69:0/3673046463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd5c100de0 msgr2=0x7fbd5c103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:32.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.544+0000 7fbd609ff700 1 --2- 172.21.15.69:0/3673046463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd5c100de0 0x7fbd5c103260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009990 tx=0x7fbd4c009ca0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.544+0000 7fbd609ff700 1 -- 172.21.15.69:0/3673046463 shutdown_connections 2023-07-06T20:32:32.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.544+0000 7fbd609ff700 1 --2- 172.21.15.69:0/3673046463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c1037a0 0x7fbd5c103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.547 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.544+0000 7fbd609ff700 1 --2- 172.21.15.69:0/3673046463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c0fe480 0x7fbd5c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.544+0000 7fbd609ff700 1 --2- 172.21.15.69:0/3673046463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd5c100de0 0x7fbd5c103260 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.544+0000 7fbd609ff700 1 -- 172.21.15.69:0/3673046463 >> 172.21.15.69:0/3673046463 conn(0x7fbd5c0fa130 msgr2=0x7fbd5c0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:32:32.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.544+0000 7fbd609ff700 1 -- 172.21.15.69:0/3673046463 shutdown_connections 2023-07-06T20:32:32.548 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.544+0000 7fbd609ff700 1 -- 172.21.15.69:0/3673046463 wait complete. 2023-07-06T20:32:32.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.545+0000 7fbd609ff700 1 Processor -- start 2023-07-06T20:32:32.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.546+0000 7fbd609ff700 1 -- start start 2023-07-06T20:32:32.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.546+0000 7fbd609ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd5c0fe480 0x7fbd5c19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.546+0000 7fbd609ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c100de0 0x7fbd5c19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.546+0000 7fbd609ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c1037a0 0x7fbd5c1a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.546+0000 7fbd609ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c1a3b10 con 0x7fbd5c1037a0 2023-07-06T20:32:32.549 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.546+0000 7fbd609ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c1a3c50 con 0x7fbd5c0fe480 2023-07-06T20:32:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.546+0000 7fbd609ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c1a3de0 con 0x7fbd5c100de0 2023-07-06T20:32:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.547+0000 7fbd5a59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd5c0fe480 0x7fbd5c19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.547+0000 7fbd5a59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd5c0fe480 0x7fbd5c19bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:50800/0 (socket says 172.21.15.69:50800) 2023-07-06T20:32:32.550 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.547+0000 7fbd5a59c700 1 -- 172.21.15.69:0/1671520439 learned_addr learned my addr 172.21.15.69:0/1671520439 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:32:32.551 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.548+0000 7fbd59d9b700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c100de0 0x7fbd5c19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.552 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.549+0000 7fbd5a59c700 1 -- 172.21.15.69:0/1671520439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c100de0 msgr2=0x7fbd5c19c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:32.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.549+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c100de0 0x7fbd5c19c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.549+0000 7fbd5a59c700 1 -- 172.21.15.69:0/1671520439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c1037a0 msgr2=0x7fbd5c1a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:32.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.549+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c1037a0 0x7fbd5c1a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.553 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.549+0000 7fbd5a59c700 1 -- 172.21.15.69:0/1671520439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd54009c40 con 0x7fbd5c0fe480 2023-07-06T20:32:32.554 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.551+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd5c0fe480 0x7fbd5c19bf90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000d950 tx=0x7fbd5000dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:32:32.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.552+0000 7fbd4b7fe700 1 -- 172.21.15.69:0/1671520439 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50009b40 con 0x7fbd5c0fe480 2023-07-06T20:32:32.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.552+0000 7fbd4b7fe700 1 -- 172.21.15.69:0/1671520439 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd50009ca0 con 0x7fbd5c0fe480 2023-07-06T20:32:32.555 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.552+0000 7fbd4b7fe700 1 -- 172.21.15.69:0/1671520439 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5000f620 con 0x7fbd5c0fe480 2023-07-06T20:32:32.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.553+0000 7fbd609ff700 1 -- 172.21.15.69:0/1671520439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4c0095f0 con 0x7fbd5c0fe480 2023-07-06T20:32:32.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.553+0000 7fbd609ff700 1 -- 172.21.15.69:0/1671520439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c1a43f0 con 0x7fbd5c0fe480 2023-07-06T20:32:32.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.560+0000 7fbd743dcb80 1 Processor -- start 2023-07-06T20:32:32.563 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.560+0000 7fbd4b7fe700 1 -- 172.21.15.69:0/1671520439 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbd50010460 con 0x7fbd5c0fe480 2023-07-06T20:32:32.563 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.560+0000 7fbd743dcb80 1 -- start start 2023-07-06T20:32:32.563 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.561+0000 7fbd743dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6af20 0x559421d6b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.561+0000 7fbd743dcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6b870 0x559421d6be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.561+0000 7fbd743dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6c390 0x559421d6c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.561+0000 7fbd743dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559421c6fcb0 con 0x559421d6af20 2023-07-06T20:32:32.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.561+0000 7fbd743dcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559421cc2a40 con 0x559421d6b870 2023-07-06T20:32:32.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.561+0000 7fbd743dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559421cbbed0 con 0x559421d6c390 2023-07-06T20:32:32.565 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.561+0000 7fbd5a59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6b870 0x559421d6be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.566 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.561+0000 7fbd5a59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6b870 0x559421d6be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:50806/0 (socket says 172.21.15.69:50806) 2023-07-06T20:32:32.566 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.561+0000 7fbd5a59c700 1 -- 172.21.15.69:0/1265493914 learned_addr learned my addr 172.21.15.69:0/1265493914 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:32:32.566 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.561+0000 7fbd5ad9d700 1 --2- 172.21.15.69:0/1265493914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6af20 0x559421d6b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.566 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.561+0000 7fbd59d9b700 1 --2- 172.21.15.69:0/1265493914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6c390 0x559421d6c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.566 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.562+0000 7fbd5a59c700 1 -- 172.21.15.69:0/1265493914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6c390 msgr2=0x559421d6c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:32.566 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.562+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/1265493914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6c390 0x559421d6c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.567 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.562+0000 7fbd5a59c700 1 -- 172.21.15.69:0/1265493914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6af20 msgr2=0x559421d6b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:32.567 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.562+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/1265493914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6af20 0x559421d6b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.567 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.562+0000 7fbd5a59c700 1 -- 172.21.15.69:0/1265493914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559421ae9de0 con 0x559421d6b870 2023-07-06T20:32:32.567 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.562+0000 7fbd5ad9d700 1 --2- 172.21.15.69:0/1265493914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6af20 0x559421d6b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:32:32.567 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.562+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/1265493914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6b870 0x559421d6be00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd500d2640 tx=0x7fbd500d2950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2091af30f6c31a39 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:32:32.568 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.563+0000 7fbd48ff9700 1 -- 172.21.15.69:0/1265493914 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd500ccdf0 con 0x559421d6b870 2023-07-06T20:32:32.568 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.563+0000 7fbd48ff9700 1 -- 172.21.15.69:0/1265493914 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd500ce590 con 0x559421d6b870 2023-07-06T20:32:32.568 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.563+0000 7fbd48ff9700 1 -- 172.21.15.69:0/1265493914 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd500d89a0 con 0x559421d6b870 2023-07-06T20:32:32.568 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.563+0000 7fbd4b7fe700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbd440a12a0 0x7fbd440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.563+0000 7fbd4b7fe700 1 -- 172.21.15.69:0/1671520439 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fbd500cc630 con 0x7fbd5c0fe480 2023-07-06T20:32:32.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.563+0000 7fbd5ad9d700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbd440a12a0 0x7fbd440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.563+0000 7fbd743dcb80 1 -- 172.21.15.69:0/1265493914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6b870 msgr2=0x559421d6be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:32.569 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.563+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/1265493914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6b870 0x559421d6be00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd500d2640 tx=0x7fbd500d2950 comp rx=0 tx=0).stop 2023-07-06T20:32:32.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.564+0000 7fbd743dcb80 1 -- 172.21.15.69:0/1265493914 shutdown_connections 2023-07-06T20:32:32.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.564+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/1265493914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6c390 0x559421d6c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.564+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/1265493914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6af20 0x559421d6b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.564+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/1265493914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6b870 0x559421d6be00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.564+0000 7fbd743dcb80 1 -- 172.21.15.69:0/1265493914 >> 172.21.15.69:0/1265493914 conn(0x559421d60640 msgr2=0x559421d659f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:32:32.570 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.564+0000 7fbd5ad9d700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbd440a12a0 0x7fbd440a3750 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009fd0 tx=0x7fbd5400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:32:32.572 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.570+0000 7fbd743dcb80 1 -- 172.21.15.69:0/1265493914 shutdown_connections 2023-07-06T20:32:32.572 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.570+0000 7fbd743dcb80 1 -- 172.21.15.69:0/1265493914 wait complete. 2023-07-06T20:32:32.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.571+0000 7fbd743dcb80 1 -- start start 2023-07-06T20:32:32.573 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.571+0000 7fbd743dcb80 1 Processor -- start 2023-07-06T20:32:32.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.572+0000 7fbd743dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6af20 0x559421d748e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.572+0000 7fbd743dcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6b870 0x559421d74e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.574 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.572+0000 7fbd743dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6c390 0x559421d7be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.575 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.572+0000 7fbd743dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559421c5bac0 con 0x559421d6c390 2023-07-06T20:32:32.575 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.572+0000 7fbd743dcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559421c5b7c0 con 0x559421d6b870 2023-07-06T20:32:32.575 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.572+0000 7fbd743dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559421b3f920 con 0x559421d6af20 2023-07-06T20:32:32.576 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.572+0000 7fbd743dcb80 10 client.0 _ll_register_callbacks cb 0x559421c59760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:32:32.576 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.572+0000 7fbd5a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6af20 0x559421d748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.573+0000 7fbd5a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6af20 0x559421d748e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:44864/0 (socket says 172.21.15.69:44864) 2023-07-06T20:32:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.573+0000 7fbd5a59c700 1 -- 172.21.15.69:0/139483226 learned_addr learned my addr 172.21.15.69:0/139483226 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:32:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.573+0000 7fbd5a59c700 1 -- 172.21.15.69:0/139483226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6b870 msgr2=0x559421d74e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.573+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/139483226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6b870 0x559421d74e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.573+0000 7fbd5a59c700 1 -- 172.21.15.69:0/139483226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6c390 msgr2=0x559421d7be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:32.577 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.573+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/139483226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6c390 0x559421d7be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.573+0000 7fbd5a59c700 1 -- 172.21.15.69:0/139483226 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5400e140 con 0x559421d6af20 2023-07-06T20:32:32.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.573+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/139483226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6af20 0x559421d748e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbd50096420 tx=0x7fbd500cc870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e18397b986d7f8e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:32:32.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.573+0000 7fbd41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:32:32.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.574+0000 7fbd41ffb700 1 -- 172.21.15.69:0/139483226 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50095cf0 con 0x559421d6af20 2023-07-06T20:32:32.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.574+0000 7fbd41ffb700 1 -- 172.21.15.69:0/139483226 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fbd500999b0 con 0x559421d6af20 2023-07-06T20:32:32.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.574+0000 7fbd743dcb80 1 -- 172.21.15.69:0/139483226 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559421c154f0 con 0x559421d6af20 2023-07-06T20:32:32.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.574+0000 7fbd41ffb700 1 -- 172.21.15.69:0/139483226 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd50099780 con 0x559421d6af20 2023-07-06T20:32:32.578 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.574+0000 7fbd743dcb80 10 client.25765 fetch_fsmap learned FSMap version 30 2023-07-06T20:32:32.579 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.574+0000 7fbd743dcb80 1 -- 172.21.15.69:0/139483226 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fbd5c1a43f0 con 0x559421d6af20 2023-07-06T20:32:32.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.574+0000 7fbd41ffb700 1 -- 172.21.15.69:0/139483226 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559421c154f0 con 0x559421d6af20 2023-07-06T20:32:32.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.575+0000 7fbd41ffb700 1 -- 172.21.15.69:0/139483226 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbd5009cd90 con 0x559421d6af20 2023-07-06T20:32:32.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.575+0000 7fbd743dcb80 10 client.25765 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:32:32.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.575+0000 7fbd743dcb80 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:32:32.580 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.575+0000 7fbd743dcb80 20 client.25765 populate_metadata read hostname 'smithi069' 2023-07-06T20:32:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.575+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/139483226 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x559421d80100 0x559421d82510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.575+0000 7fbd743dcb80 4 client.25765 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:32:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.575+0000 7fbd743dcb80 1 -- 172.21.15.69:0/139483226 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559421d39d60 con 0x559421d80100 2023-07-06T20:32:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.575+0000 7fbd59d9b700 1 --2- 172.21.15.69:0/139483226 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x559421d80100 0x559421d82510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.576+0000 7fbd59d9b700 1 --2- 172.21.15.69:0/139483226 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x559421d80100 0x559421d82510 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696e807f05b3d61a server_cookie=64d15ebdc8020aef in_seq=0 out_seq=0 2023-07-06T20:32:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.576+0000 7fbd41ffb700 10 client.25765 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:32:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.577+0000 7fbd41ffb700 1 -- 172.21.15.69:0/139483226 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559421d39d60 con 0x559421d80100 2023-07-06T20:32:32.581 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.577+0000 7fbd41ffb700 10 client.25765 handle_command_reply: tid=0 2023-07-06T20:32:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.577+0000 7fbd743dcb80 1 client.25765 shutdown 2023-07-06T20:32:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.577+0000 7fbd743dcb80 1 -- 172.21.15.69:0/139483226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6af20 msgr2=0x559421d748e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.577+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/139483226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6af20 0x559421d748e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbd50096420 tx=0x7fbd500cc870 comp rx=0 tx=0).stop 2023-07-06T20:32:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.578+0000 7fbd743dcb80 1 -- 172.21.15.69:0/139483226 shutdown_connections 2023-07-06T20:32:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.578+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/139483226 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x559421d80100 0x559421d82510 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.578+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/139483226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6c390 0x559421d7be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.578+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/139483226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6af20 0x559421d748e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.582 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.578+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/139483226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6b870 0x559421d74e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.583 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.578+0000 7fbd743dcb80 1 -- 172.21.15.69:0/139483226 >> 172.21.15.69:0/139483226 conn(0x559421d60640 msgr2=0x559421d650b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:32:32.583 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.578+0000 7fbd41ffb700 0 client.25765 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:32:32.583 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.578+0000 7fbd743dcb80 1 -- 172.21.15.69:0/139483226 shutdown_connections 2023-07-06T20:32:32.584 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.578+0000 7fbd743dcb80 1 -- 172.21.15.69:0/139483226 wait complete. 2023-07-06T20:32:32.584 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.579+0000 7fbd743dcb80 20 client.25765 trim_cache size 0 max 16384 2023-07-06T20:32:32.611 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.608+0000 7fbd743dcb80 1 Processor -- start 2023-07-06T20:32:32.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.609+0000 7fbd743dcb80 1 -- start start 2023-07-06T20:32:32.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.609+0000 7fbd743dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6af20 0x559421d7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.609+0000 7fbd743dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6b870 0x559421d7be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.609+0000 7fbd743dcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6c390 0x559421d72760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.612 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.609+0000 7fbd743dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559421d60210 con 0x559421d6af20 2023-07-06T20:32:32.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.609+0000 7fbd743dcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559421d75a60 con 0x559421d6c390 2023-07-06T20:32:32.613 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.609+0000 7fbd743dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559421d60450 con 0x559421d6b870 2023-07-06T20:32:32.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.612+0000 7fbd59d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6b870 0x559421d7be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.612+0000 7fbd59d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6b870 0x559421d7be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:44866/0 (socket says 172.21.15.69:44866) 2023-07-06T20:32:32.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.612+0000 7fbd59d9b700 1 -- 172.21.15.69:0/2409658632 learned_addr learned my addr 172.21.15.69:0/2409658632 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:32:32.614 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.612+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/2409658632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6af20 0x559421d7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.612+0000 7fbd5ad9d700 1 --2- 172.21.15.69:0/2409658632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6c390 0x559421d72760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.612+0000 7fbd59d9b700 1 -- 172.21.15.69:0/2409658632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6c390 msgr2=0x559421d72760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:32.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.612+0000 7fbd59d9b700 1 --2- 172.21.15.69:0/2409658632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6c390 0x559421d72760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.615 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.612+0000 7fbd59d9b700 1 -- 172.21.15.69:0/2409658632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6af20 msgr2=0x559421d7c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:32.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.612+0000 7fbd59d9b700 1 --2- 172.21.15.69:0/2409658632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6af20 0x559421d7c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.612+0000 7fbd59d9b700 1 -- 172.21.15.69:0/2409658632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559421ad2c60 con 0x559421d6b870 2023-07-06T20:32:32.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.613+0000 7fbd59d9b700 1 --2- 172.21.15.69:0/2409658632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6b870 0x559421d7be20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009990 tx=0x7fbd4c01b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a92743ddfd319e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:32:32.616 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.613+0000 7fbd497fa700 1 -- 172.21.15.69:0/2409658632 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c01e040 con 0x559421d6b870 2023-07-06T20:32:32.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.614+0000 7fbd497fa700 1 -- 172.21.15.69:0/2409658632 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd4c01f040 con 0x559421d6b870 2023-07-06T20:32:32.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.614+0000 7fbd497fa700 1 -- 172.21.15.69:0/2409658632 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c005270 con 0x559421d6b870 2023-07-06T20:32:32.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.614+0000 7fbd743dcb80 1 -- 172.21.15.69:0/2409658632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6b870 msgr2=0x559421d7be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:32.617 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.614+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/2409658632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6b870 0x559421d7be20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009990 tx=0x7fbd4c01b040 comp rx=0 tx=0).stop 2023-07-06T20:32:32.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.615+0000 7fbd743dcb80 1 -- 172.21.15.69:0/2409658632 shutdown_connections 2023-07-06T20:32:32.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.615+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/2409658632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6c390 0x559421d72760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.615+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/2409658632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6af20 0x559421d7c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.615+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/2409658632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6b870 0x559421d7be20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.615+0000 7fbd743dcb80 1 -- 172.21.15.69:0/2409658632 >> 172.21.15.69:0/2409658632 conn(0x559421d60640 msgr2=0x559421d73600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:32:32.618 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.615+0000 7fbd743dcb80 1 -- 172.21.15.69:0/2409658632 shutdown_connections 2023-07-06T20:32:32.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.615+0000 7fbd743dcb80 1 -- 172.21.15.69:0/2409658632 wait complete. 2023-07-06T20:32:32.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.615+0000 7fbd743dcb80 1 -- start start 2023-07-06T20:32:32.619 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.616+0000 7fbd743dcb80 1 Processor -- start 2023-07-06T20:32:32.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.616+0000 7fbd743dcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6af20 0x559421d75fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd743dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6b870 0x559421d764e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd743dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6c390 0x559421d7b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd743dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559421d7b830 con 0x559421d6b870 2023-07-06T20:32:32.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd743dcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559421d7b9a0 con 0x559421d6af20 2023-07-06T20:32:32.620 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd743dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559421d7bb60 con 0x559421d6c390 2023-07-06T20:32:32.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd743dcb80 10 client.0 _ll_register_callbacks cb 0x7fbd4c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:32:32.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd5a59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6af20 0x559421d75fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd5a59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6af20 0x559421d75fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:50822/0 (socket says 172.21.15.69:50822) 2023-07-06T20:32:32.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd5a59c700 1 -- 172.21.15.69:0/3652320922 learned_addr learned my addr 172.21.15.69:0/3652320922 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:32:32.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd5ad9d700 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6c390 0x559421d7b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd59d9b700 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6b870 0x559421d764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd5a59c700 1 -- 172.21.15.69:0/3652320922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6c390 msgr2=0x559421d7b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:32.621 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6c390 0x559421d7b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd5a59c700 1 -- 172.21.15.69:0/3652320922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6b870 msgr2=0x559421d764e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:32:32.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6b870 0x559421d764e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd5a59c700 1 -- 172.21.15.69:0/3652320922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c104740 con 0x559421d6af20 2023-07-06T20:32:32.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.617+0000 7fbd59d9b700 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6b870 0x559421d764e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:32:32.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.618+0000 7fbd5a59c700 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6af20 0x559421d75fa0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbd50099940 tx=0x7fbd50097f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=517e4288aa8c6f25 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:32:32.622 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.618+0000 7fbd41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:32:32.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.618+0000 7fbd41ffb700 1 -- 172.21.15.69:0/3652320922 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50099780 con 0x559421d6af20 2023-07-06T20:32:32.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.618+0000 7fbd41ffb700 1 -- 172.21.15.69:0/3652320922 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fbd500a1e00 con 0x559421d6af20 2023-07-06T20:32:32.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.618+0000 7fbd743dcb80 1 -- 172.21.15.69:0/3652320922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd500ce590 con 0x559421d6af20 2023-07-06T20:32:32.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.619+0000 7fbd41ffb700 1 -- 172.21.15.69:0/3652320922 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd50095cf0 con 0x559421d6af20 2023-07-06T20:32:32.623 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.620+0000 7fbd743dcb80 10 client.25832 fetch_fsmap learned FSMap version 30 2023-07-06T20:32:32.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.620+0000 7fbd743dcb80 1 -- 172.21.15.69:0/3652320922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559421b10250 con 0x559421d6af20 2023-07-06T20:32:32.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.621+0000 7fbd41ffb700 1 -- 172.21.15.69:0/3652320922 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd500ce590 con 0x559421d6af20 2023-07-06T20:32:32.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.621+0000 7fbd41ffb700 1 -- 172.21.15.69:0/3652320922 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbd50099c10 con 0x559421d6af20 2023-07-06T20:32:32.624 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.621+0000 7fbd743dcb80 10 client.25832 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:32:32.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.621+0000 7fbd743dcb80 10 client.25832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:32:32.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.621+0000 7fbd743dcb80 20 client.25832 populate_metadata read hostname 'smithi069' 2023-07-06T20:32:32.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.621+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x559421de5c70 0x559421de8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:32:32.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.621+0000 7fbd743dcb80 4 client.25832 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:32:32.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.621+0000 7fbd743dcb80 1 -- 172.21.15.69:0/3652320922 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559421cb4eb0 con 0x559421de5c70 2023-07-06T20:32:32.625 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.623+0000 7fbd59d9b700 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x559421de5c70 0x559421de8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:32:32.626 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.624+0000 7fbd59d9b700 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x559421de5c70 0x559421de8050 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72f91ac0b4b43cf8 server_cookie=a8900344eb4ffb0a in_seq=0 out_seq=0 2023-07-06T20:32:32.626 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.624+0000 7fbd41ffb700 10 client.25832 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:32:32.626 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.624+0000 7fbd41ffb700 1 -- 172.21.15.69:0/3652320922 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559421cb4eb0 con 0x559421de5c70 2023-07-06T20:32:32.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.624+0000 7fbd41ffb700 10 client.25832 handle_command_reply: tid=0 2023-07-06T20:32:32.627 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.624+0000 7fbd743dcb80 1 client.25832 shutdown 2023-07-06T20:32:32.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.626+0000 7fbd743dcb80 1 -- 172.21.15.69:0/3652320922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6af20 msgr2=0x559421d75fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:32.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.626+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6af20 0x559421d75fa0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbd50099940 tx=0x7fbd50097f60 comp rx=0 tx=0).stop 2023-07-06T20:32:32.628 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.626+0000 7fbd743dcb80 1 -- 172.21.15.69:0/3652320922 shutdown_connections 2023-07-06T20:32:32.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.626+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x559421de5c70 0x559421de8050 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.626+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559421d6c390 0x559421d7b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.626+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559421d6af20 0x559421d75fa0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.626+0000 7fbd743dcb80 1 --2- 172.21.15.69:0/3652320922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559421d6b870 0x559421d764e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.626+0000 7fbd743dcb80 1 -- 172.21.15.69:0/3652320922 >> 172.21.15.69:0/3652320922 conn(0x559421d60640 msgr2=0x559421d7c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:32:32.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.626+0000 7fbd41ffb700 0 client.25832 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:32:32.629 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.626+0000 7fbd743dcb80 1 -- 172.21.15.69:0/3652320922 shutdown_connections 2023-07-06T20:32:32.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.626+0000 7fbd743dcb80 1 -- 172.21.15.69:0/3652320922 wait complete. 2023-07-06T20:32:32.631 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.627+0000 7fbd743dcb80 20 client.25832 trim_cache size 0 max 16384 2023-07-06T20:32:32.632 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:32:32.632 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:32:32.632 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:32:32.632 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:32:32.632 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:32:32.633 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:32:32.633 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:32:32.633 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:32:32.633 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:32:32.633 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:32:32.634 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.627+0000 7fbd609ff700 1 -- 172.21.15.69:0/1671520439 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbd440a12a0 msgr2=0x7fbd440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:32.634 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.627+0000 7fbd609ff700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbd440a12a0 0x7fbd440a3750 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009fd0 tx=0x7fbd5400b040 comp rx=0 tx=0).stop 2023-07-06T20:32:32.634 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.627+0000 7fbd609ff700 1 -- 172.21.15.69:0/1671520439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd5c0fe480 msgr2=0x7fbd5c19bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:32:32.634 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.627+0000 7fbd609ff700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd5c0fe480 0x7fbd5c19bf90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000d950 tx=0x7fbd5000dd10 comp rx=0 tx=0).stop 2023-07-06T20:32:32.634 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.628+0000 7fbd609ff700 1 -- 172.21.15.69:0/1671520439 shutdown_connections 2023-07-06T20:32:32.634 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.628+0000 7fbd609ff700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fbd440a12a0 0x7fbd440a3750 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.634 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.628+0000 7fbd609ff700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd5c1037a0 0x7fbd5c1a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.628+0000 7fbd609ff700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd5c0fe480 0x7fbd5c19bf90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.628+0000 7fbd609ff700 1 --2- 172.21.15.69:0/1671520439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c100de0 0x7fbd5c19c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:32:32.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.628+0000 7fbd609ff700 1 -- 172.21.15.69:0/1671520439 >> 172.21.15.69:0/1671520439 conn(0x7fbd5c0fa130 msgr2=0x7fbd5c0ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:32:32.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.628+0000 7fbd609ff700 1 -- 172.21.15.69:0/1671520439 shutdown_connections 2023-07-06T20:32:32.635 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:32:32.628+0000 7fbd609ff700 1 -- 172.21.15.69:0/1671520439 wait complete. 2023-07-06T20:32:32.659 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:32:34.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 645.0 s, 2213.4 tps, lat 45.056 ms stddev 32.358 2023-07-06T20:32:34.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:33 smithi069 ceph-mon[126399]: pgmap v724: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 MiB/s wr, 3.39k op/s 2023-07-06T20:32:34.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:33 smithi156 ceph-mon[129280]: pgmap v724: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 MiB/s wr, 3.39k op/s 2023-07-06T20:32:34.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:33 smithi098 ceph-mon[128968]: pgmap v724: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 MiB/s wr, 3.39k op/s 2023-07-06T20:32:36.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:36 smithi156 ceph-mon[129280]: pgmap v725: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 2.72k op/s 2023-07-06T20:32:36.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:36 smithi098 ceph-mon[128968]: pgmap v725: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 2.72k op/s 2023-07-06T20:32:36.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:36 smithi069 ceph-mon[126399]: pgmap v725: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 2.72k op/s 2023-07-06T20:32:38.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:38 smithi156 ceph-mon[129280]: pgmap v726: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 MiB/s wr, 2.63k op/s 2023-07-06T20:32:38.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:38 smithi098 ceph-mon[128968]: pgmap v726: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 MiB/s wr, 2.63k op/s 2023-07-06T20:32:38.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:38 smithi069 ceph-mon[126399]: pgmap v726: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 MiB/s wr, 2.63k op/s 2023-07-06T20:32:39.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 650.0 s, 1979.4 tps, lat 47.822 ms stddev 54.127 2023-07-06T20:32:40.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:40 smithi098 ceph-mon[128968]: pgmap v727: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 27 MiB/s wr, 1.80k op/s 2023-07-06T20:32:40.514 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:40 smithi069 ceph-mon[126399]: pgmap v727: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 27 MiB/s wr, 1.80k op/s 2023-07-06T20:32:40.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:40 smithi156 ceph-mon[129280]: pgmap v727: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 27 MiB/s wr, 1.80k op/s 2023-07-06T20:32:42.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:42 smithi069 ceph-mon[126399]: pgmap v728: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 1.64k op/s 2023-07-06T20:32:42.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:42 smithi156 ceph-mon[129280]: pgmap v728: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 1.64k op/s 2023-07-06T20:32:42.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:42 smithi098 ceph-mon[128968]: pgmap v728: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 1.64k op/s 2023-07-06T20:32:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 655.0 s, 1835.2 tps, lat 57.712 ms stddev 65.793 2023-07-06T20:32:44.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:44 smithi069 ceph-mon[126399]: pgmap v729: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.36k op/s 2023-07-06T20:32:44.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:44 smithi156 ceph-mon[129280]: pgmap v729: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.36k op/s 2023-07-06T20:32:44.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:44 smithi098 ceph-mon[128968]: pgmap v729: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.36k op/s 2023-07-06T20:32:46.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:46 smithi156 ceph-mon[129280]: pgmap v730: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 617 op/s 2023-07-06T20:32:46.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:46 smithi098 ceph-mon[128968]: pgmap v730: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 617 op/s 2023-07-06T20:32:46.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:46 smithi069 ceph-mon[126399]: pgmap v730: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 617 op/s 2023-07-06T20:32:47.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:47 smithi156 ceph-mon[129280]: pgmap v731: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 28 MiB/s wr, 812 op/s 2023-07-06T20:32:47.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:47 smithi098 ceph-mon[128968]: pgmap v731: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 28 MiB/s wr, 812 op/s 2023-07-06T20:32:47.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:47 smithi069 ceph-mon[126399]: pgmap v731: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 28 MiB/s wr, 812 op/s 2023-07-06T20:32:49.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 660.0 s, 1853.4 tps, lat 53.885 ms stddev 66.599 2023-07-06T20:32:49.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:49 smithi069 ceph-mon[126399]: pgmap v732: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 21 MiB/s wr, 691 op/s 2023-07-06T20:32:49.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:49 smithi156 ceph-mon[129280]: pgmap v732: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 21 MiB/s wr, 691 op/s 2023-07-06T20:32:49.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:49 smithi098 ceph-mon[128968]: pgmap v732: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 21 MiB/s wr, 691 op/s 2023-07-06T20:32:51.867 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:51 smithi156 ceph-mon[129280]: pgmap v733: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 24 MiB/s wr, 981 op/s 2023-07-06T20:32:51.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:51 smithi098 ceph-mon[128968]: pgmap v733: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 24 MiB/s wr, 981 op/s 2023-07-06T20:32:52.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:51 smithi069 ceph-mon[126399]: pgmap v733: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 24 MiB/s wr, 981 op/s 2023-07-06T20:32:54.015 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:53 smithi069 ceph-mon[126399]: pgmap v734: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 34 MiB/s wr, 2.02k op/s 2023-07-06T20:32:54.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:53 smithi156 ceph-mon[129280]: pgmap v734: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 34 MiB/s wr, 2.02k op/s 2023-07-06T20:32:54.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:53 smithi098 ceph-mon[128968]: pgmap v734: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 34 MiB/s wr, 2.02k op/s 2023-07-06T20:32:54.233 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 665.0 s, 437.5 tps, lat 223.296 ms stddev 255.171 2023-07-06T20:32:56.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:55 smithi069 ceph-mon[126399]: pgmap v735: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 33 MiB/s wr, 2.37k op/s 2023-07-06T20:32:56.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:55 smithi156 ceph-mon[129280]: pgmap v735: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 33 MiB/s wr, 2.37k op/s 2023-07-06T20:32:56.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:55 smithi098 ceph-mon[128968]: pgmap v735: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 33 MiB/s wr, 2.37k op/s 2023-07-06T20:32:58.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:57 smithi069 ceph-mon[126399]: pgmap v736: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 44 MiB/s wr, 3.44k op/s 2023-07-06T20:32:58.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:57 smithi156 ceph-mon[129280]: pgmap v736: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 44 MiB/s wr, 3.44k op/s 2023-07-06T20:32:58.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:57 smithi098 ceph-mon[128968]: pgmap v736: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 44 MiB/s wr, 3.44k op/s 2023-07-06T20:32:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 670.0 s, 2111.5 tps, lat 48.660 ms stddev 40.453 2023-07-06T20:33:00.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:32:59 smithi156 ceph-mon[129280]: pgmap v737: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 MiB/s wr, 3.45k op/s 2023-07-06T20:33:00.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:32:59 smithi098 ceph-mon[128968]: pgmap v737: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 MiB/s wr, 3.45k op/s 2023-07-06T20:33:00.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:32:59 smithi069 ceph-mon[126399]: pgmap v737: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 MiB/s wr, 3.45k op/s 2023-07-06T20:33:02.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:01 smithi156 ceph-mon[129280]: pgmap v738: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 MiB/s wr, 3.45k op/s 2023-07-06T20:33:02.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:01 smithi098 ceph-mon[128968]: pgmap v738: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 MiB/s wr, 3.45k op/s 2023-07-06T20:33:02.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:01 smithi069 ceph-mon[126399]: pgmap v738: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 MiB/s wr, 3.45k op/s 2023-07-06T20:33:02.660 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:33:03.065 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.032+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3873852910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0071960 msgr2=0x7f9ca010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.032+0000 7f9ca4fff700 1 --2- 172.21.15.69:0/3873852910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0071960 0x7f9ca010b900 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c940098f0 tx=0x7f9c94009c00 comp rx=0 tx=0).stop 2023-07-06T20:33:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.032+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3873852910 shutdown_connections 2023-07-06T20:33:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.032+0000 7f9ca4fff700 1 --2- 172.21.15.69:0/3873852910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ca010be90 0x7f9ca010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.032+0000 7f9ca4fff700 1 --2- 172.21.15.69:0/3873852910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca0071010 0x7f9ca0071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.032+0000 7f9ca4fff700 1 --2- 172.21.15.69:0/3873852910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0071960 0x7f9ca010b900 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.032+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3873852910 >> 172.21.15.69:0/3873852910 conn(0x7f9ca006cbc0 msgr2=0x7f9ca006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:33:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.033+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3873852910 shutdown_connections 2023-07-06T20:33:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.033+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3873852910 wait complete. 2023-07-06T20:33:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.034+0000 7f9ca4fff700 1 Processor -- start 2023-07-06T20:33:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.034+0000 7f9ca4fff700 1 -- start start 2023-07-06T20:33:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.035+0000 7f9ca4fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca0071010 0x7f9ca0115f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.035+0000 7f9ca4fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ca0071960 0x7f9ca0116440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.035+0000 7f9ca4fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca010be90 0x7f9ca011e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.035+0000 7f9ca4fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca011eef0 con 0x7f9ca010be90 2023-07-06T20:33:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.035+0000 7f9ca4fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca011f060 con 0x7f9ca0071960 2023-07-06T20:33:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.035+0000 7f9ca4fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca011f220 con 0x7f9ca0071010 2023-07-06T20:33:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.036+0000 7f9c9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca0071010 0x7f9ca0115f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.036+0000 7f9c9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca0071010 0x7f9ca0115f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:33218/0 (socket says 172.21.15.69:33218) 2023-07-06T20:33:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.036+0000 7f9c9e59c700 1 -- 172.21.15.69:0/3013973747 learned_addr learned my addr 172.21.15.69:0/3013973747 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:33:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.036+0000 7f9c9ed9d700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca010be90 0x7f9ca011e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.037+0000 7f9c9e59c700 1 -- 172.21.15.69:0/3013973747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ca0071960 msgr2=0x7f9ca0116440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:03.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.037+0000 7f9c9e59c700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ca0071960 0x7f9ca0116440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.037+0000 7f9c9e59c700 1 -- 172.21.15.69:0/3013973747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca010be90 msgr2=0x7f9ca011e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:03.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.037+0000 7f9c9e59c700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca010be90 0x7f9ca011e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.069 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.037+0000 7f9c9e59c700 1 -- 172.21.15.69:0/3013973747 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c94009520 con 0x7f9ca0071010 2023-07-06T20:33:03.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.076+0000 7f9c9e59c700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca0071010 0x7f9ca0115f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000e9c0 tx=0x7f9c9000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:33:03.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.077+0000 7f9c8f7fe700 1 -- 172.21.15.69:0/3013973747 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000cbd0 con 0x7f9ca0071010 2023-07-06T20:33:03.085 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.077+0000 7f9c8f7fe700 1 -- 172.21.15.69:0/3013973747 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c9000eeb0 con 0x7f9ca0071010 2023-07-06T20:33:03.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.078+0000 7f9c8f7fe700 1 -- 172.21.15.69:0/3013973747 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90005580 con 0x7f9ca0071010 2023-07-06T20:33:03.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.078+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3013973747 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca011f500 con 0x7f9ca0071010 2023-07-06T20:33:03.086 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.078+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3013973747 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca011f9c0 con 0x7f9ca0071010 2023-07-06T20:33:03.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.092+0000 7f9cb898fb80 1 Processor -- start 2023-07-06T20:33:03.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.092+0000 7f9cb898fb80 1 -- start start 2023-07-06T20:33:03.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.093+0000 7f9cb898fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560e8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.093+0000 7f9cb898fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e8870 0x5640560e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.093+0000 7f9cb898fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e9390 0x5640560e9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.093+0000 7f9cb898fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564055feccb0 con 0x5640560e8870 2023-07-06T20:33:03.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.093+0000 7f9cb898fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405603fa40 con 0x5640560e9390 2023-07-06T20:33:03.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.093+0000 7f9cb898fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564056038ed0 con 0x5640560e7f20 2023-07-06T20:33:03.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.095+0000 7f9c9ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.095+0000 7f9c9dd9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e9390 0x5640560e9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.095+0000 7f9c9ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560e8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:33220/0 (socket says 172.21.15.69:33220) 2023-07-06T20:33:03.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.095+0000 7f9c9ed9d700 1 -- 172.21.15.69:0/3519495639 learned_addr learned my addr 172.21.15.69:0/3519495639 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:33:03.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.097+0000 7f9c8f7fe700 1 -- 172.21.15.69:0/3013973747 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9c90004750 con 0x7f9ca0071010 2023-07-06T20:33:03.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.100+0000 7f9c8f7fe700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9c880a1350 0x7f9c880a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.101+0000 7f9c8f7fe700 1 -- 172.21.15.69:0/3013973747 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f9c90014070 con 0x7f9ca0071010 2023-07-06T20:33:03.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.102+0000 7f9c9ed9d700 1 -- 172.21.15.69:0/3519495639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e9390 msgr2=0x5640560e9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:33:03.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.102+0000 7f9c9ed9d700 1 --2- 172.21.15.69:0/3519495639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e9390 0x5640560e9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.102+0000 7f9c9ed9d700 1 -- 172.21.15.69:0/3519495639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e8870 msgr2=0x5640560e8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:33:03.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.102+0000 7f9c9ed9d700 1 --2- 172.21.15.69:0/3519495639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e8870 0x5640560e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.102+0000 7f9c9ed9d700 1 -- 172.21.15.69:0/3519495639 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564055e66de0 con 0x5640560e7f20 2023-07-06T20:33:03.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.102+0000 7f9c9ed9d700 1 --2- 172.21.15.69:0/3519495639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560e8330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800a760 tx=0x7f9c980078d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=905d91c11ca4446a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:33:03.105 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.102+0000 7f9c9ed9d700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9c880a1350 0x7f9c880a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.103+0000 7f9c8cff9700 1 -- 172.21.15.69:0/3519495639 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800f050 con 0x5640560e7f20 2023-07-06T20:33:03.106 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.103+0000 7f9c8cff9700 1 -- 172.21.15.69:0/3519495639 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c9801b070 con 0x5640560e7f20 2023-07-06T20:33:03.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.103+0000 7f9c8cff9700 1 -- 172.21.15.69:0/3519495639 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c980149a0 con 0x5640560e7f20 2023-07-06T20:33:03.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.104+0000 7f9cb898fb80 1 -- 172.21.15.69:0/3519495639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 msgr2=0x5640560e8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:03.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.104+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/3519495639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560e8330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800a760 tx=0x7f9c980078d0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.105+0000 7f9c9ed9d700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9c880a1350 0x7f9c880a3800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9c9801e000 tx=0x7f9c98014be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:33:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.108+0000 7f9cb898fb80 1 -- 172.21.15.69:0/3519495639 shutdown_connections 2023-07-06T20:33:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.108+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/3519495639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e9390 0x5640560e9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.108+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/3519495639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560e8330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.108+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/3519495639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e8870 0x5640560e8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.108+0000 7f9cb898fb80 1 -- 172.21.15.69:0/3519495639 >> 172.21.15.69:0/3519495639 conn(0x5640560dd640 msgr2=0x5640560e29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:33:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.108+0000 7f9cb898fb80 1 -- 172.21.15.69:0/3519495639 shutdown_connections 2023-07-06T20:33:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.108+0000 7f9cb898fb80 1 -- 172.21.15.69:0/3519495639 wait complete. 2023-07-06T20:33:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.109+0000 7f9cb898fb80 1 -- start start 2023-07-06T20:33:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.109+0000 7f9cb898fb80 1 Processor -- start 2023-07-06T20:33:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.110+0000 7f9cb898fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.110+0000 7f9cb898fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e8870 0x5640560f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.110+0000 7f9cb898fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 0x5640560f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.110+0000 7f9cb898fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564055fd8ac0 con 0x5640560e9390 2023-07-06T20:33:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.110+0000 7f9cb898fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564055fd87c0 con 0x5640560e8870 2023-07-06T20:33:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.110+0000 7f9cb898fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564055ebc920 con 0x5640560e7f20 2023-07-06T20:33:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.110+0000 7f9cb898fb80 10 client.0 _ll_register_callbacks cb 0x564055fd6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:33:03.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.112+0000 7f9c9ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 0x5640560f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.112+0000 7f9c9ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 0x5640560f8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49354/0 (socket says 172.21.15.69:49354) 2023-07-06T20:33:03.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.112+0000 7f9c9ed9d700 1 -- 172.21.15.69:0/436681103 learned_addr learned my addr 172.21.15.69:0/436681103 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:33:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.114+0000 7f9c9ed9d700 1 -- 172.21.15.69:0/436681103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 msgr2=0x5640560f18b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:33:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.114+0000 7f9c9ed9d700 1 --2- 172.21.15.69:0/436681103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f18b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.114+0000 7f9c9ed9d700 1 -- 172.21.15.69:0/436681103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e8870 msgr2=0x5640560f1df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:33:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.114+0000 7f9c9ed9d700 1 --2- 172.21.15.69:0/436681103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e8870 0x5640560f1df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.114+0000 7f9c9ed9d700 1 -- 172.21.15.69:0/436681103 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9800a410 con 0x5640560e9390 2023-07-06T20:33:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.116+0000 7f9c9ed9d700 1 --2- 172.21.15.69:0/436681103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 0x5640560f8e30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800ad50 tx=0x7f9c98014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7269876314a2a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:33:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.116+0000 7f9c85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:33:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.116+0000 7f9c85ffb700 1 -- 172.21.15.69:0/436681103 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800f040 con 0x5640560e9390 2023-07-06T20:33:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.117+0000 7f9c85ffb700 1 -- 172.21.15.69:0/436681103 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f9c98027030 con 0x5640560e9390 2023-07-06T20:33:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.117+0000 7f9cb898fb80 1 -- 172.21.15.69:0/436681103 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564055f924f0 con 0x5640560e9390 2023-07-06T20:33:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.117+0000 7f9c85ffb700 1 -- 172.21.15.69:0/436681103 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c980126e0 con 0x5640560e9390 2023-07-06T20:33:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.118+0000 7f9cb898fb80 10 client.15669 fetch_fsmap learned FSMap version 30 2023-07-06T20:33:03.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.118+0000 7f9cb898fb80 1 -- 172.21.15.69:0/436681103 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564055e4fc60 con 0x5640560e9390 2023-07-06T20:33:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.119+0000 7f9c85ffb700 1 -- 172.21.15.69:0/436681103 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564055f924f0 con 0x5640560e9390 2023-07-06T20:33:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.119+0000 7f9c85ffb700 1 -- 172.21.15.69:0/436681103 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9c9802e820 con 0x5640560e9390 2023-07-06T20:33:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.119+0000 7f9cb898fb80 10 client.15669 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:33:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.119+0000 7f9cb898fb80 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:33:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.119+0000 7f9cb898fb80 20 client.15669 populate_metadata read hostname 'smithi069' 2023-07-06T20:33:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.119+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/436681103 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5640560fd700 0x5640560ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.119+0000 7f9cb898fb80 4 client.15669 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:33:03.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.119+0000 7f9cb898fb80 1 -- 172.21.15.69:0/436681103 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640560b6d60 con 0x5640560fd700 2023-07-06T20:33:03.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.120+0000 7f9c9e59c700 1 --2- 172.21.15.69:0/436681103 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5640560fd700 0x5640560ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.121+0000 7f9c85ffb700 10 client.15669 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:33:03.123 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.121+0000 7f9c9e59c700 1 --2- 172.21.15.69:0/436681103 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5640560fd700 0x5640560ffb10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89a0160a338fd59a server_cookie=6a41d49866d332d2 in_seq=0 out_seq=0 2023-07-06T20:33:03.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.122+0000 7f9c85ffb700 1 -- 172.21.15.69:0/436681103 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5640560b6d60 con 0x5640560fd700 2023-07-06T20:33:03.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.122+0000 7f9c85ffb700 10 client.15669 handle_command_reply: tid=0 2023-07-06T20:33:03.124 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.122+0000 7f9cb898fb80 1 client.15669 shutdown 2023-07-06T20:33:03.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.122+0000 7f9cb898fb80 1 -- 172.21.15.69:0/436681103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 msgr2=0x5640560f8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:03.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.122+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/436681103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 0x5640560f8e30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800ad50 tx=0x7f9c98014000 comp rx=0 tx=0).stop 2023-07-06T20:33:03.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.122+0000 7f9cb898fb80 1 -- 172.21.15.69:0/436681103 shutdown_connections 2023-07-06T20:33:03.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.122+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/436681103 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5640560fd700 0x5640560ffb10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.122+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/436681103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 0x5640560f8e30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.122+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/436681103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.125 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.122+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/436681103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e8870 0x5640560f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.126 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.122+0000 7f9cb898fb80 1 -- 172.21.15.69:0/436681103 >> 172.21.15.69:0/436681103 conn(0x5640560dd640 msgr2=0x5640560e20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:33:03.126 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.122+0000 7f9c85ffb700 0 client.15669 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:33:03.126 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.123+0000 7f9cb898fb80 1 -- 172.21.15.69:0/436681103 shutdown_connections 2023-07-06T20:33:03.126 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.123+0000 7f9cb898fb80 1 -- 172.21.15.69:0/436681103 wait complete. 2023-07-06T20:33:03.131 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.129+0000 7f9cb898fb80 20 client.15669 trim_cache size 0 max 16384 2023-07-06T20:33:03.158 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.155+0000 7f9cb898fb80 1 Processor -- start 2023-07-06T20:33:03.158 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.156+0000 7f9cb898fb80 1 -- start start 2023-07-06T20:33:03.158 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.156+0000 7f9cb898fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.159 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.156+0000 7f9cb898fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e8870 0x5640560f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.159 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.156+0000 7f9cb898fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e9390 0x5640560ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.159 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.156+0000 7f9cb898fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640560dd210 con 0x5640560e8870 2023-07-06T20:33:03.159 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.156+0000 7f9cb898fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640560ef540 con 0x5640560e9390 2023-07-06T20:33:03.159 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.156+0000 7f9cb898fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640560fd380 con 0x5640560e7f20 2023-07-06T20:33:03.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.157+0000 7f9c9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.157+0000 7f9c9ed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e9390 0x5640560ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.157+0000 7f9c9ed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e9390 0x5640560ef790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:41310/0 (socket says 172.21.15.69:41310) 2023-07-06T20:33:03.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.157+0000 7f9c9ed9d700 1 -- 172.21.15.69:0/1660417553 learned_addr learned my addr 172.21.15.69:0/1660417553 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:33:03.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.157+0000 7f9c9e59c700 1 -- 172.21.15.69:0/1660417553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e9390 msgr2=0x5640560ef790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:33:03.160 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.157+0000 7f9c9dd9b700 1 --2- 172.21.15.69:0/1660417553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e8870 0x5640560f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.157+0000 7f9c9e59c700 1 --2- 172.21.15.69:0/1660417553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e9390 0x5640560ef790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.157+0000 7f9c9e59c700 1 -- 172.21.15.69:0/1660417553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e8870 msgr2=0x5640560f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:33:03.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.157+0000 7f9c9e59c700 1 --2- 172.21.15.69:0/1660417553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e8870 0x5640560f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.157+0000 7f9c9e59c700 1 -- 172.21.15.69:0/1660417553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564055e8d250 con 0x5640560e7f20 2023-07-06T20:33:03.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.158+0000 7f9c9ed9d700 1 --2- 172.21.15.69:0/1660417553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e9390 0x5640560ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:33:03.161 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.158+0000 7f9c9dd9b700 1 --2- 172.21.15.69:0/1660417553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e8870 0x5640560f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:33:03.162 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.158+0000 7f9c9e59c700 1 --2- 172.21.15.69:0/1660417553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f0600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9c900ca680 tx=0x7f9c90009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f44c1d1490e5b226 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:33:03.164 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.162+0000 7f9c8d7fa700 1 -- 172.21.15.69:0/1660417553 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c900b1070 con 0x5640560e7f20 2023-07-06T20:33:03.164 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.162+0000 7f9c8d7fa700 1 -- 172.21.15.69:0/1660417553 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c9009dff0 con 0x5640560e7f20 2023-07-06T20:33:03.164 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.162+0000 7f9c8d7fa700 1 -- 172.21.15.69:0/1660417553 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9009a220 con 0x5640560e7f20 2023-07-06T20:33:03.165 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.163+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1660417553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 msgr2=0x5640560f0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:03.165 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.163+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/1660417553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f0600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9c900ca680 tx=0x7f9c90009f90 comp rx=0 tx=0).stop 2023-07-06T20:33:03.165 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.163+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1660417553 shutdown_connections 2023-07-06T20:33:03.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.163+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/1660417553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e9390 0x5640560ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.163+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/1660417553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f0600 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.163+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/1660417553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e8870 0x5640560f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.163+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1660417553 >> 172.21.15.69:0/1660417553 conn(0x5640560dd640 msgr2=0x564056104430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:33:03.166 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.163+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1660417553 shutdown_connections 2023-07-06T20:33:03.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.163+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1660417553 wait complete. 2023-07-06T20:33:03.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.164+0000 7f9cb898fb80 1 -- start start 2023-07-06T20:33:03.167 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.164+0000 7f9cb898fb80 1 Processor -- start 2023-07-06T20:33:03.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.165+0000 7f9cb898fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.165+0000 7f9cb898fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e8870 0x5640560f5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.165+0000 7f9cb898fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 0x5640560f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.165+0000 7f9cb898fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640560f8970 con 0x5640560e9390 2023-07-06T20:33:03.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.165+0000 7f9cb898fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640560f8ae0 con 0x5640560e8870 2023-07-06T20:33:03.168 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.165+0000 7f9cb898fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640560f8ca0 con 0x5640560e7f20 2023-07-06T20:33:03.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.165+0000 7f9cb898fb80 10 client.0 _ll_register_callbacks cb 0x7f9c94009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:33:03.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.165+0000 7f9c9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.165+0000 7f9c9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:33236/0 (socket says 172.21.15.69:33236) 2023-07-06T20:33:03.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.165+0000 7f9c9e59c700 1 -- 172.21.15.69:0/1440908946 learned_addr learned my addr 172.21.15.69:0/1440908946 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:33:03.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.165+0000 7f9c9dd9b700 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e8870 0x5640560f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c9e59c700 1 -- 172.21.15.69:0/1440908946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e8870 msgr2=0x5640560f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:33:03.169 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c9e59c700 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e8870 0x5640560f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.170 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c9e59c700 1 -- 172.21.15.69:0/1440908946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 msgr2=0x5640560f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:33:03.170 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c9ed9d700 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 0x5640560f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.170 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c9e59c700 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 0x5640560f8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.170 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c9e59c700 1 -- 172.21.15.69:0/1440908946 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c900cce70 con 0x5640560e7f20 2023-07-06T20:33:03.170 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c9dd9b700 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e8870 0x5640560f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:33:03.170 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c9ed9d700 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 0x5640560f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:33:03.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c9e59c700 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f4ed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9c90096a30 tx=0x7f9c900a24c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aa510956b3511c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:33:03.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:33:03.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c85ffb700 1 -- 172.21.15.69:0/1440908946 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c900b1030 con 0x5640560e7f20 2023-07-06T20:33:03.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c85ffb700 1 -- 172.21.15.69:0/1440908946 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f9c900b2070 con 0x5640560e7f20 2023-07-06T20:33:03.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1440908946 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c9801b070 con 0x5640560e7f20 2023-07-06T20:33:03.172 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9c85ffb700 1 -- 172.21.15.69:0/1440908946 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c900ca240 con 0x5640560e7f20 2023-07-06T20:33:03.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9cb898fb80 10 client.25786 fetch_fsmap learned FSMap version 30 2023-07-06T20:33:03.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.166+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1440908946 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564055e652b0 con 0x5640560e7f20 2023-07-06T20:33:03.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.167+0000 7f9c85ffb700 1 -- 172.21.15.69:0/1440908946 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c9801b070 con 0x5640560e7f20 2023-07-06T20:33:03.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.167+0000 7f9c85ffb700 1 -- 172.21.15.69:0/1440908946 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9c900a1f80 con 0x5640560e7f20 2023-07-06T20:33:03.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.167+0000 7f9cb898fb80 10 client.25786 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:33:03.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.167+0000 7f9cb898fb80 10 client.25786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:33:03.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.167+0000 7f9cb898fb80 20 client.25786 populate_metadata read hostname 'smithi069' 2023-07-06T20:33:03.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.167+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x564056152500 0x564056154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:03.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.167+0000 7f9cb898fb80 4 client.25786 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:33:03.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.167+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1440908946 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564056031eb0 con 0x564056152500 2023-07-06T20:33:03.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.167+0000 7f9c9dd9b700 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x564056152500 0x564056154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:03.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.168+0000 7f9c85ffb700 10 client.25786 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:33:03.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.168+0000 7f9c9dd9b700 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x564056152500 0x564056154910 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3ae14e9652a586 server_cookie=f7bc346aeb3db82e in_seq=0 out_seq=0 2023-07-06T20:33:03.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.168+0000 7f9c85ffb700 1 -- 172.21.15.69:0/1440908946 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564056031eb0 con 0x564056152500 2023-07-06T20:33:03.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.168+0000 7f9c85ffb700 10 client.25786 handle_command_reply: tid=0 2023-07-06T20:33:03.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.168+0000 7f9cb898fb80 1 client.25786 shutdown 2023-07-06T20:33:03.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.169+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1440908946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 msgr2=0x5640560f4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:03.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.169+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f4ed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9c90096a30 tx=0x7f9c900a24c0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.169+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1440908946 shutdown_connections 2023-07-06T20:33:03.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.169+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x564056152500 0x564056154910 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.169+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640560e9390 0x5640560f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.169+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640560e7f20 0x5640560f4ed0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.175 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.169+0000 7f9cb898fb80 1 --2- 172.21.15.69:0/1440908946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640560e8870 0x5640560f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.169+0000 7f9c85ffb700 0 client.25786 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:33:03.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.169+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1440908946 >> 172.21.15.69:0/1440908946 conn(0x5640560dd640 msgr2=0x5640560ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:33:03.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.169+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1440908946 shutdown_connections 2023-07-06T20:33:03.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.169+0000 7f9cb898fb80 1 -- 172.21.15.69:0/1440908946 wait complete. 2023-07-06T20:33:03.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.169+0000 7f9cb898fb80 20 client.25786 trim_cache size 0 max 16384 2023-07-06T20:33:03.176 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:33:03.177 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:33:03.177 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:33:03.177 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:33:03.177 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:33:03.177 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:33:03.177 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:33:03.177 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:33:03.177 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:33:03.178 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:33:03.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.170+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3013973747 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9c880a1350 msgr2=0x7f9c880a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:03.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.170+0000 7f9ca4fff700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9c880a1350 0x7f9c880a3800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9c9801e000 tx=0x7f9c98014be0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.170+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3013973747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca0071010 msgr2=0x7f9ca0115f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:03.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.170+0000 7f9ca4fff700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca0071010 0x7f9ca0115f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000e9c0 tx=0x7f9c9000ecd0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.170+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3013973747 shutdown_connections 2023-07-06T20:33:03.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.170+0000 7f9ca4fff700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f9c880a1350 0x7f9c880a3800 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.170+0000 7f9ca4fff700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca010be90 0x7f9ca011e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.170+0000 7f9ca4fff700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca0071010 0x7f9ca0115f00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.170+0000 7f9ca4fff700 1 --2- 172.21.15.69:0/3013973747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ca0071960 0x7f9ca0116440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:03.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.170+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3013973747 >> 172.21.15.69:0/3013973747 conn(0x7f9ca006cbc0 msgr2=0x7f9ca006ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:33:03.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.170+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3013973747 shutdown_connections 2023-07-06T20:33:03.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:03.170+0000 7f9ca4fff700 1 -- 172.21.15.69:0/3013973747 wait complete. 2023-07-06T20:33:03.196 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:33:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 675.0 s, 2189.9 tps, lat 45.489 ms stddev 34.485 2023-07-06T20:33:04.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:03 smithi069 ceph-mon[126399]: pgmap v739: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 MiB/s wr, 3.28k op/s 2023-07-06T20:33:04.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:03 smithi156 ceph-mon[129280]: pgmap v739: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 MiB/s wr, 3.28k op/s 2023-07-06T20:33:04.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:03 smithi098 ceph-mon[128968]: pgmap v739: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 MiB/s wr, 3.28k op/s 2023-07-06T20:33:06.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:05 smithi069 ceph-mon[126399]: pgmap v740: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 2.21k op/s 2023-07-06T20:33:06.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:05 smithi156 ceph-mon[129280]: pgmap v740: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 2.21k op/s 2023-07-06T20:33:06.391 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:05 smithi098 ceph-mon[128968]: pgmap v740: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 2.21k op/s 2023-07-06T20:33:08.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:07 smithi069 ceph-mon[126399]: pgmap v741: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2023-07-06T20:33:08.360 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:07 smithi156 ceph-mon[129280]: pgmap v741: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2023-07-06T20:33:08.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:08 smithi098 ceph-mon[128968]: pgmap v741: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2023-07-06T20:33:09.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 680.0 s, 1504.7 tps, lat 66.376 ms stddev 96.895 2023-07-06T20:33:10.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:10 smithi156 ceph-mon[129280]: pgmap v742: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 809 op/s 2023-07-06T20:33:10.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:10 smithi098 ceph-mon[128968]: pgmap v742: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 809 op/s 2023-07-06T20:33:10.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:10 smithi069 ceph-mon[126399]: pgmap v742: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 809 op/s 2023-07-06T20:33:11.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:11 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:33:11.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:11 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:33:11.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:11 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:33:12.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:12 smithi069 ceph-mon[126399]: pgmap v743: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 572 op/s 2023-07-06T20:33:12.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:33:12.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:33:12.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:33:12.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:12 smithi156 ceph-mon[129280]: pgmap v743: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 572 op/s 2023-07-06T20:33:12.612 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:33:12.612 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:33:12.612 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:33:12.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:12 smithi098 ceph-mon[128968]: pgmap v743: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 572 op/s 2023-07-06T20:33:12.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:33:12.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:33:12.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:33:14.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 685.0 s, 2146.9 tps, lat 46.876 ms stddev 33.534 2023-07-06T20:33:14.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:14 smithi069 ceph-mon[126399]: pgmap v744: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 719 op/s 2023-07-06T20:33:14.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:14 smithi156 ceph-mon[129280]: pgmap v744: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 719 op/s 2023-07-06T20:33:14.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:14 smithi098 ceph-mon[128968]: pgmap v744: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 719 op/s 2023-07-06T20:33:16.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:16 smithi156 ceph-mon[129280]: pgmap v745: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 22 MiB/s wr, 615 op/s 2023-07-06T20:33:16.628 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:16 smithi098 ceph-mon[128968]: pgmap v745: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 22 MiB/s wr, 615 op/s 2023-07-06T20:33:16.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:16 smithi069 ceph-mon[126399]: pgmap v745: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 22 MiB/s wr, 615 op/s 2023-07-06T20:33:17.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:17 smithi069 ceph-mon[126399]: pgmap v746: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 22 MiB/s wr, 819 op/s 2023-07-06T20:33:17.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:17 smithi156 ceph-mon[129280]: pgmap v746: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 22 MiB/s wr, 819 op/s 2023-07-06T20:33:17.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:17 smithi098 ceph-mon[128968]: pgmap v746: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 22 MiB/s wr, 819 op/s 2023-07-06T20:33:19.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 690.0 s, 630.0 tps, lat 157.928 ms stddev 207.706 2023-07-06T20:33:19.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:19 smithi156 ceph-mon[129280]: pgmap v747: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 1.36k op/s 2023-07-06T20:33:19.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:19 smithi098 ceph-mon[128968]: pgmap v747: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 1.36k op/s 2023-07-06T20:33:20.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:19 smithi069 ceph-mon[126399]: pgmap v747: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 1.36k op/s 2023-07-06T20:33:21.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:21 smithi156 ceph-mon[129280]: pgmap v748: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 29 MiB/s wr, 1.80k op/s 2023-07-06T20:33:21.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:21 smithi098 ceph-mon[128968]: pgmap v748: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 29 MiB/s wr, 1.80k op/s 2023-07-06T20:33:22.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:21 smithi069 ceph-mon[126399]: pgmap v748: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 29 MiB/s wr, 1.80k op/s 2023-07-06T20:33:23.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:23 smithi156 ceph-mon[129280]: pgmap v749: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 40 MiB/s wr, 2.90k op/s 2023-07-06T20:33:23.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:23 smithi098 ceph-mon[128968]: pgmap v749: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 40 MiB/s wr, 2.90k op/s 2023-07-06T20:33:24.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:23 smithi069 ceph-mon[126399]: pgmap v749: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 40 MiB/s wr, 2.90k op/s 2023-07-06T20:33:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 695.0 s, 1101.8 tps, lat 90.889 ms stddev 149.412 2023-07-06T20:33:25.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:25 smithi098 ceph-mon[128968]: pgmap v750: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 3.54k op/s 2023-07-06T20:33:26.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:25 smithi069 ceph-mon[126399]: pgmap v750: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 3.54k op/s 2023-07-06T20:33:26.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:25 smithi156 ceph-mon[129280]: pgmap v750: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 3.54k op/s 2023-07-06T20:33:28.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:27 smithi069 ceph-mon[126399]: pgmap v751: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 MiB/s wr, 3.62k op/s 2023-07-06T20:33:28.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:27 smithi156 ceph-mon[129280]: pgmap v751: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 MiB/s wr, 3.62k op/s 2023-07-06T20:33:28.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:27 smithi098 ceph-mon[128968]: pgmap v751: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 MiB/s wr, 3.62k op/s 2023-07-06T20:33:29.233 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 700.0 s, 2202.6 tps, lat 45.202 ms stddev 32.434 2023-07-06T20:33:30.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:29 smithi069 ceph-mon[126399]: pgmap v752: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 MiB/s wr, 3.57k op/s 2023-07-06T20:33:30.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:29 smithi156 ceph-mon[129280]: pgmap v752: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 MiB/s wr, 3.57k op/s 2023-07-06T20:33:30.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:29 smithi098 ceph-mon[128968]: pgmap v752: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 MiB/s wr, 3.57k op/s 2023-07-06T20:33:32.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:31 smithi156 ceph-mon[129280]: pgmap v753: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 2.93k op/s 2023-07-06T20:33:32.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:31 smithi098 ceph-mon[128968]: pgmap v753: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 2.93k op/s 2023-07-06T20:33:32.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:31 smithi069 ceph-mon[126399]: pgmap v753: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 2.93k op/s 2023-07-06T20:33:33.197 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:33:33.685 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.681+0000 7fd7b9bff700 1 -- 172.21.15.69:0/4278932313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7b4071010 msgr2=0x7fd7b4071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:33.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.681+0000 7fd7b9bff700 1 --2- 172.21.15.69:0/4278932313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7b4071010 0x7fd7b4071420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd7a401a9a0 tx=0x7fd7a401acb0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.681+0000 7fd7b9bff700 1 -- 172.21.15.69:0/4278932313 shutdown_connections 2023-07-06T20:33:33.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.681+0000 7fd7b9bff700 1 --2- 172.21.15.69:0/4278932313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7b410be90 0x7fd7b410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.681+0000 7fd7b9bff700 1 --2- 172.21.15.69:0/4278932313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7b4071010 0x7fd7b4071420 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.681+0000 7fd7b9bff700 1 --2- 172.21.15.69:0/4278932313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7b4071960 0x7fd7b410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.686 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.681+0000 7fd7b9bff700 1 -- 172.21.15.69:0/4278932313 >> 172.21.15.69:0/4278932313 conn(0x7fd7b406cbc0 msgr2=0x7fd7b406eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:33:33.687 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.683+0000 7fd7b9bff700 1 -- 172.21.15.69:0/4278932313 shutdown_connections 2023-07-06T20:33:33.687 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.683+0000 7fd7b9bff700 1 -- 172.21.15.69:0/4278932313 wait complete. 2023-07-06T20:33:33.687 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.685+0000 7fd7b9bff700 1 Processor -- start 2023-07-06T20:33:33.688 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.685+0000 7fd7b9bff700 1 -- start start 2023-07-06T20:33:33.688 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.685+0000 7fd7b9bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7b4071010 0x7fd7b41c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.685+0000 7fd7b9bff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7b4071960 0x7fd7b41c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.685+0000 7fd7b9bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7b410be90 0x7fd7b41c5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.685+0000 7fd7b9bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b41c5be0 con 0x7fd7b410be90 2023-07-06T20:33:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.685+0000 7fd7b9bff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b41c5d50 con 0x7fd7b4071960 2023-07-06T20:33:33.689 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.685+0000 7fd7b9bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b41c5f10 con 0x7fd7b4071010 2023-07-06T20:33:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.686+0000 7fd7b37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7b4071010 0x7fd7b41c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.686+0000 7fd7b37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7b4071010 0x7fd7b41c08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:58934/0 (socket says 172.21.15.69:58934) 2023-07-06T20:33:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.686+0000 7fd7b37fe700 1 -- 172.21.15.69:0/3754419467 learned_addr learned my addr 172.21.15.69:0/3754419467 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:33:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.686+0000 7fd7b2ffd700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7b4071960 0x7fd7b41c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.686+0000 7fd7b3fff700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7b410be90 0x7fd7b41c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.686+0000 7fd7b37fe700 1 -- 172.21.15.69:0/3754419467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7b4071960 msgr2=0x7fd7b41c2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:33.690 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.686+0000 7fd7b37fe700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7b4071960 0x7fd7b41c2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.686+0000 7fd7b37fe700 1 -- 172.21.15.69:0/3754419467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7b410be90 msgr2=0x7fd7b41c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.686+0000 7fd7b37fe700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7b410be90 0x7fd7b41c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.686+0000 7fd7b37fe700 1 -- 172.21.15.69:0/3754419467 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a401a600 con 0x7fd7b4071010 2023-07-06T20:33:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.686+0000 7fd7b3fff700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7b410be90 0x7fd7b41c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T20:33:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.686+0000 7fd7b37fe700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7b4071010 0x7fd7b41c08d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4003710 tx=0x7fd7a4003740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:33:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.687+0000 7fd7b0ff9700 1 -- 172.21.15.69:0/3754419467 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4004150 con 0x7fd7b4071010 2023-07-06T20:33:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.687+0000 7fd7b0ff9700 1 -- 172.21.15.69:0/3754419467 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7a40042b0 con 0x7fd7b4071010 2023-07-06T20:33:33.691 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.687+0000 7fd7b0ff9700 1 -- 172.21.15.69:0/3754419467 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a401f720 con 0x7fd7b4071010 2023-07-06T20:33:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.687+0000 7fd7b9bff700 1 -- 172.21.15.69:0/3754419467 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b41c6190 con 0x7fd7b4071010 2023-07-06T20:33:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.687+0000 7fd7b9bff700 1 -- 172.21.15.69:0/3754419467 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b41c65e0 con 0x7fd7b4071010 2023-07-06T20:33:33.692 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.690+0000 7fd7b0ff9700 1 -- 172.21.15.69:0/3754419467 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd7a400fa40 con 0x7fd7b4071010 2023-07-06T20:33:33.693 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.691+0000 7fd7cd5b1b80 1 Processor -- start 2023-07-06T20:33:33.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.691+0000 7fd7cd5b1b80 1 -- start start 2023-07-06T20:33:33.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.692+0000 7fd7cd5b1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764cfe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.692+0000 7fd7cd5b1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 0x55d764cfee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.692+0000 7fd7cd5b1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 0x55d764cff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.694 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.692+0000 7fd7cd5b1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d764c02cb0 con 0x55d764cfdf20 2023-07-06T20:33:33.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.692+0000 7fd7cd5b1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d764c55a40 con 0x55d764cfe870 2023-07-06T20:33:33.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.692+0000 7fd7cd5b1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d764c4eed0 con 0x55d764cff390 2023-07-06T20:33:33.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.692+0000 7fd7b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764cfe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.695 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.692+0000 7fd7b2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 0x55d764cff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.692+0000 7fd7b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764cfe330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51490/0 (socket says 172.21.15.69:51490) 2023-07-06T20:33:33.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.692+0000 7fd7b3fff700 1 -- 172.21.15.69:0/3280657713 learned_addr learned my addr 172.21.15.69:0/3280657713 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:33:33.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.693+0000 7fd7b2ffd700 1 -- 172.21.15.69:0/3280657713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 msgr2=0x55d764cfee00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:33:33.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.693+0000 7fd7b2ffd700 1 --2- 172.21.15.69:0/3280657713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 0x55d764cfee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.693+0000 7fd7b2ffd700 1 -- 172.21.15.69:0/3280657713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 msgr2=0x55d764cfe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:33:33.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.693+0000 7fd7b2ffd700 1 --2- 172.21.15.69:0/3280657713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764cfe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.696 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.693+0000 7fd7b2ffd700 1 -- 172.21.15.69:0/3280657713 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d764a7cde0 con 0x55d764cff390 2023-07-06T20:33:33.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.693+0000 7fd7b0ff9700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd79c0a1220 0x7fd79c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.693+0000 7fd7b0ff9700 1 -- 172.21.15.69:0/3754419467 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd7a40ce2a0 con 0x7fd7b4071010 2023-07-06T20:33:33.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.693+0000 7fd7b2ffd700 1 --2- 172.21.15.69:0/3280657713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 0x55d764cff830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4072670 tx=0x7fd7a800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24c4155e779e31d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:33:33.697 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.693+0000 7fd7b3fff700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd79c0a1220 0x7fd79c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.698 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.693+0000 7fd799ffb700 1 -- 172.21.15.69:0/3280657713 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8011590 con 0x55d764cff390 2023-07-06T20:33:33.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.694+0000 7fd799ffb700 1 -- 172.21.15.69:0/3280657713 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7a8011bb0 con 0x55d764cff390 2023-07-06T20:33:33.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.694+0000 7fd799ffb700 1 -- 172.21.15.69:0/3280657713 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8012cf0 con 0x55d764cff390 2023-07-06T20:33:33.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.694+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3280657713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 msgr2=0x55d764cff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:33.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.694+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/3280657713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 0x55d764cff830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4072670 tx=0x7fd7a800ba90 comp rx=0 tx=0).stop 2023-07-06T20:33:33.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.694+0000 7fd7b3fff700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd79c0a1220 0x7fd79c0a36d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00f4c0 tx=0x7fd7ac007800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:33:33.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.694+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3280657713 shutdown_connections 2023-07-06T20:33:33.699 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.694+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/3280657713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 0x55d764cff830 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.694+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/3280657713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764cfe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.694+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/3280657713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 0x55d764cfee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.695+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3280657713 >> 172.21.15.69:0/3280657713 conn(0x55d764cf3640 msgr2=0x55d764cf89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:33:33.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.695+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3280657713 shutdown_connections 2023-07-06T20:33:33.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.695+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3280657713 wait complete. 2023-07-06T20:33:33.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.695+0000 7fd7cd5b1b80 1 -- start start 2023-07-06T20:33:33.700 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.695+0000 7fd7cd5b1b80 1 Processor -- start 2023-07-06T20:33:33.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.696+0000 7fd7cd5b1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764d078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.696+0000 7fd7cd5b1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 0x55d764d07e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.696+0000 7fd7cd5b1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 0x55d764d0ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.696+0000 7fd7cd5b1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d764beeac0 con 0x55d764cfdf20 2023-07-06T20:33:33.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.696+0000 7fd7cd5b1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d764bee7c0 con 0x55d764cfe870 2023-07-06T20:33:33.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.696+0000 7fd7cd5b1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d764ad2920 con 0x55d764cff390 2023-07-06T20:33:33.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.696+0000 7fd7cd5b1b80 10 client.0 _ll_register_callbacks cb 0x55d764bec760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:33:33.701 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.696+0000 7fd7b37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764d078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.696+0000 7fd7b2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 0x55d764d07e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.696+0000 7fd7b2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 0x55d764d07e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:42186/0 (socket says 172.21.15.69:42186) 2023-07-06T20:33:33.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.696+0000 7fd7b2ffd700 1 -- 172.21.15.69:0/425509438 learned_addr learned my addr 172.21.15.69:0/425509438 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:33:33.702 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.696+0000 7fd7b3fff700 1 --2- 172.21.15.69:0/425509438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 0x55d764d0ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd7b37fe700 1 -- 172.21.15.69:0/425509438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 msgr2=0x55d764d0ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:33:33.703 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd7b37fe700 1 --2- 172.21.15.69:0/425509438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 0x55d764d0ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd7b37fe700 1 -- 172.21.15.69:0/425509438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 msgr2=0x55d764d07e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:33:33.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd7b37fe700 1 --2- 172.21.15.69:0/425509438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 0x55d764d07e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd7b37fe700 1 -- 172.21.15.69:0/425509438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a800e040 con 0x55d764cfdf20 2023-07-06T20:33:33.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd7b37fe700 1 --2- 172.21.15.69:0/425509438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764d078d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd7a401c5d0 tx=0x7fd7a409b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c201331ab182aca6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:33:33.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd792ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-07-06T20:33:33.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd792ffd700 1 -- 172.21.15.69:0/425509438 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a409b100 con 0x55d764cfdf20 2023-07-06T20:33:33.704 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd792ffd700 1 -- 172.21.15.69:0/425509438 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd7a400d340 con 0x55d764cfdf20 2023-07-06T20:33:33.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/425509438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d764ba84f0 con 0x55d764cfdf20 2023-07-06T20:33:33.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd792ffd700 1 -- 172.21.15.69:0/425509438 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a409b7a0 con 0x55d764cfdf20 2023-07-06T20:33:33.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd7cd5b1b80 10 client.15675 fetch_fsmap learned FSMap version 30 2023-07-06T20:33:33.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.697+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/425509438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d764a65c60 con 0x55d764cfdf20 2023-07-06T20:33:33.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.698+0000 7fd792ffd700 1 -- 172.21.15.69:0/425509438 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d764ba84f0 con 0x55d764cfdf20 2023-07-06T20:33:33.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.698+0000 7fd792ffd700 1 -- 172.21.15.69:0/425509438 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd7a40a36d0 con 0x55d764cfdf20 2023-07-06T20:33:33.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.698+0000 7fd7cd5b1b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:33:33.705 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.698+0000 7fd7cd5b1b80 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:33:33.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.698+0000 7fd7cd5b1b80 20 client.15675 populate_metadata read hostname 'smithi069' 2023-07-06T20:33:33.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.698+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/425509438 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d764d132e0 0x55d764d156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.698+0000 7fd7cd5b1b80 4 client.15675 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:33:33.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.698+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/425509438 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d764cccd60 con 0x55d764d132e0 2023-07-06T20:33:33.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.700+0000 7fd7b2ffd700 1 --2- 172.21.15.69:0/425509438 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d764d132e0 0x55d764d156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.700+0000 7fd7b2ffd700 1 --2- 172.21.15.69:0/425509438 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d764d132e0 0x55d764d156c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adade24c2f130d4f server_cookie=99602e498d7f669d in_seq=0 out_seq=0 2023-07-06T20:33:33.706 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.700+0000 7fd792ffd700 10 client.15675 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:33:33.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.701+0000 7fd792ffd700 1 -- 172.21.15.69:0/425509438 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d764cccd60 con 0x55d764d132e0 2023-07-06T20:33:33.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.701+0000 7fd792ffd700 10 client.15675 handle_command_reply: tid=0 2023-07-06T20:33:33.707 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd7cd5b1b80 1 client.15675 shutdown 2023-07-06T20:33:33.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/425509438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 msgr2=0x55d764d078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:33.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/425509438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764d078d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd7a401c5d0 tx=0x7fd7a409b5f0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/425509438 shutdown_connections 2023-07-06T20:33:33.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/425509438 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d764d132e0 0x55d764d156c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/425509438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 0x55d764d0ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd792ffd700 0 client.15675 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:33:33.708 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/425509438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764d078d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/425509438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 0x55d764d07e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/425509438 >> 172.21.15.69:0/425509438 conn(0x55d764cf3640 msgr2=0x55d764cf80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:33:33.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/425509438 shutdown_connections 2023-07-06T20:33:33.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/425509438 wait complete. 2023-07-06T20:33:33.709 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.702+0000 7fd7cd5b1b80 20 client.15675 trim_cache size 0 max 16384 2023-07-06T20:33:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.727+0000 7fd7cd5b1b80 1 Processor -- start 2023-07-06T20:33:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.727+0000 7fd7cd5b1b80 1 -- start start 2023-07-06T20:33:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.728+0000 7fd7cd5b1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764d02290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.728+0000 7fd7cd5b1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 0x55d764d0ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.728+0000 7fd7cd5b1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 0x55d764d027d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.728+0000 7fd7cd5b1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d764cf3210 con 0x55d764cfdf20 2023-07-06T20:33:33.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.728+0000 7fd7cd5b1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d764d06d80 con 0x55d764cfe870 2023-07-06T20:33:33.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.728+0000 7fd7cd5b1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d764cfec40 con 0x55d764cff390 2023-07-06T20:33:33.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.728+0000 7fd7b37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764d02290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.728+0000 7fd7b2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 0x55d764d0ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.728+0000 7fd7b37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764d02290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51508/0 (socket says 172.21.15.69:51508) 2023-07-06T20:33:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.728+0000 7fd7b37fe700 1 -- 172.21.15.69:0/1519946538 learned_addr learned my addr 172.21.15.69:0/1519946538 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:33:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.729+0000 7fd7b37fe700 1 -- 172.21.15.69:0/1519946538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 msgr2=0x55d764d027d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:33:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.729+0000 7fd7b37fe700 1 --2- 172.21.15.69:0/1519946538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 0x55d764d027d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.729+0000 7fd7b37fe700 1 -- 172.21.15.69:0/1519946538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 msgr2=0x55d764d0ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:33:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.729+0000 7fd7b37fe700 1 --2- 172.21.15.69:0/1519946538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 0x55d764d0ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.729+0000 7fd7b37fe700 1 -- 172.21.15.69:0/1519946538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d764cffc80 con 0x55d764cfdf20 2023-07-06T20:33:33.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.729+0000 7fd7b37fe700 1 --2- 172.21.15.69:0/1519946538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764d02290 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd7a409b730 tx=0x7fd7a4099d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bde4860cd4cfa77 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:33:33.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.729+0000 7fd79a7fc700 1 -- 172.21.15.69:0/1519946538 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a409b100 con 0x55d764cfdf20 2023-07-06T20:33:33.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.729+0000 7fd79a7fc700 1 -- 172.21.15.69:0/1519946538 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7a400d340 con 0x55d764cfdf20 2023-07-06T20:33:33.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.729+0000 7fd79a7fc700 1 -- 172.21.15.69:0/1519946538 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a409a850 con 0x55d764cfdf20 2023-07-06T20:33:33.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.730+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/1519946538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 msgr2=0x55d764d02290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:33.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.730+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/1519946538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764d02290 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd7a409b730 tx=0x7fd7a4099d50 comp rx=0 tx=0).stop 2023-07-06T20:33:33.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.732+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/1519946538 shutdown_connections 2023-07-06T20:33:33.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.732+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/1519946538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cff390 0x55d764d027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.732+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/1519946538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cfdf20 0x55d764d02290 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.732+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/1519946538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfe870 0x55d764d0ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.732+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/1519946538 >> 172.21.15.69:0/1519946538 conn(0x55d764cf3640 msgr2=0x55d764d065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:33:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.733+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/1519946538 shutdown_connections 2023-07-06T20:33:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.733+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/1519946538 wait complete. 2023-07-06T20:33:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.733+0000 7fd7cd5b1b80 1 -- start start 2023-07-06T20:33:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.734+0000 7fd7cd5b1b80 1 Processor -- start 2023-07-06T20:33:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.734+0000 7fd7cd5b1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfdf20 0x55d764d0af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.734+0000 7fd7cd5b1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cfe870 0x55d764d0b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.734+0000 7fd7cd5b1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cff390 0x55d764d0e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.734+0000 7fd7cd5b1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d764d0ea10 con 0x55d764cff390 2023-07-06T20:33:33.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.734+0000 7fd7cd5b1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d764d0eb80 con 0x55d764cfdf20 2023-07-06T20:33:33.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.734+0000 7fd7cd5b1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d764d0ed40 con 0x55d764cfe870 2023-07-06T20:33:33.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.734+0000 7fd7cd5b1b80 10 client.0 _ll_register_callbacks cb 0x55d764bec760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:33:33.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.735+0000 7fd7b2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cfe870 0x55d764d0b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.735+0000 7fd7b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cff390 0x55d764d0e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.735+0000 7fd7b37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfdf20 0x55d764d0af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.735+0000 7fd7b2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cfe870 0x55d764d0b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:58966/0 (socket says 172.21.15.69:58966) 2023-07-06T20:33:33.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.735+0000 7fd7b2ffd700 1 -- 172.21.15.69:0/3836884784 learned_addr learned my addr 172.21.15.69:0/3836884784 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:33:33.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.735+0000 7fd7b2ffd700 1 -- 172.21.15.69:0/3836884784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfdf20 msgr2=0x55d764d0af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:33:33.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.735+0000 7fd7b2ffd700 1 --2- 172.21.15.69:0/3836884784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfdf20 0x55d764d0af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.735+0000 7fd7b2ffd700 1 -- 172.21.15.69:0/3836884784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cff390 msgr2=0x55d764d0e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:33:33.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.735+0000 7fd7b2ffd700 1 --2- 172.21.15.69:0/3836884784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cff390 0x55d764d0e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.735+0000 7fd7b2ffd700 1 -- 172.21.15.69:0/3836884784 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d764a65c60 con 0x55d764cfe870 2023-07-06T20:33:33.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.736+0000 7fd7b2ffd700 1 --2- 172.21.15.69:0/3836884784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cfe870 0x55d764d0b4b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8006e70 tx=0x7fd7a8012540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=231919117d15dadc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:33:33.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.736+0000 7fd792ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:33:33.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.736+0000 7fd792ffd700 1 -- 172.21.15.69:0/3836884784 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8024480 con 0x55d764cfe870 2023-07-06T20:33:33.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.736+0000 7fd792ffd700 1 -- 172.21.15.69:0/3836884784 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd7a8022050 con 0x55d764cfe870 2023-07-06T20:33:33.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.736+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3836884784 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7a8011bb0 con 0x55d764cfe870 2023-07-06T20:33:33.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.737+0000 7fd792ffd700 1 -- 172.21.15.69:0/3836884784 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a80245e0 con 0x55d764cfe870 2023-07-06T20:33:33.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.738+0000 7fd7cd5b1b80 10 client.25801 fetch_fsmap learned FSMap version 30 2023-07-06T20:33:33.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.738+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3836884784 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d764aa3250 con 0x55d764cfe870 2023-07-06T20:33:33.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.738+0000 7fd792ffd700 1 -- 172.21.15.69:0/3836884784 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7a8011bb0 con 0x55d764cfe870 2023-07-06T20:33:33.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.739+0000 7fd792ffd700 1 -- 172.21.15.69:0/3836884784 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd7a802c030 con 0x55d764cfe870 2023-07-06T20:33:33.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.739+0000 7fd7cd5b1b80 10 client.25801 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:33:33.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.739+0000 7fd7cd5b1b80 10 client.25801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:33:33.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.739+0000 7fd7cd5b1b80 20 client.25801 populate_metadata read hostname 'smithi069' 2023-07-06T20:33:33.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.739+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/3836884784 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d764dac420 0x55d764dae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:33:33.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.739+0000 7fd7cd5b1b80 4 client.25801 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:33:33.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.739+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3836884784 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d764c47eb0 con 0x55d764dac420 2023-07-06T20:33:33.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.740+0000 7fd7b37fe700 1 --2- 172.21.15.69:0/3836884784 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d764dac420 0x55d764dae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:33:33.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.742+0000 7fd7b37fe700 1 --2- 172.21.15.69:0/3836884784 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d764dac420 0x55d764dae830 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f363c9706b9cec server_cookie=e92349e8730a1ea in_seq=0 out_seq=0 2023-07-06T20:33:33.756 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.754+0000 7fd792ffd700 10 client.25801 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:33:33.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.771+0000 7fd792ffd700 1 -- 172.21.15.69:0/3836884784 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d764c47eb0 con 0x55d764dac420 2023-07-06T20:33:33.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.771+0000 7fd792ffd700 10 client.25801 handle_command_reply: tid=0 2023-07-06T20:33:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.771+0000 7fd7cd5b1b80 1 client.25801 shutdown 2023-07-06T20:33:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.771+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3836884784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cfe870 msgr2=0x55d764d0b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.771+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/3836884784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cfe870 0x55d764d0b4b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8006e70 tx=0x7fd7a8012540 comp rx=0 tx=0).stop 2023-07-06T20:33:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.771+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3836884784 shutdown_connections 2023-07-06T20:33:33.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.771+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/3836884784 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55d764dac420 0x55d764dae830 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.771+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/3836884784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d764cff390 0x55d764d0e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.771+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/3836884784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d764cfdf20 0x55d764d0af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.771+0000 7fd7cd5b1b80 1 --2- 172.21.15.69:0/3836884784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d764cfe870 0x55d764d0b4b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.771+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3836884784 >> 172.21.15.69:0/3836884784 conn(0x55d764cf3640 msgr2=0x55d764d02290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:33:33.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.772+0000 7fd792ffd700 0 client.25801 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:33:33.776 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:33:33.776 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:33:33.776 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:33:33.776 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:33:33.776 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:33:33.776 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:33:33.777 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:33:33.777 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:33:33.777 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:33:33.777 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:33:33.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.772+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3836884784 shutdown_connections 2023-07-06T20:33:33.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.772+0000 7fd7cd5b1b80 1 -- 172.21.15.69:0/3836884784 wait complete. 2023-07-06T20:33:33.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.772+0000 7fd7cd5b1b80 20 client.25801 trim_cache size 0 max 16384 2023-07-06T20:33:33.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.773+0000 7fd7b9bff700 1 -- 172.21.15.69:0/3754419467 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd79c0a1220 msgr2=0x7fd79c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:33.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.773+0000 7fd7b9bff700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd79c0a1220 0x7fd79c0a36d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00f4c0 tx=0x7fd7ac007800 comp rx=0 tx=0).stop 2023-07-06T20:33:33.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.773+0000 7fd7b9bff700 1 -- 172.21.15.69:0/3754419467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7b4071010 msgr2=0x7fd7b41c08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:33:33.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.773+0000 7fd7b9bff700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7b4071010 0x7fd7b41c08d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4003710 tx=0x7fd7a4003740 comp rx=0 tx=0).stop 2023-07-06T20:33:33.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.775+0000 7fd7b9bff700 1 -- 172.21.15.69:0/3754419467 shutdown_connections 2023-07-06T20:33:33.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.775+0000 7fd7b9bff700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd79c0a1220 0x7fd79c0a36d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.775+0000 7fd7b9bff700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7b410be90 0x7fd7b41c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.775+0000 7fd7b9bff700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7b4071010 0x7fd7b41c08d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.775+0000 7fd7b9bff700 1 --2- 172.21.15.69:0/3754419467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7b4071960 0x7fd7b41c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:33:33.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.775+0000 7fd7b9bff700 1 -- 172.21.15.69:0/3754419467 >> 172.21.15.69:0/3754419467 conn(0x7fd7b406cbc0 msgr2=0x7fd7b406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:33:33.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.799+0000 7fd7b9bff700 1 -- 172.21.15.69:0/3754419467 shutdown_connections 2023-07-06T20:33:33.810 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:33:33.807+0000 7fd7b9bff700 1 -- 172.21.15.69:0/3754419467 wait complete. 2023-07-06T20:33:33.851 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:33:34.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:33 smithi156 ceph-mon[129280]: pgmap v754: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 2.45k op/s 2023-07-06T20:33:34.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:33 smithi098 ceph-mon[128968]: pgmap v754: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 2.45k op/s 2023-07-06T20:33:34.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 705.0 s, 1602.9 tps, lat 62.140 ms stddev 93.948 2023-07-06T20:33:34.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:33 smithi069 ceph-mon[126399]: pgmap v754: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 2.45k op/s 2023-07-06T20:33:36.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:35 smithi069 ceph-mon[126399]: pgmap v755: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.45k op/s 2023-07-06T20:33:36.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:35 smithi156 ceph-mon[129280]: pgmap v755: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.45k op/s 2023-07-06T20:33:36.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:35 smithi098 ceph-mon[128968]: pgmap v755: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.45k op/s 2023-07-06T20:33:38.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:37 smithi069 ceph-mon[126399]: pgmap v756: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 700 op/s 2023-07-06T20:33:38.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:37 smithi156 ceph-mon[129280]: pgmap v756: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 700 op/s 2023-07-06T20:33:38.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:37 smithi098 ceph-mon[128968]: pgmap v756: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 700 op/s 2023-07-06T20:33:39.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 710.0 s, 2035.9 tps, lat 49.671 ms stddev 35.978 2023-07-06T20:33:40.283 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:39 smithi069 ceph-mon[126399]: pgmap v757: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 761 op/s 2023-07-06T20:33:40.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:39 smithi156 ceph-mon[129280]: pgmap v757: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 761 op/s 2023-07-06T20:33:40.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:39 smithi098 ceph-mon[128968]: pgmap v757: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 761 op/s 2023-07-06T20:33:42.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:41 smithi069 ceph-mon[126399]: pgmap v758: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 20 MiB/s wr, 589 op/s 2023-07-06T20:33:42.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:41 smithi156 ceph-mon[129280]: pgmap v758: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 20 MiB/s wr, 589 op/s 2023-07-06T20:33:42.378 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:41 smithi098 ceph-mon[128968]: pgmap v758: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 20 MiB/s wr, 589 op/s 2023-07-06T20:33:44.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 715.0 s, 1463.1 tps, lat 67.634 ms stddev 96.290 2023-07-06T20:33:44.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:44 smithi156 ceph-mon[129280]: pgmap v759: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 659 op/s 2023-07-06T20:33:44.380 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:44 smithi098 ceph-mon[128968]: pgmap v759: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 659 op/s 2023-07-06T20:33:44.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:44 smithi069 ceph-mon[126399]: pgmap v759: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 659 op/s 2023-07-06T20:33:46.358 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:46 smithi156 ceph-mon[129280]: pgmap v760: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 26 MiB/s wr, 994 op/s 2023-07-06T20:33:46.387 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:46 smithi098 ceph-mon[128968]: pgmap v760: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 26 MiB/s wr, 994 op/s 2023-07-06T20:33:46.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:46 smithi069 ceph-mon[126399]: pgmap v760: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 26 MiB/s wr, 994 op/s 2023-07-06T20:33:48.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:48 smithi069 ceph-mon[126399]: pgmap v761: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 25 MiB/s wr, 1.27k op/s 2023-07-06T20:33:48.612 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:48 smithi156 ceph-mon[129280]: pgmap v761: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 25 MiB/s wr, 1.27k op/s 2023-07-06T20:33:48.631 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:48 smithi098 ceph-mon[128968]: pgmap v761: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 25 MiB/s wr, 1.27k op/s 2023-07-06T20:33:49.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 720.0 s, 880.2 tps, lat 114.315 ms stddev 165.182 2023-07-06T20:33:50.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:50 smithi069 ceph-mon[126399]: pgmap v762: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 34 MiB/s wr, 2.35k op/s 2023-07-06T20:33:50.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:50 smithi156 ceph-mon[129280]: pgmap v762: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 34 MiB/s wr, 2.35k op/s 2023-07-06T20:33:50.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:50 smithi098 ceph-mon[128968]: pgmap v762: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 34 MiB/s wr, 2.35k op/s 2023-07-06T20:33:52.615 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:52 smithi156 ceph-mon[129280]: pgmap v763: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 MiB/s wr, 2.87k op/s 2023-07-06T20:33:52.630 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:52 smithi098 ceph-mon[128968]: pgmap v763: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 MiB/s wr, 2.87k op/s 2023-07-06T20:33:52.765 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:52 smithi069 ceph-mon[126399]: pgmap v763: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 MiB/s wr, 2.87k op/s 2023-07-06T20:33:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 725.0 s, 1528.2 tps, lat 65.734 ms stddev 140.103 2023-07-06T20:33:54.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:54 smithi069 ceph-mon[126399]: pgmap v764: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 MiB/s wr, 3.24k op/s 2023-07-06T20:33:54.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:54 smithi156 ceph-mon[129280]: pgmap v764: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 MiB/s wr, 3.24k op/s 2023-07-06T20:33:54.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:54 smithi098 ceph-mon[128968]: pgmap v764: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 42 MiB/s wr, 3.24k op/s 2023-07-06T20:33:56.764 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:56 smithi069 ceph-mon[126399]: pgmap v765: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 MiB/s wr, 3.55k op/s 2023-07-06T20:33:56.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:56 smithi156 ceph-mon[129280]: pgmap v765: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 MiB/s wr, 3.55k op/s 2023-07-06T20:33:56.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:56 smithi098 ceph-mon[128968]: pgmap v765: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 MiB/s wr, 3.55k op/s 2023-07-06T20:33:58.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:33:58 smithi069 ceph-mon[126399]: pgmap v766: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 38 MiB/s wr, 3.29k op/s 2023-07-06T20:33:58.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:33:58 smithi156 ceph-mon[129280]: pgmap v766: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 38 MiB/s wr, 3.29k op/s 2023-07-06T20:33:58.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:33:58 smithi098 ceph-mon[128968]: pgmap v766: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 38 MiB/s wr, 3.29k op/s 2023-07-06T20:33:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 730.0 s, 2087.9 tps, lat 47.491 ms stddev 32.308 2023-07-06T20:34:00.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:00 smithi069 ceph-mon[126399]: pgmap v767: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 39 MiB/s wr, 3.16k op/s 2023-07-06T20:34:00.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:00 smithi156 ceph-mon[129280]: pgmap v767: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 39 MiB/s wr, 3.16k op/s 2023-07-06T20:34:00.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:00 smithi098 ceph-mon[128968]: pgmap v767: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 39 MiB/s wr, 3.16k op/s 2023-07-06T20:34:01.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:01 smithi069 ceph-mon[126399]: pgmap v768: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 2.15k op/s 2023-07-06T20:34:01.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:01 smithi156 ceph-mon[129280]: pgmap v768: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 2.15k op/s 2023-07-06T20:34:01.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:01 smithi098 ceph-mon[128968]: pgmap v768: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 2.15k op/s 2023-07-06T20:34:03.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:03 smithi069 ceph-mon[126399]: pgmap v769: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 MiB/s wr, 1.58k op/s 2023-07-06T20:34:03.852 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:34:03.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:03 smithi156 ceph-mon[129280]: pgmap v769: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 MiB/s wr, 1.58k op/s 2023-07-06T20:34:03.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:03 smithi098 ceph-mon[128968]: pgmap v769: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 MiB/s wr, 1.58k op/s 2023-07-06T20:34:04.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.229+0000 7f6595fff700 1 -- 172.21.15.69:0/4171267971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6590077340 msgr2=0x7f65900797c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.229+0000 7f6595fff700 1 --2- 172.21.15.69:0/4171267971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6590077340 0x7f65900797c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6584009990 tx=0x7f6584009ca0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.232 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 735.0 s, 1408.7 tps, lat 70.127 ms stddev 73.487 2023-07-06T20:34:04.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.229+0000 7f6595fff700 1 -- 172.21.15.69:0/4171267971 shutdown_connections 2023-07-06T20:34:04.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.229+0000 7f6595fff700 1 --2- 172.21.15.69:0/4171267971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6590077340 0x7f65900797c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.229+0000 7f6595fff700 1 --2- 172.21.15.69:0/4171267971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6590071330 0x7f6590071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.229+0000 7f6595fff700 1 --2- 172.21.15.69:0/4171267971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6590074980 0x7f6590076e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.229+0000 7f6595fff700 1 -- 172.21.15.69:0/4171267971 >> 172.21.15.69:0/4171267971 conn(0x7f659006cbc0 msgr2=0x7f659006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:34:04.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.230+0000 7f6595fff700 1 -- 172.21.15.69:0/4171267971 shutdown_connections 2023-07-06T20:34:04.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.230+0000 7f6595fff700 1 -- 172.21.15.69:0/4171267971 wait complete. 2023-07-06T20:34:04.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.232+0000 7f6595fff700 1 Processor -- start 2023-07-06T20:34:04.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.232+0000 7f6595fff700 1 -- start start 2023-07-06T20:34:04.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.232+0000 7f6595fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6590071330 0x7f6590082410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.232+0000 7f6595fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6590074980 0x7f6590082970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.232+0000 7f6595fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6590077340 0x7f65901b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.232+0000 7f6595fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65901b4a90 con 0x7f6590071330 2023-07-06T20:34:04.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.232+0000 7f6595fff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65901b4c00 con 0x7f6590077340 2023-07-06T20:34:04.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.232+0000 7f6595fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65901b4dc0 con 0x7f6590074980 2023-07-06T20:34:04.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.233+0000 7f658effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6590074980 0x7f6590082970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:04.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.233+0000 7f658effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6590074980 0x7f6590082970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:37120/0 (socket says 172.21.15.69:37120) 2023-07-06T20:34:04.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.233+0000 7f658effd700 1 -- 172.21.15.69:0/4080305400 learned_addr learned my addr 172.21.15.69:0/4080305400 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:34:04.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.234+0000 7f658effd700 1 -- 172.21.15.69:0/4080305400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6590077340 msgr2=0x7f65901b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:04.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.234+0000 7f658effd700 1 --2- 172.21.15.69:0/4080305400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6590077340 0x7f65901b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.234+0000 7f658effd700 1 -- 172.21.15.69:0/4080305400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6590071330 msgr2=0x7f6590082410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:04.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.234+0000 7f658effd700 1 --2- 172.21.15.69:0/4080305400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6590071330 0x7f6590082410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.234+0000 7f658effd700 1 -- 172.21.15.69:0/4080305400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65840095f0 con 0x7f6590074980 2023-07-06T20:34:04.240 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.234+0000 7f658effd700 1 --2- 172.21.15.69:0/4080305400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6590074980 0x7f6590082970 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f658000c920 tx=0x7f658000cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:34:04.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.236+0000 7f658cff9700 1 -- 172.21.15.69:0/4080305400 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580007b00 con 0x7f6590074980 2023-07-06T20:34:04.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.236+0000 7f658cff9700 1 -- 172.21.15.69:0/4080305400 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6580004cf0 con 0x7f6590074980 2023-07-06T20:34:04.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.236+0000 7f658cff9700 1 -- 172.21.15.69:0/4080305400 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580005530 con 0x7f6590074980 2023-07-06T20:34:04.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.237+0000 7f6595fff700 1 -- 172.21.15.69:0/4080305400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65901b50a0 con 0x7f6590074980 2023-07-06T20:34:04.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.237+0000 7f6595fff700 1 -- 172.21.15.69:0/4080305400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65901b5560 con 0x7f6590074980 2023-07-06T20:34:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.243+0000 7f65a9961b80 1 Processor -- start 2023-07-06T20:34:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.244+0000 7f65a9961b80 1 -- start start 2023-07-06T20:34:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.244+0000 7f65a9961b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc2f20 0x55c73bfc3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.244+0000 7f65a9961b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 0x55c73bfc3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.244+0000 7f65a9961b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc4390 0x55c73bfc4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.244+0000 7f65a9961b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73bec7cb0 con 0x55c73bfc3870 2023-07-06T20:34:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.244+0000 7f65a9961b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73bf1aa40 con 0x55c73bfc2f20 2023-07-06T20:34:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.244+0000 7f65a9961b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73bf13ed0 con 0x55c73bfc4390 2023-07-06T20:34:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.245+0000 7f658ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc2f20 0x55c73bfc3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.245+0000 7f658ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc2f20 0x55c73bfc3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:53512/0 (socket says 172.21.15.69:53512) 2023-07-06T20:34:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.245+0000 7f658ffff700 1 -- 172.21.15.69:0/91052800 learned_addr learned my addr 172.21.15.69:0/91052800 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:34:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.245+0000 7f658ffff700 1 -- 172.21.15.69:0/91052800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc4390 msgr2=0x55c73bfc4830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:34:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.245+0000 7f658ffff700 1 --2- 172.21.15.69:0/91052800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc4390 0x55c73bfc4830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.245+0000 7f658ffff700 1 -- 172.21.15.69:0/91052800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 msgr2=0x55c73bfc3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.245+0000 7f658ffff700 1 --2- 172.21.15.69:0/91052800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 0x55c73bfc3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.245+0000 7f658ffff700 1 -- 172.21.15.69:0/91052800 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c73bd41de0 con 0x55c73bfc2f20 2023-07-06T20:34:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.246+0000 7f658ffff700 1 --2- 172.21.15.69:0/91052800 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc2f20 0x55c73bfc3330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65840095b0 tx=0x7f6584005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e70fdca5e88bf62 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:34:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.247+0000 7f657dffb700 1 -- 172.21.15.69:0/91052800 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658401d070 con 0x55c73bfc2f20 2023-07-06T20:34:04.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.247+0000 7f657dffb700 1 -- 172.21.15.69:0/91052800 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f658400be50 con 0x55c73bfc2f20 2023-07-06T20:34:04.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.247+0000 7f657dffb700 1 -- 172.21.15.69:0/91052800 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584021410 con 0x55c73bfc2f20 2023-07-06T20:34:04.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.247+0000 7f658cff9700 1 -- 172.21.15.69:0/4080305400 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6580004830 con 0x7f6590074980 2023-07-06T20:34:04.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.249+0000 7f65a9961b80 1 -- 172.21.15.69:0/91052800 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc2f20 msgr2=0x55c73bfc3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:04.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.249+0000 7f65a9961b80 1 --2- 172.21.15.69:0/91052800 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc2f20 0x55c73bfc3330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65840095b0 tx=0x7f6584005bc0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.252 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.249+0000 7f65a9961b80 1 -- 172.21.15.69:0/91052800 shutdown_connections 2023-07-06T20:34:04.253 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.249+0000 7f65a9961b80 1 --2- 172.21.15.69:0/91052800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc4390 0x55c73bfc4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.253 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.249+0000 7f65a9961b80 1 --2- 172.21.15.69:0/91052800 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc2f20 0x55c73bfc3330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.253 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.249+0000 7f65a9961b80 1 --2- 172.21.15.69:0/91052800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 0x55c73bfc3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.253 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.249+0000 7f65a9961b80 1 -- 172.21.15.69:0/91052800 >> 172.21.15.69:0/91052800 conn(0x55c73bfb8640 msgr2=0x55c73bfbd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:34:04.254 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.250+0000 7f658cff9700 1 --2- 172.21.15.69:0/4080305400 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f65780a12d0 0x7f65780a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.250+0000 7f658cff9700 1 -- 172.21.15.69:0/4080305400 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f65800cc730 con 0x7f6590074980 2023-07-06T20:34:04.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.251+0000 7f65a9961b80 1 -- 172.21.15.69:0/91052800 shutdown_connections 2023-07-06T20:34:04.255 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.251+0000 7f658f7fe700 1 --2- 172.21.15.69:0/4080305400 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f65780a12d0 0x7f65780a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:04.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.252+0000 7f65a9961b80 1 -- 172.21.15.69:0/91052800 wait complete. 2023-07-06T20:34:04.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.253+0000 7f65a9961b80 1 -- start start 2023-07-06T20:34:04.258 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.253+0000 7f65a9961b80 1 Processor -- start 2023-07-06T20:34:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.257+0000 7f65a9961b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc2f20 0x55c73bfcc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.257+0000 7f65a9961b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc3870 0x55c73bfcce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.257+0000 7f65a9961b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc4390 0x55c73bfd3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.258+0000 7f65a9961b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73beb3ac0 con 0x55c73bfc4390 2023-07-06T20:34:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.258+0000 7f65a9961b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73beb37c0 con 0x55c73bfc3870 2023-07-06T20:34:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.258+0000 7f65a9961b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73bd97920 con 0x55c73bfc2f20 2023-07-06T20:34:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.258+0000 7f65a9961b80 10 client.0 _ll_register_callbacks cb 0x55c73beb1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:34:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.259+0000 7f658f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc2f20 0x55c73bfcc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:04.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.259+0000 7f658f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc2f20 0x55c73bfcc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:37132/0 (socket says 172.21.15.69:37132) 2023-07-06T20:34:04.263 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.259+0000 7f658f7fe700 1 -- 172.21.15.69:0/3655630094 learned_addr learned my addr 172.21.15.69:0/3655630094 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:34:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.262+0000 7f658effd700 1 --2- 172.21.15.69:0/3655630094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc3870 0x55c73bfcce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.262+0000 7f658f7fe700 1 -- 172.21.15.69:0/3655630094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc3870 msgr2=0x55c73bfcce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:04.264 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.262+0000 7f658f7fe700 1 --2- 172.21.15.69:0/3655630094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc3870 0x55c73bfcce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.262+0000 7f658f7fe700 1 -- 172.21.15.69:0/3655630094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc4390 msgr2=0x55c73bfd3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.262+0000 7f658f7fe700 1 --2- 172.21.15.69:0/3655630094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc4390 0x55c73bfd3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.265 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.262+0000 7f658f7fe700 1 -- 172.21.15.69:0/3655630094 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659007a1f0 con 0x55c73bfc2f20 2023-07-06T20:34:04.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.262+0000 7f658f7fe700 1 --2- 172.21.15.69:0/4080305400 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f65780a12d0 0x7f65780a3780 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6590083850 tx=0x7f658800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:34:04.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.262+0000 7f658f7fe700 1 --2- 172.21.15.69:0/3655630094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc2f20 0x55c73bfcc920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65880159e0 tx=0x7f6588015da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bacd974fb3de79ee server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:34:04.266 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.263+0000 7f6572ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:34:04.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.264+0000 7f6572ffd700 1 -- 172.21.15.69:0/3655630094 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658801a070 con 0x55c73bfc2f20 2023-07-06T20:34:04.267 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.264+0000 7f6572ffd700 1 -- 172.21.15.69:0/3655630094 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f65880149b0 con 0x55c73bfc2f20 2023-07-06T20:34:04.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.264+0000 7f65a9961b80 1 -- 172.21.15.69:0/3655630094 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c73be6d4f0 con 0x55c73bfc2f20 2023-07-06T20:34:04.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.265+0000 7f6572ffd700 1 -- 172.21.15.69:0/3655630094 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6588014b70 con 0x55c73bfc2f20 2023-07-06T20:34:04.268 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.265+0000 7f65a9961b80 10 client.25810 fetch_fsmap learned FSMap version 30 2023-07-06T20:34:04.269 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.265+0000 7f65a9961b80 1 -- 172.21.15.69:0/3655630094 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c73bd2ac60 con 0x55c73bfc2f20 2023-07-06T20:34:04.269 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.265+0000 7f6572ffd700 1 -- 172.21.15.69:0/3655630094 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f658802d020 con 0x55c73bfc2f20 2023-07-06T20:34:04.270 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.266+0000 7f6572ffd700 1 -- 172.21.15.69:0/3655630094 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6588027400 con 0x55c73bfc2f20 2023-07-06T20:34:04.270 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.266+0000 7f65a9961b80 10 client.25810 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:34:04.270 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.266+0000 7f65a9961b80 10 client.25810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:34:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.266+0000 7f65a9961b80 20 client.25810 populate_metadata read hostname 'smithi069' 2023-07-06T20:34:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.266+0000 7f65a9961b80 1 --2- 172.21.15.69:0/3655630094 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c73bfd87d0 0x55c73bfdabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.266+0000 7f65a9961b80 4 client.25810 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:34:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.266+0000 7f65a9961b80 1 -- 172.21.15.69:0/3655630094 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c73bf91d60 con 0x55c73bfd87d0 2023-07-06T20:34:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.266+0000 7f658effd700 1 --2- 172.21.15.69:0/3655630094 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c73bfd87d0 0x55c73bfdabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.267+0000 7f6572ffd700 10 client.25810 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:34:04.271 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.267+0000 7f658effd700 1 --2- 172.21.15.69:0/3655630094 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c73bfd87d0 0x55c73bfdabe0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06edec9fc112aae server_cookie=1df4fef20747e855 in_seq=0 out_seq=0 2023-07-06T20:34:04.272 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.268+0000 7f6572ffd700 1 -- 172.21.15.69:0/3655630094 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c73bf91d60 con 0x55c73bfd87d0 2023-07-06T20:34:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.268+0000 7f6572ffd700 10 client.25810 handle_command_reply: tid=0 2023-07-06T20:34:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.268+0000 7f65a9961b80 1 client.25810 shutdown 2023-07-06T20:34:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.268+0000 7f65a9961b80 1 -- 172.21.15.69:0/3655630094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc2f20 msgr2=0x55c73bfcc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:04.273 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.268+0000 7f65a9961b80 1 --2- 172.21.15.69:0/3655630094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc2f20 0x55c73bfcc920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65880159e0 tx=0x7f6588015da0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.269+0000 7f65a9961b80 1 -- 172.21.15.69:0/3655630094 shutdown_connections 2023-07-06T20:34:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.269+0000 7f65a9961b80 1 --2- 172.21.15.69:0/3655630094 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c73bfd87d0 0x55c73bfdabe0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.269+0000 7f65a9961b80 1 --2- 172.21.15.69:0/3655630094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc4390 0x55c73bfd3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.269+0000 7f65a9961b80 1 --2- 172.21.15.69:0/3655630094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc2f20 0x55c73bfcc920 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.269+0000 7f6572ffd700 0 client.25810 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:34:04.274 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.269+0000 7f65a9961b80 1 --2- 172.21.15.69:0/3655630094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc3870 0x55c73bfcce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.269+0000 7f65a9961b80 1 -- 172.21.15.69:0/3655630094 >> 172.21.15.69:0/3655630094 conn(0x55c73bfb8640 msgr2=0x55c73bfbd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:34:04.275 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.269+0000 7f65a9961b80 1 -- 172.21.15.69:0/3655630094 shutdown_connections 2023-07-06T20:34:04.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.269+0000 7f65a9961b80 1 -- 172.21.15.69:0/3655630094 wait complete. 2023-07-06T20:34:04.276 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.269+0000 7f65a9961b80 20 client.25810 trim_cache size 0 max 16384 2023-07-06T20:34:04.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.292+0000 7f65a9961b80 1 Processor -- start 2023-07-06T20:34:04.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.292+0000 7f65a9961b80 1 -- start start 2023-07-06T20:34:04.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.292+0000 7f65a9961b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc2f20 0x55c73bfcb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.292+0000 7f65a9961b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 0x55c73bfd3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.295 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.293+0000 7f65a9961b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc4390 0x55c73bfc7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.293+0000 7f65a9961b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73bfbf460 con 0x55c73bfc3870 2023-07-06T20:34:04.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.293+0000 7f65a9961b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73bfbe6d0 con 0x55c73bfc4390 2023-07-06T20:34:04.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.293+0000 7f65a9961b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73bfc3c40 con 0x55c73bfc2f20 2023-07-06T20:34:04.296 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.293+0000 7f658effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 0x55c73bfd3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:04.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.293+0000 7f658effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 0x55c73bfd3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58458/0 (socket says 172.21.15.69:58458) 2023-07-06T20:34:04.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.293+0000 7f658effd700 1 -- 172.21.15.69:0/2008897381 learned_addr learned my addr 172.21.15.69:0/2008897381 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:34:04.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.294+0000 7f658effd700 1 -- 172.21.15.69:0/2008897381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc2f20 msgr2=0x55c73bfcb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:04.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.294+0000 7f658effd700 1 --2- 172.21.15.69:0/2008897381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc2f20 0x55c73bfcb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.294+0000 7f658effd700 1 -- 172.21.15.69:0/2008897381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc4390 msgr2=0x55c73bfc7330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T20:34:04.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.294+0000 7f658effd700 1 --2- 172.21.15.69:0/2008897381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc4390 0x55c73bfc7330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.297 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.294+0000 7f658effd700 1 -- 172.21.15.69:0/2008897381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c73bd68250 con 0x55c73bfc3870 2023-07-06T20:34:04.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.294+0000 7f658effd700 1 --2- 172.21.15.69:0/2008897381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 0x55c73bfd3e60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f65800ca200 tx=0x7f65800cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad7c1e40a991876 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:34:04.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.295+0000 7f657e7fc700 1 -- 172.21.15.69:0/2008897381 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65800a1f60 con 0x55c73bfc3870 2023-07-06T20:34:04.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.295+0000 7f657e7fc700 1 -- 172.21.15.69:0/2008897381 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65800b9040 con 0x55c73bfc3870 2023-07-06T20:34:04.298 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.295+0000 7f657e7fc700 1 -- 172.21.15.69:0/2008897381 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580097f40 con 0x55c73bfc3870 2023-07-06T20:34:04.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.296+0000 7f65a9961b80 1 -- 172.21.15.69:0/2008897381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 msgr2=0x55c73bfd3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:04.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.296+0000 7f65a9961b80 1 --2- 172.21.15.69:0/2008897381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 0x55c73bfd3e60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f65800ca200 tx=0x7f65800cc040 comp rx=0 tx=0).stop 2023-07-06T20:34:04.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.296+0000 7f65a9961b80 1 -- 172.21.15.69:0/2008897381 shutdown_connections 2023-07-06T20:34:04.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.296+0000 7f65a9961b80 1 --2- 172.21.15.69:0/2008897381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc4390 0x55c73bfc7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.299 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.296+0000 7f65a9961b80 1 --2- 172.21.15.69:0/2008897381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfc2f20 0x55c73bfcb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.296+0000 7f65a9961b80 1 --2- 172.21.15.69:0/2008897381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 0x55c73bfd3e60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.296+0000 7f65a9961b80 1 -- 172.21.15.69:0/2008897381 >> 172.21.15.69:0/2008897381 conn(0x55c73bfb8640 msgr2=0x55c73bfca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:34:04.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.298+0000 7f65a9961b80 1 -- 172.21.15.69:0/2008897381 shutdown_connections 2023-07-06T20:34:04.300 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.298+0000 7f65a9961b80 1 -- 172.21.15.69:0/2008897381 wait complete. 2023-07-06T20:34:04.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.299+0000 7f65a9961b80 1 -- start start 2023-07-06T20:34:04.302 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.299+0000 7f65a9961b80 1 Processor -- start 2023-07-06T20:34:04.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.303+0000 7f65a9961b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 0x55c73bfd3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.303+0000 7f65a9961b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc4390 0x55c73bfd22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.303+0000 7f65a9961b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfd27e0 0x55c73bfd2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.303+0000 7f65a9961b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73c02c680 con 0x55c73bfc3870 2023-07-06T20:34:04.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.303+0000 7f65a9961b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73c02c7c0 con 0x55c73bfc4390 2023-07-06T20:34:04.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.303+0000 7f65a9961b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73c02c900 con 0x55c73bfd27e0 2023-07-06T20:34:04.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.303+0000 7f65a9961b80 10 client.0 _ll_register_callbacks cb 0x7f658800ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:34:04.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.304+0000 7f658effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc4390 0x55c73bfd22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:04.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.304+0000 7f658effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc4390 0x55c73bfd22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:53544/0 (socket says 172.21.15.69:53544) 2023-07-06T20:34:04.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.304+0000 7f658effd700 1 -- 172.21.15.69:0/1271824851 learned_addr learned my addr 172.21.15.69:0/1271824851 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:34:04.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.304+0000 7f658ffff700 1 --2- 172.21.15.69:0/1271824851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfd27e0 0x55c73bfd2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:04.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.305+0000 7f658ffff700 1 -- 172.21.15.69:0/1271824851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc4390 msgr2=0x55c73bfd22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:04.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.305+0000 7f658ffff700 1 --2- 172.21.15.69:0/1271824851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc4390 0x55c73bfd22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.305+0000 7f658ffff700 1 -- 172.21.15.69:0/1271824851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 msgr2=0x55c73bfd3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:04.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.305+0000 7f658ffff700 1 --2- 172.21.15.69:0/1271824851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 0x55c73bfd3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.305+0000 7f658ffff700 1 -- 172.21.15.69:0/1271824851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65901b5560 con 0x55c73bfd27e0 2023-07-06T20:34:04.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.305+0000 7f658ffff700 1 --2- 172.21.15.69:0/1271824851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfd27e0 0x55c73bfd2c50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6584023000 tx=0x7f658400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b751cf01dab35e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:34:04.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.306+0000 7f6572ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:34:04.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.306+0000 7f6572ffd700 1 -- 172.21.15.69:0/1271824851 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658401d050 con 0x55c73bfd27e0 2023-07-06T20:34:04.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.306+0000 7f6572ffd700 1 -- 172.21.15.69:0/1271824851 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f658400fd30 con 0x55c73bfd27e0 2023-07-06T20:34:04.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.306+0000 7f65a9961b80 1 -- 172.21.15.69:0/1271824851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f658400be50 con 0x55c73bfd27e0 2023-07-06T20:34:04.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.307+0000 7f6572ffd700 1 -- 172.21.15.69:0/1271824851 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6584021410 con 0x55c73bfd27e0 2023-07-06T20:34:04.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.307+0000 7f65a9961b80 10 client.25813 fetch_fsmap learned FSMap version 30 2023-07-06T20:34:04.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.307+0000 7f65a9961b80 1 -- 172.21.15.69:0/1271824851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c73bd402b0 con 0x55c73bfd27e0 2023-07-06T20:34:04.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.307+0000 7f6572ffd700 1 -- 172.21.15.69:0/1271824851 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f658400be50 con 0x55c73bfd27e0 2023-07-06T20:34:04.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.307+0000 7f6572ffd700 1 -- 172.21.15.69:0/1271824851 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6584033070 con 0x55c73bfd27e0 2023-07-06T20:34:04.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.307+0000 7f65a9961b80 10 client.25813 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:34:04.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.307+0000 7f65a9961b80 10 client.25813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:34:04.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.307+0000 7f65a9961b80 20 client.25813 populate_metadata read hostname 'smithi069' 2023-07-06T20:34:04.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.308+0000 7f65a9961b80 1 --2- 172.21.15.69:0/1271824851 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c73c02dd20 0x55c73c030130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:04.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.308+0000 7f65a9961b80 4 client.25813 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:34:04.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.308+0000 7f65a9961b80 1 -- 172.21.15.69:0/1271824851 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c73bf0ceb0 con 0x55c73c02dd20 2023-07-06T20:34:04.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.308+0000 7f658f7fe700 1 --2- 172.21.15.69:0/1271824851 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c73c02dd20 0x55c73c030130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:04.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.308+0000 7f658f7fe700 1 --2- 172.21.15.69:0/1271824851 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c73c02dd20 0x55c73c030130 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3423f5e1f938721c server_cookie=5f3457cb4e1133e in_seq=0 out_seq=0 2023-07-06T20:34:04.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.308+0000 7f6572ffd700 10 client.25813 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:34:04.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f6572ffd700 1 -- 172.21.15.69:0/1271824851 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c73bf0ceb0 con 0x55c73c02dd20 2023-07-06T20:34:04.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f6572ffd700 10 client.25813 handle_command_reply: tid=0 2023-07-06T20:34:04.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f65a9961b80 1 client.25813 shutdown 2023-07-06T20:34:04.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f65a9961b80 1 -- 172.21.15.69:0/1271824851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfd27e0 msgr2=0x55c73bfd2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:04.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f65a9961b80 1 --2- 172.21.15.69:0/1271824851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfd27e0 0x55c73bfd2c50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6584023000 tx=0x7f658400f690 comp rx=0 tx=0).stop 2023-07-06T20:34:04.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f65a9961b80 1 -- 172.21.15.69:0/1271824851 shutdown_connections 2023-07-06T20:34:04.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f65a9961b80 1 --2- 172.21.15.69:0/1271824851 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x55c73c02dd20 0x55c73c030130 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f65a9961b80 1 --2- 172.21.15.69:0/1271824851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c73bfd27e0 0x55c73bfd2c50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f65a9961b80 1 --2- 172.21.15.69:0/1271824851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73bfc3870 0x55c73bfd3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f65a9961b80 1 --2- 172.21.15.69:0/1271824851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c73bfc4390 0x55c73bfd22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f65a9961b80 1 -- 172.21.15.69:0/1271824851 >> 172.21.15.69:0/1271824851 conn(0x55c73bfb8640 msgr2=0x55c73bfc7330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:34:04.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f6572ffd700 0 client.25813 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:34:04.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f65a9961b80 1 -- 172.21.15.69:0/1271824851 shutdown_connections 2023-07-06T20:34:04.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f65a9961b80 1 -- 172.21.15.69:0/1271824851 wait complete. 2023-07-06T20:34:04.315 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.309+0000 7f65a9961b80 20 client.25813 trim_cache size 0 max 16384 2023-07-06T20:34:04.315 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:34:04.316 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:34:04.316 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:34:04.316 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:34:04.316 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:34:04.316 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:34:04.316 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:34:04.317 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:34:04.317 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:34:04.317 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:34:04.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.310+0000 7f6595fff700 1 -- 172.21.15.69:0/4080305400 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f65780a12d0 msgr2=0x7f65780a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:04.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.310+0000 7f6595fff700 1 --2- 172.21.15.69:0/4080305400 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f65780a12d0 0x7f65780a3780 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6590083850 tx=0x7f658800b040 comp rx=0 tx=0).stop 2023-07-06T20:34:04.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.310+0000 7f6595fff700 1 -- 172.21.15.69:0/4080305400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6590074980 msgr2=0x7f6590082970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:04.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.310+0000 7f6595fff700 1 --2- 172.21.15.69:0/4080305400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6590074980 0x7f6590082970 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f658000c920 tx=0x7f658000cce0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.310+0000 7f6595fff700 1 -- 172.21.15.69:0/4080305400 shutdown_connections 2023-07-06T20:34:04.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.310+0000 7f6595fff700 1 --2- 172.21.15.69:0/4080305400 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f65780a12d0 0x7f65780a3780 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.310+0000 7f6595fff700 1 --2- 172.21.15.69:0/4080305400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6590077340 0x7f65901b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.310+0000 7f6595fff700 1 --2- 172.21.15.69:0/4080305400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6590071330 0x7f6590082410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.318 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.310+0000 7f6595fff700 1 --2- 172.21.15.69:0/4080305400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6590074980 0x7f6590082970 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:04.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.310+0000 7f6595fff700 1 -- 172.21.15.69:0/4080305400 >> 172.21.15.69:0/4080305400 conn(0x7f659006cbc0 msgr2=0x7f6590076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:34:04.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.310+0000 7f6595fff700 1 -- 172.21.15.69:0/4080305400 shutdown_connections 2023-07-06T20:34:04.319 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:04.310+0000 7f6595fff700 1 -- 172.21.15.69:0/4080305400 wait complete. 2023-07-06T20:34:04.334 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:34:05.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:05 smithi156 ceph-mon[129280]: pgmap v770: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 1.35k op/s 2023-07-06T20:34:05.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:05 smithi098 ceph-mon[128968]: pgmap v770: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 1.35k op/s 2023-07-06T20:34:06.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:05 smithi069 ceph-mon[126399]: pgmap v770: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 1.35k op/s 2023-07-06T20:34:07.877 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:07 smithi098 ceph-mon[128968]: pgmap v771: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 982 op/s 2023-07-06T20:34:08.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:07 smithi069 ceph-mon[126399]: pgmap v771: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 982 op/s 2023-07-06T20:34:08.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:07 smithi156 ceph-mon[129280]: pgmap v771: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 982 op/s 2023-07-06T20:34:09.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 740.0 s, 2028.7 tps, lat 50.148 ms stddev 43.500 2023-07-06T20:34:10.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:09 smithi069 ceph-mon[126399]: pgmap v772: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 884 op/s 2023-07-06T20:34:10.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:09 smithi156 ceph-mon[129280]: pgmap v772: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 884 op/s 2023-07-06T20:34:10.130 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:09 smithi098 ceph-mon[128968]: pgmap v772: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 884 op/s 2023-07-06T20:34:12.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:11 smithi069 ceph-mon[126399]: pgmap v773: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 730 op/s 2023-07-06T20:34:12.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:11 smithi156 ceph-mon[129280]: pgmap v773: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 730 op/s 2023-07-06T20:34:12.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:11 smithi098 ceph-mon[128968]: pgmap v773: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 730 op/s 2023-07-06T20:34:13.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:12 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:34:13.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:12 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:34:13.267 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:12 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:34:14.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:13 smithi156 ceph-mon[129280]: pgmap v774: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 835 op/s 2023-07-06T20:34:14.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:13 smithi098 ceph-mon[128968]: pgmap v774: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 835 op/s 2023-07-06T20:34:14.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 745.0 s, 1049.6 tps, lat 94.505 ms stddev 129.098 2023-07-06T20:34:14.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:13 smithi069 ceph-mon[126399]: pgmap v774: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 835 op/s 2023-07-06T20:34:15.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:15 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:34:15.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:15 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:34:15.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:15 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:34:15.611 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:15 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:34:15.612 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:15 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:34:15.612 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:15 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:34:15.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:15 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:34:15.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:15 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:34:15.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:15 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:34:16.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:16 smithi098 ceph-mon[128968]: pgmap v775: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 1.54k op/s 2023-07-06T20:34:16.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:16 smithi069 ceph-mon[126399]: pgmap v775: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 1.54k op/s 2023-07-06T20:34:16.612 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:16 smithi156 ceph-mon[129280]: pgmap v775: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 1.54k op/s 2023-07-06T20:34:18.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:18 smithi069 ceph-mon[126399]: pgmap v776: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.81k op/s 2023-07-06T20:34:18.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:18 smithi156 ceph-mon[129280]: pgmap v776: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.81k op/s 2023-07-06T20:34:18.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:18 smithi098 ceph-mon[128968]: pgmap v776: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.81k op/s 2023-07-06T20:34:19.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 750.0 s, 1527.9 tps, lat 66.193 ms stddev 119.344 2023-07-06T20:34:20.515 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:20 smithi069 ceph-mon[126399]: pgmap v777: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 2.41k op/s 2023-07-06T20:34:20.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:20 smithi156 ceph-mon[129280]: pgmap v777: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 2.41k op/s 2023-07-06T20:34:20.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:20 smithi098 ceph-mon[128968]: pgmap v777: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 2.41k op/s 2023-07-06T20:34:22.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:22 smithi069 ceph-mon[126399]: pgmap v778: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 39 MiB/s wr, 2.86k op/s 2023-07-06T20:34:22.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:22 smithi156 ceph-mon[129280]: pgmap v778: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 39 MiB/s wr, 2.86k op/s 2023-07-06T20:34:22.628 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:22 smithi098 ceph-mon[128968]: pgmap v778: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 39 MiB/s wr, 2.86k op/s 2023-07-06T20:34:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 755.0 s, 1765.3 tps, lat 48.898 ms stddev 38.642 2023-07-06T20:34:24.615 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:24 smithi156 ceph-mon[129280]: pgmap v779: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 2.86k op/s 2023-07-06T20:34:24.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:24 smithi098 ceph-mon[128968]: pgmap v779: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 2.86k op/s 2023-07-06T20:34:24.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:24 smithi069 ceph-mon[126399]: pgmap v779: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 MiB/s wr, 2.86k op/s 2023-07-06T20:34:26.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:26 smithi156 ceph-mon[129280]: pgmap v780: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 2.58k op/s 2023-07-06T20:34:26.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:26 smithi098 ceph-mon[128968]: pgmap v780: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 2.58k op/s 2023-07-06T20:34:26.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:26 smithi069 ceph-mon[126399]: pgmap v780: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 2.58k op/s 2023-07-06T20:34:28.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:28 smithi098 ceph-mon[128968]: pgmap v781: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 MiB/s wr, 2.47k op/s 2023-07-06T20:34:28.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:28 smithi069 ceph-mon[126399]: pgmap v781: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 MiB/s wr, 2.47k op/s 2023-07-06T20:34:28.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:28 smithi156 ceph-mon[129280]: pgmap v781: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 MiB/s wr, 2.47k op/s 2023-07-06T20:34:29.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 760.0 s, 1366.8 tps, lat 80.704 ms stddev 228.011 2023-07-06T20:34:30.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:30 smithi069 ceph-mon[126399]: pgmap v782: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 MiB/s wr, 2.45k op/s 2023-07-06T20:34:30.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:30 smithi156 ceph-mon[129280]: pgmap v782: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 MiB/s wr, 2.45k op/s 2023-07-06T20:34:30.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:30 smithi098 ceph-mon[128968]: pgmap v782: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 MiB/s wr, 2.45k op/s 2023-07-06T20:34:32.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:32 smithi069 ceph-mon[126399]: pgmap v783: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 2.09k op/s 2023-07-06T20:34:32.856 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:32 smithi156 ceph-mon[129280]: pgmap v783: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 2.09k op/s 2023-07-06T20:34:32.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:32 smithi098 ceph-mon[128968]: pgmap v783: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 2.09k op/s 2023-07-06T20:34:34.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 765.0 s, 1614.7 tps, lat 63.623 ms stddev 71.952 2023-07-06T20:34:34.335 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-06T20:34:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.713+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/1572082194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08071960 msgr2=0x7f7b0810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.713+0000 7f7b0c9ff700 1 --2- 172.21.15.69:0/1572082194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08071960 0x7f7b0810b900 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009990 tx=0x7f7afc009ca0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.713+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/1572082194 shutdown_connections 2023-07-06T20:34:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.713+0000 7f7b0c9ff700 1 --2- 172.21.15.69:0/1572082194 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b0810be90 0x7f7b0810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.713+0000 7f7b0c9ff700 1 --2- 172.21.15.69:0/1572082194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b08071010 0x7f7b08071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.713+0000 7f7b0c9ff700 1 --2- 172.21.15.69:0/1572082194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08071960 0x7f7b0810b900 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.713+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/1572082194 >> 172.21.15.69:0/1572082194 conn(0x7f7b0806cbc0 msgr2=0x7f7b0806eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:34:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.713+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/1572082194 shutdown_connections 2023-07-06T20:34:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.713+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/1572082194 wait complete. 2023-07-06T20:34:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.715+0000 7f7b0c9ff700 1 Processor -- start 2023-07-06T20:34:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.715+0000 7f7b0c9ff700 1 -- start start 2023-07-06T20:34:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.716+0000 7f7b0c9ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b08071010 0x7f7b081a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.716+0000 7f7b0c9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08071960 0x7f7b081a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.716+0000 7f7b0c9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b0810be90 0x7f7b081abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.716+0000 7f7b0c9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b081ac370 con 0x7f7b0810be90 2023-07-06T20:34:34.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.716+0000 7f7b0c9ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b081ac4b0 con 0x7f7b08071010 2023-07-06T20:34:34.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.716+0000 7f7b0c9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b081ac640 con 0x7f7b08071960 2023-07-06T20:34:34.719 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.716+0000 7f7b0659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b08071010 0x7f7b081a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.716+0000 7f7b05d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08071960 0x7f7b081a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.716+0000 7f7b05d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08071960 0x7f7b081a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43724/0 (socket says 172.21.15.69:43724) 2023-07-06T20:34:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.716+0000 7f7b05d9b700 1 -- 172.21.15.69:0/340143191 learned_addr learned my addr 172.21.15.69:0/340143191 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:34:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.716+0000 7f7b06d9d700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b0810be90 0x7f7b081abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.717+0000 7f7b05d9b700 1 -- 172.21.15.69:0/340143191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b08071010 msgr2=0x7f7b081a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.717+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b08071010 0x7f7b081a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.717+0000 7f7b05d9b700 1 -- 172.21.15.69:0/340143191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b0810be90 msgr2=0x7f7b081abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.717+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b0810be90 0x7f7b081abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.717+0000 7f7b05d9b700 1 -- 172.21.15.69:0/340143191 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b00009c40 con 0x7f7b08071960 2023-07-06T20:34:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.717+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08071960 0x7f7b081a4d30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7afc005f50 tx=0x7f7afc0057b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:34:34.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.718+0000 7f7af77fe700 1 -- 172.21.15.69:0/340143191 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc01d070 con 0x7f7b08071960 2023-07-06T20:34:34.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.718+0000 7f7af77fe700 1 -- 172.21.15.69:0/340143191 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7afc00fbb0 con 0x7f7b08071960 2023-07-06T20:34:34.722 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.718+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/340143191 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc0095f0 con 0x7f7b08071960 2023-07-06T20:34:34.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.718+0000 7f7af77fe700 1 -- 172.21.15.69:0/340143191 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc017650 con 0x7f7b08071960 2023-07-06T20:34:34.723 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.718+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/340143191 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b081acbb0 con 0x7f7b08071960 2023-07-06T20:34:34.724 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.722+0000 7f7af77fe700 1 -- 172.21.15.69:0/340143191 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7afc00fd20 con 0x7f7b08071960 2023-07-06T20:34:34.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.723+0000 7f7b20411b80 1 Processor -- start 2023-07-06T20:34:34.725 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.723+0000 7f7b20411b80 1 -- start start 2023-07-06T20:34:34.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.723+0000 7f7b20411b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c8f20 0x5610031c9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.723+0000 7f7b20411b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031c9870 0x5610031c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.723+0000 7f7b20411b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031ca390 0x5610031ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.724+0000 7f7b20411b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610030cdcb0 con 0x5610031c8f20 2023-07-06T20:34:34.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.724+0000 7f7b20411b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561003120a40 con 0x5610031ca390 2023-07-06T20:34:34.726 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.724+0000 7f7b20411b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561003119ed0 con 0x5610031c9870 2023-07-06T20:34:34.727 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.724+0000 7f7b05d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031ca390 0x5610031ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.724+0000 7f7b05d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031ca390 0x5610031ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:59480/0 (socket says 172.21.15.69:59480) 2023-07-06T20:34:34.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.724+0000 7f7b05d9b700 1 -- 172.21.15.69:0/4263784860 learned_addr learned my addr 172.21.15.69:0/4263784860 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:34:34.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.724+0000 7f7b0659c700 1 --2- 172.21.15.69:0/4263784860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031c9870 0x5610031c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.724+0000 7f7b06d9d700 1 --2- 172.21.15.69:0/4263784860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c8f20 0x5610031c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.724+0000 7f7b05d9b700 1 -- 172.21.15.69:0/4263784860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031c9870 msgr2=0x5610031c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:34.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.724+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/4263784860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031c9870 0x5610031c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.728 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.725+0000 7f7b05d9b700 1 -- 172.21.15.69:0/4263784860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c8f20 msgr2=0x5610031c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:34.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.725+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/4263784860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c8f20 0x5610031c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.724+0000 7f7af77fe700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7af00a11d0 0x7f7af00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.725+0000 7f7b05d9b700 1 -- 172.21.15.69:0/4263784860 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561002f47de0 con 0x5610031ca390 2023-07-06T20:34:34.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.725+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/4263784860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031ca390 0x5610031ca830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7afc09eee0 tx=0x7f7afc0976b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1144a9c620b9b899 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:34:34.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.725+0000 7f7af77fe700 1 -- 172.21.15.69:0/340143191 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f7afc0cdf60 con 0x7f7b08071960 2023-07-06T20:34:34.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.725+0000 7f7b06d9d700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7af00a11d0 0x7f7af00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.729 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.725+0000 7f7af4ff9700 1 -- 172.21.15.69:0/4263784860 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc09b140 con 0x5610031ca390 2023-07-06T20:34:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.725+0000 7f7af4ff9700 1 -- 172.21.15.69:0/4263784860 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7afc09aeb0 con 0x5610031ca390 2023-07-06T20:34:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.725+0000 7f7af4ff9700 1 -- 172.21.15.69:0/4263784860 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0a8140 con 0x5610031ca390 2023-07-06T20:34:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.726+0000 7f7b20411b80 1 -- 172.21.15.69:0/4263784860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031ca390 msgr2=0x5610031ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.726+0000 7f7b20411b80 1 --2- 172.21.15.69:0/4263784860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031ca390 0x5610031ca830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7afc09eee0 tx=0x7f7afc0976b0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.726+0000 7f7b06d9d700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7af00a11d0 0x7f7af00a3680 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7b000076a0 tx=0x7f7b00009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:34:34.731 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.727+0000 7f7b20411b80 1 -- 172.21.15.69:0/4263784860 shutdown_connections 2023-07-06T20:34:34.731 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.727+0000 7f7b20411b80 1 --2- 172.21.15.69:0/4263784860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031ca390 0x5610031ca830 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.731 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.727+0000 7f7b20411b80 1 --2- 172.21.15.69:0/4263784860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c8f20 0x5610031c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.731 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.727+0000 7f7b20411b80 1 --2- 172.21.15.69:0/4263784860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031c9870 0x5610031c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.731 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.727+0000 7f7b20411b80 1 -- 172.21.15.69:0/4263784860 >> 172.21.15.69:0/4263784860 conn(0x5610031be640 msgr2=0x5610031c39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:34:34.735 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.733+0000 7f7b20411b80 1 -- 172.21.15.69:0/4263784860 shutdown_connections 2023-07-06T20:34:34.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.733+0000 7f7b20411b80 1 -- 172.21.15.69:0/4263784860 wait complete. 2023-07-06T20:34:34.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.734+0000 7f7b20411b80 1 -- start start 2023-07-06T20:34:34.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.734+0000 7f7b20411b80 1 Processor -- start 2023-07-06T20:34:34.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.735+0000 7f7b20411b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.735+0000 7f7b20411b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031c9870 0x5610031d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.735+0000 7f7b20411b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031ca390 0x5610031d9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.735+0000 7f7b20411b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610030b9ac0 con 0x5610031ca390 2023-07-06T20:34:34.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.735+0000 7f7b20411b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610030b97c0 con 0x5610031c8f20 2023-07-06T20:34:34.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.735+0000 7f7b20411b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561002f9d920 con 0x5610031c9870 2023-07-06T20:34:34.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.735+0000 7f7b20411b80 10 client.0 _ll_register_callbacks cb 0x5610030b7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:34:34.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.736+0000 7f7b0659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.736+0000 7f7b0659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:59492/0 (socket says 172.21.15.69:59492) 2023-07-06T20:34:34.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.736+0000 7f7b0659c700 1 -- 172.21.15.69:0/4006693532 learned_addr learned my addr 172.21.15.69:0/4006693532 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:34:34.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.736+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031c9870 0x5610031d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.737+0000 7f7b05d9b700 1 -- 172.21.15.69:0/4006693532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 msgr2=0x5610031d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:34.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.737+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.737+0000 7f7b05d9b700 1 -- 172.21.15.69:0/4006693532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031ca390 msgr2=0x5610031d9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:34.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.737+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031ca390 0x5610031d9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.741 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.737+0000 7f7b05d9b700 1 -- 172.21.15.69:0/4006693532 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b081acbb0 con 0x5610031c9870 2023-07-06T20:34:34.741 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.737+0000 7f7b0659c700 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:34:34.741 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.737+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031c9870 0x5610031d2a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0cb9d0 tx=0x7f7afc0974d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa69dee16fd2d08 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:34:34.741 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.738+0000 7f7aedffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-06T20:34:34.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.738+0000 7f7aedffb700 1 -- 172.21.15.69:0/4006693532 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0a8140 con 0x5610031c9870 2023-07-06T20:34:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.738+0000 7f7aedffb700 1 -- 172.21.15.69:0/4006693532 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f7afc09b370 con 0x5610031c9870 2023-07-06T20:34:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.738+0000 7f7b20411b80 1 -- 172.21.15.69:0/4006693532 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610030734f0 con 0x5610031c9870 2023-07-06T20:34:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.738+0000 7f7aedffb700 1 -- 172.21.15.69:0/4006693532 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7afc09b140 con 0x5610031c9870 2023-07-06T20:34:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.738+0000 7f7b20411b80 10 client.25825 fetch_fsmap learned FSMap version 30 2023-07-06T20:34:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.738+0000 7f7b20411b80 1 -- 172.21.15.69:0/4006693532 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561002f30c60 con 0x5610031c9870 2023-07-06T20:34:34.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.739+0000 7f7aedffb700 1 -- 172.21.15.69:0/4006693532 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7afc0b2030 con 0x5610031c9870 2023-07-06T20:34:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.739+0000 7f7aedffb700 1 -- 172.21.15.69:0/4006693532 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7afc0a32a0 con 0x5610031c9870 2023-07-06T20:34:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.739+0000 7f7b20411b80 10 client.25825 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:34:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.739+0000 7f7b20411b80 10 client.25825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:34:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.739+0000 7f7b20411b80 20 client.25825 populate_metadata read hostname 'smithi069' 2023-07-06T20:34:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.739+0000 7f7b20411b80 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5610031dde30 0x5610031e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.739+0000 7f7b20411b80 4 client.25825 mds_command: new command op to 24584 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T20:34:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.739+0000 7f7b20411b80 1 -- 172.21.15.69:0/4006693532 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561003197d60 con 0x5610031dde30 2023-07-06T20:34:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.739+0000 7f7b0659c700 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5610031dde30 0x5610031e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.740+0000 7f7b0659c700 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5610031dde30 0x5610031e0240 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709a1b6de896f38b server_cookie=8ebfe0e4738e4d85 in_seq=0 out_seq=0 2023-07-06T20:34:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.740+0000 7f7aedffb700 10 client.25825 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:34:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7aedffb700 1 -- 172.21.15.69:0/4006693532 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561003197d60 con 0x5610031dde30 2023-07-06T20:34:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7aedffb700 10 client.25825 handle_command_reply: tid=0 2023-07-06T20:34:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7b20411b80 1 client.25825 shutdown 2023-07-06T20:34:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7b20411b80 1 -- 172.21.15.69:0/4006693532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031c9870 msgr2=0x5610031d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7b20411b80 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031c9870 0x5610031d2a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0cb9d0 tx=0x7f7afc0974d0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7b20411b80 1 -- 172.21.15.69:0/4006693532 shutdown_connections 2023-07-06T20:34:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7b20411b80 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x5610031dde30 0x5610031e0240 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7b20411b80 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031ca390 0x5610031d9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7b20411b80 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7aedffb700 0 client.25825 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:34:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7b20411b80 1 --2- 172.21.15.69:0/4006693532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031c9870 0x5610031d2a00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7b20411b80 1 -- 172.21.15.69:0/4006693532 >> 172.21.15.69:0/4006693532 conn(0x5610031be640 msgr2=0x5610031c30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:34:34.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7b20411b80 1 -- 172.21.15.69:0/4006693532 shutdown_connections 2023-07-06T20:34:34.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7b20411b80 1 -- 172.21.15.69:0/4006693532 wait complete. 2023-07-06T20:34:34.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.741+0000 7f7b20411b80 20 client.25825 trim_cache size 0 max 16384 2023-07-06T20:34:34.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:34 smithi069 ceph-mon[126399]: pgmap v784: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 MiB/s wr, 1.80k op/s 2023-07-06T20:34:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.771+0000 7f7b20411b80 1 Processor -- start 2023-07-06T20:34:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.771+0000 7f7b20411b80 1 -- start start 2023-07-06T20:34:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.771+0000 7f7b20411b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.771+0000 7f7b20411b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c9870 0x5610031d9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.771+0000 7f7b20411b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 0x5610031d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.771+0000 7f7b20411b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610031be450 con 0x5610031c9870 2023-07-06T20:34:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.771+0000 7f7b20411b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610031d1970 con 0x5610031c8f20 2023-07-06T20:34:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.771+0000 7f7b20411b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610031d2030 con 0x5610031ca390 2023-07-06T20:34:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.772+0000 7f7b06d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 0x5610031d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.772+0000 7f7b06d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 0x5610031d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.69:43744/0 (socket says 172.21.15.69:43744) 2023-07-06T20:34:34.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.772+0000 7f7b06d9d700 1 -- 172.21.15.69:0/1872513691 learned_addr learned my addr 172.21.15.69:0/1872513691 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:34:34.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.772+0000 7f7b0659c700 1 --2- 172.21.15.69:0/1872513691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.772+0000 7f7b06d9d700 1 -- 172.21.15.69:0/1872513691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 msgr2=0x5610031d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:34.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.772+0000 7f7b06d9d700 1 --2- 172.21.15.69:0/1872513691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.772+0000 7f7b06d9d700 1 -- 172.21.15.69:0/1872513691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c9870 msgr2=0x5610031d9a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T20:34:34.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.773+0000 7f7b06d9d700 1 --2- 172.21.15.69:0/1872513691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c9870 0x5610031d9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.773+0000 7f7b06d9d700 1 -- 172.21.15.69:0/1872513691 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610031cac80 con 0x5610031ca390 2023-07-06T20:34:34.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.773+0000 7f7b06d9d700 1 --2- 172.21.15.69:0/1872513691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 0x5610031d2a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b0003c8e0 tx=0x7f7b00004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e042039f1af32121 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:34:34.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.773+0000 7f7af57fa700 1 -- 172.21.15.69:0/1872513691 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00041070 con 0x5610031ca390 2023-07-06T20:34:34.781 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.773+0000 7f7af57fa700 1 -- 172.21.15.69:0/1872513691 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b0000f040 con 0x5610031ca390 2023-07-06T20:34:34.782 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.773+0000 7f7af57fa700 1 -- 172.21.15.69:0/1872513691 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0003aea0 con 0x5610031ca390 2023-07-06T20:34:34.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.774+0000 7f7b20411b80 1 -- 172.21.15.69:0/1872513691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 msgr2=0x5610031d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:34.786 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.774+0000 7f7b20411b80 1 --2- 172.21.15.69:0/1872513691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 0x5610031d2a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b0003c8e0 tx=0x7f7b00004b40 comp rx=0 tx=0).stop 2023-07-06T20:34:34.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.774+0000 7f7b20411b80 1 -- 172.21.15.69:0/1872513691 shutdown_connections 2023-07-06T20:34:34.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.774+0000 7f7b20411b80 1 --2- 172.21.15.69:0/1872513691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 0x5610031d2a00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.787 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.774+0000 7f7b20411b80 1 --2- 172.21.15.69:0/1872513691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.774+0000 7f7b20411b80 1 --2- 172.21.15.69:0/1872513691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c9870 0x5610031d9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.774+0000 7f7b20411b80 1 -- 172.21.15.69:0/1872513691 >> 172.21.15.69:0/1872513691 conn(0x5610031be640 msgr2=0x5610031c30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:34:34.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.774+0000 7f7b20411b80 1 -- 172.21.15.69:0/1872513691 shutdown_connections 2023-07-06T20:34:34.788 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.774+0000 7f7b20411b80 1 -- 172.21.15.69:0/1872513691 wait complete. 2023-07-06T20:34:34.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.775+0000 7f7b20411b80 1 -- start start 2023-07-06T20:34:34.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.775+0000 7f7b20411b80 1 Processor -- start 2023-07-06T20:34:34.789 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.775+0000 7f7b20411b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.775+0000 7f7b20411b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c9870 0x5610031d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.775+0000 7f7b20411b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 0x5610031d6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.775+0000 7f7b20411b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610031d6c50 con 0x5610031c9870 2023-07-06T20:34:34.790 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.775+0000 7f7b20411b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610031d6dc0 con 0x5610031c8f20 2023-07-06T20:34:34.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.775+0000 7f7b20411b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561003231ca0 con 0x5610031ca390 2023-07-06T20:34:34.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b20411b80 10 client.0 _ll_register_callbacks cb 0x5610030b7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T20:34:34.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b0659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.791 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b0659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:59506/0 (socket says 172.21.15.69:59506) 2023-07-06T20:34:34.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b0659c700 1 -- 172.21.15.69:0/481447323 learned_addr learned my addr 172.21.15.69:0/481447323 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:34:34.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c9870 0x5610031d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b06d9d700 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 0x5610031d6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.792 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b0659c700 1 -- 172.21.15.69:0/481447323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 msgr2=0x5610031d6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:34.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b0659c700 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 0x5610031d6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.793 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b0659c700 1 -- 172.21.15.69:0/481447323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c9870 msgr2=0x5610031d61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T20:34:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b0659c700 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c9870 0x5610031d61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b0659c700 1 -- 172.21.15.69:0/481447323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0003c650 con 0x5610031c8f20 2023-07-06T20:34:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b06d9d700 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 0x5610031d6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:34:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7b0659c700 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d9640 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x5610031e0ab0 tx=0x7f7af801ff80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b24f60362949da9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:34:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.776+0000 7f7aedffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-07-06T20:34:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.777+0000 7f7aedffb700 1 -- 172.21.15.69:0/481447323 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af8016070 con 0x5610031c8f20 2023-07-06T20:34:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.777+0000 7f7b20411b80 1 -- 172.21.15.69:0/481447323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7afc09aeb0 con 0x5610031c8f20 2023-07-06T20:34:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.777+0000 7f7aedffb700 1 -- 172.21.15.69:0/481447323 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f7af8004830 con 0x5610031c8f20 2023-07-06T20:34:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.777+0000 7f7aedffb700 1 -- 172.21.15.69:0/481447323 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7af801f5d0 con 0x5610031c8f20 2023-07-06T20:34:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.777+0000 7f7b20411b80 10 client.25877 fetch_fsmap learned FSMap version 30 2023-07-06T20:34:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.777+0000 7f7b20411b80 1 -- 172.21.15.69:0/481447323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561002f6e250 con 0x5610031c8f20 2023-07-06T20:34:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.777+0000 7f7aedffb700 1 -- 172.21.15.69:0/481447323 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7afc09aeb0 con 0x5610031c8f20 2023-07-06T20:34:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.778+0000 7f7aedffb700 1 -- 172.21.15.69:0/481447323 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7af80116c0 con 0x5610031c8f20 2023-07-06T20:34:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.778+0000 7f7b20411b80 10 client.25877 fetch_fsmap finished waiting for FSMap version 30 2023-07-06T20:34:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.778+0000 7f7b20411b80 10 client.25877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-06T20:34:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.778+0000 7f7b20411b80 20 client.25877 populate_metadata read hostname 'smithi069' 2023-07-06T20:34:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.778+0000 7f7b20411b80 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x561003233660 0x561003235a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:34:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.778+0000 7f7b20411b80 4 client.25877 mds_command: new command op to 24584 tid=0[{"prefix": "scrub status"}] 2023-07-06T20:34:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.778+0000 7f7b20411b80 1 -- 172.21.15.69:0/481447323 --> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561003112eb0 con 0x561003233660 2023-07-06T20:34:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.778+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x561003233660 0x561003235a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:34:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.785+0000 7f7b05d9b700 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x561003233660 0x561003235a70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b50f16cd1134b2 server_cookie=dd76446789343012 in_seq=0 out_seq=0 2023-07-06T20:34:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.785+0000 7f7aedffb700 10 client.25877 ms_handle_connect on v2:172.21.15.98:6838/3235525224 2023-07-06T20:34:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.786+0000 7f7aedffb700 1 -- 172.21.15.69:0/481447323 <== mds.0 v2:172.21.15.98:6838/3235525224 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561003112eb0 con 0x561003233660 2023-07-06T20:34:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.786+0000 7f7aedffb700 10 client.25877 handle_command_reply: tid=0 2023-07-06T20:34:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.786+0000 7f7b20411b80 1 client.25877 shutdown 2023-07-06T20:34:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.787+0000 7f7b20411b80 1 -- 172.21.15.69:0/481447323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 msgr2=0x5610031d9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.787+0000 7f7b20411b80 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d9640 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x5610031e0ab0 tx=0x7f7af801ff80 comp rx=0 tx=0).stop 2023-07-06T20:34:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.787+0000 7f7b20411b80 1 -- 172.21.15.69:0/481447323 shutdown_connections 2023-07-06T20:34:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.787+0000 7f7b20411b80 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.98:6838/3235525224,v1:172.21.15.98:6839/3235525224] conn(0x561003233660 0x561003235a70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.787+0000 7f7b20411b80 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5610031ca390 0x5610031d6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.787+0000 7f7b20411b80 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610031c8f20 0x5610031d9640 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.787+0000 7f7b20411b80 1 --2- 172.21.15.69:0/481447323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610031c9870 0x5610031d61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.787+0000 7f7b20411b80 1 -- 172.21.15.69:0/481447323 >> 172.21.15.69:0/481447323 conn(0x5610031be640 msgr2=0x5610031cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:34:34.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.787+0000 7f7aedffb700 0 client.25877 ms_handle_reset on v2:172.21.15.98:6838/3235525224 2023-07-06T20:34:34.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.787+0000 7f7b20411b80 1 -- 172.21.15.69:0/481447323 shutdown_connections 2023-07-06T20:34:34.801 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.787+0000 7f7b20411b80 1 -- 172.21.15.69:0/481447323 wait complete. 2023-07-06T20:34:34.802 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:34:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-06T20:34:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2023-07-06T20:34:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9": { 2023-07-06T20:34:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2023-07-06T20:34:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9", 2023-07-06T20:34:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2023-07-06T20:34:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:34:34.803 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:34:34.803 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:34:34.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.789+0000 7f7b20411b80 20 client.25877 trim_cache size 0 max 16384 2023-07-06T20:34:34.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.791+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/340143191 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7af00a11d0 msgr2=0x7f7af00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:34.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.791+0000 7f7b0c9ff700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7af00a11d0 0x7f7af00a3680 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7b000076a0 tx=0x7f7b00009040 comp rx=0 tx=0).stop 2023-07-06T20:34:34.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.791+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/340143191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08071960 msgr2=0x7f7b081a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:34:34.804 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.791+0000 7f7b0c9ff700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08071960 0x7f7b081a4d30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7afc005f50 tx=0x7f7afc0057b0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.798+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/340143191 shutdown_connections 2023-07-06T20:34:34.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.798+0000 7f7b0c9ff700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7af00a11d0 0x7f7af00a3680 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.798+0000 7f7b0c9ff700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b0810be90 0x7f7b081abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.798+0000 7f7b0c9ff700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b08071010 0x7f7b081a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.798+0000 7f7b0c9ff700 1 --2- 172.21.15.69:0/340143191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08071960 0x7f7b081a4d30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:34:34.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.798+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/340143191 >> 172.21.15.69:0/340143191 conn(0x7f7b0806cbc0 msgr2=0x7f7b0806df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:34:34.805 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.798+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/340143191 shutdown_connections 2023-07-06T20:34:34.806 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:34:34.804+0000 7f7b0c9ff700 1 -- 172.21.15.69:0/340143191 wait complete. 2023-07-06T20:34:34.846 INFO:tasks.cephfs.filesystem:scrub status for tag:a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9 - {'path': '/', 'tag': 'a5a963aa-fb9f-4e8a-b7ee-9b18553e09a9', 'options': 'recursive,force'} 2023-07-06T20:34:34.847 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/filesystem.py", line 1681, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-07-06T20:34:34.860 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:34 smithi156 ceph-mon[129280]: pgmap v784: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 MiB/s wr, 1.80k op/s 2023-07-06T20:34:34.880 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:34 smithi098 ceph-mon[128968]: pgmap v784: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 MiB/s wr, 1.80k op/s 2023-07-06T20:34:36.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:36 smithi069 ceph-mon[126399]: pgmap v785: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 1.68k op/s 2023-07-06T20:34:36.862 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:36 smithi156 ceph-mon[129280]: pgmap v785: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 1.68k op/s 2023-07-06T20:34:36.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:36 smithi098 ceph-mon[128968]: pgmap v785: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 1.68k op/s 2023-07-06T20:34:38.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:38 smithi156 ceph-mon[129280]: pgmap v786: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 MiB/s wr, 1.85k op/s 2023-07-06T20:34:38.878 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:38 smithi098 ceph-mon[128968]: pgmap v786: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 MiB/s wr, 1.85k op/s 2023-07-06T20:34:39.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:38 smithi069 ceph-mon[126399]: pgmap v786: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 MiB/s wr, 1.85k op/s 2023-07-06T20:34:39.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 770.0 s, 2075.2 tps, lat 48.443 ms stddev 35.266 2023-07-06T20:34:40.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:39 smithi069 ceph-mon[126399]: pgmap v787: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 1.19k op/s 2023-07-06T20:34:40.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:39 smithi156 ceph-mon[129280]: pgmap v787: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 1.19k op/s 2023-07-06T20:34:40.132 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:39 smithi098 ceph-mon[128968]: pgmap v787: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 1.19k op/s 2023-07-06T20:34:42.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:41 smithi069 ceph-mon[126399]: pgmap v788: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 958 op/s 2023-07-06T20:34:42.114 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:41 smithi156 ceph-mon[129280]: pgmap v788: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 958 op/s 2023-07-06T20:34:42.131 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:41 smithi098 ceph-mon[128968]: pgmap v788: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 958 op/s 2023-07-06T20:34:44.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:43 smithi069 ceph-mon[126399]: pgmap v789: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 1.34k op/s 2023-07-06T20:34:44.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:43 smithi156 ceph-mon[129280]: pgmap v789: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 1.34k op/s 2023-07-06T20:34:44.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:43 smithi098 ceph-mon[128968]: pgmap v789: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 1.34k op/s 2023-07-06T20:34:44.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 775.0 s, 777.2 tps, lat 127.639 ms stddev 139.693 2023-07-06T20:34:46.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:45 smithi069 ceph-mon[126399]: pgmap v790: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 22 MiB/s wr, 1.11k op/s 2023-07-06T20:34:46.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:45 smithi156 ceph-mon[129280]: pgmap v790: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 22 MiB/s wr, 1.11k op/s 2023-07-06T20:34:46.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:45 smithi098 ceph-mon[128968]: pgmap v790: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 22 MiB/s wr, 1.11k op/s 2023-07-06T20:34:48.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:47 smithi069 ceph-mon[126399]: pgmap v791: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 MiB/s wr, 2.21k op/s 2023-07-06T20:34:48.112 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:47 smithi156 ceph-mon[129280]: pgmap v791: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 MiB/s wr, 2.21k op/s 2023-07-06T20:34:48.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:47 smithi098 ceph-mon[128968]: pgmap v791: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 MiB/s wr, 2.21k op/s 2023-07-06T20:34:49.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 780.0 s, 2054.0 tps, lat 49.013 ms stddev 36.074 2023-07-06T20:34:50.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:49 smithi156 ceph-mon[129280]: pgmap v792: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 2.29k op/s 2023-07-06T20:34:50.129 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:49 smithi098 ceph-mon[128968]: pgmap v792: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 2.29k op/s 2023-07-06T20:34:50.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:49 smithi069 ceph-mon[126399]: pgmap v792: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 MiB/s wr, 2.29k op/s 2023-07-06T20:34:52.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:51 smithi156 ceph-mon[129280]: pgmap v793: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 2.31k op/s 2023-07-06T20:34:52.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:51 smithi098 ceph-mon[128968]: pgmap v793: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 2.31k op/s 2023-07-06T20:34:52.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:51 smithi069 ceph-mon[126399]: pgmap v793: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 2.31k op/s 2023-07-06T20:34:54.235 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 785.0 s, 2098.8 tps, lat 47.597 ms stddev 35.018 2023-07-06T20:34:54.265 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:53 smithi069 ceph-mon[126399]: pgmap v794: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 2.31k op/s 2023-07-06T20:34:54.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:53 smithi156 ceph-mon[129280]: pgmap v794: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 2.31k op/s 2023-07-06T20:34:54.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:53 smithi098 ceph-mon[128968]: pgmap v794: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 2.31k op/s 2023-07-06T20:34:56.268 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:55 smithi069 ceph-mon[126399]: pgmap v795: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 1.63k op/s 2023-07-06T20:34:56.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:55 smithi156 ceph-mon[129280]: pgmap v795: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 1.63k op/s 2023-07-06T20:34:56.377 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:55 smithi098 ceph-mon[128968]: pgmap v795: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 1.63k op/s 2023-07-06T20:34:58.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:34:58 smithi156 ceph-mon[129280]: pgmap v796: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 2.20k op/s 2023-07-06T20:34:58.378 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:34:58 smithi098 ceph-mon[128968]: pgmap v796: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 2.20k op/s 2023-07-06T20:34:58.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:34:58 smithi069 ceph-mon[126399]: pgmap v796: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 2.20k op/s 2023-07-06T20:34:59.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 790.0 s, 954.6 tps, lat 103.562 ms stddev 202.182 2023-07-06T20:35:00.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:00 smithi156 ceph-mon[129280]: pgmap v797: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2023-07-06T20:35:00.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:00 smithi098 ceph-mon[128968]: pgmap v797: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2023-07-06T20:35:00.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:00 smithi069 ceph-mon[126399]: pgmap v797: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2023-07-06T20:35:02.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:02 smithi098 ceph-mon[128968]: pgmap v798: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 26 MiB/s wr, 1.54k op/s 2023-07-06T20:35:02.515 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:02 smithi069 ceph-mon[126399]: pgmap v798: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 26 MiB/s wr, 1.54k op/s 2023-07-06T20:35:02.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:02 smithi156 ceph-mon[129280]: pgmap v798: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 26 MiB/s wr, 1.54k op/s 2023-07-06T20:35:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 795.0 s, 2040.9 tps, lat 49.647 ms stddev 36.277 2023-07-06T20:35:04.518 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:04 smithi069 ceph-mon[126399]: pgmap v799: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 1.86k op/s 2023-07-06T20:35:04.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:04 smithi156 ceph-mon[129280]: pgmap v799: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 1.86k op/s 2023-07-06T20:35:04.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:04 smithi098 ceph-mon[128968]: pgmap v799: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 1.86k op/s 2023-07-06T20:35:06.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:06 smithi069 ceph-mon[126399]: pgmap v800: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 1.68k op/s 2023-07-06T20:35:06.610 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:06 smithi156 ceph-mon[129280]: pgmap v800: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 1.68k op/s 2023-07-06T20:35:06.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:06 smithi098 ceph-mon[128968]: pgmap v800: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 1.68k op/s 2023-07-06T20:35:08.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:08 smithi156 ceph-mon[129280]: pgmap v801: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 31 MiB/s wr, 1.81k op/s 2023-07-06T20:35:08.644 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:08 smithi098 ceph-mon[128968]: pgmap v801: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 31 MiB/s wr, 1.81k op/s 2023-07-06T20:35:08.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:08 smithi069 ceph-mon[126399]: pgmap v801: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 31 MiB/s wr, 1.81k op/s 2023-07-06T20:35:09.236 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 800.0 s, 1287.8 tps, lat 73.086 ms stddev 91.898 2023-07-06T20:35:10.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:10 smithi069 ceph-mon[126399]: pgmap v802: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 1.41k op/s 2023-07-06T20:35:10.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:10 smithi156 ceph-mon[129280]: pgmap v802: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 1.41k op/s 2023-07-06T20:35:10.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:10 smithi098 ceph-mon[128968]: pgmap v802: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 1.41k op/s 2023-07-06T20:35:12.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:12 smithi069 ceph-mon[126399]: pgmap v803: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 967 op/s 2023-07-06T20:35:12.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:12 smithi156 ceph-mon[129280]: pgmap v803: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 967 op/s 2023-07-06T20:35:12.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:12 smithi098 ceph-mon[128968]: pgmap v803: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 967 op/s 2023-07-06T20:35:14.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 805.0 s, 1305.0 tps, lat 81.259 ms stddev 108.018 2023-07-06T20:35:14.778 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:14 smithi069 ceph-mon[126399]: pgmap v804: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 MiB/s wr, 2.44k op/s 2023-07-06T20:35:14.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:14 smithi156 ceph-mon[129280]: pgmap v804: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 MiB/s wr, 2.44k op/s 2023-07-06T20:35:14.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:14 smithi098 ceph-mon[128968]: pgmap v804: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 MiB/s wr, 2.44k op/s 2023-07-06T20:35:15.764 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:15 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:35:15.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:15 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:35:15.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:15 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:35:16.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:16 smithi069 ceph-mon[126399]: pgmap v805: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 2.17k op/s 2023-07-06T20:35:16.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:35:16.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:35:16.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:35:16.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:16 smithi156 ceph-mon[129280]: pgmap v805: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 2.17k op/s 2023-07-06T20:35:16.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:35:16.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:35:16.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:35:16.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:16 smithi098 ceph-mon[128968]: pgmap v805: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 2.17k op/s 2023-07-06T20:35:16.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:35:16.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:35:16.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:35:18.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:18 smithi069 ceph-mon[126399]: pgmap v806: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 36 MiB/s wr, 2.41k op/s 2023-07-06T20:35:18.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:18 smithi156 ceph-mon[129280]: pgmap v806: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 36 MiB/s wr, 2.41k op/s 2023-07-06T20:35:18.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:18 smithi098 ceph-mon[128968]: pgmap v806: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 36 MiB/s wr, 2.41k op/s 2023-07-06T20:35:19.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 810.0 s, 1920.2 tps, lat 51.604 ms stddev 37.925 2023-07-06T20:35:20.759 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:20 smithi069 ceph-mon[126399]: pgmap v807: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 MiB/s wr, 2.38k op/s 2023-07-06T20:35:20.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:20 smithi156 ceph-mon[129280]: pgmap v807: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 MiB/s wr, 2.38k op/s 2023-07-06T20:35:20.883 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:20 smithi098 ceph-mon[128968]: pgmap v807: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 MiB/s wr, 2.38k op/s 2023-07-06T20:35:22.858 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:22 smithi156 ceph-mon[129280]: pgmap v808: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 2.23k op/s 2023-07-06T20:35:22.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:22 smithi098 ceph-mon[128968]: pgmap v808: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 2.23k op/s 2023-07-06T20:35:23.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:22 smithi069 ceph-mon[126399]: pgmap v808: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 2.23k op/s 2023-07-06T20:35:24.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 815.0 s, 1683.1 tps, lat 56.325 ms stddev 63.813 2023-07-06T20:35:24.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:24 smithi156 ceph-mon[129280]: pgmap v809: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 2.32k op/s 2023-07-06T20:35:24.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:24 smithi098 ceph-mon[128968]: pgmap v809: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 2.32k op/s 2023-07-06T20:35:25.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:24 smithi069 ceph-mon[126399]: pgmap v809: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 2.32k op/s 2023-07-06T20:35:26.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:25 smithi069 ceph-mon[126399]: pgmap v810: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 764 op/s 2023-07-06T20:35:26.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:25 smithi156 ceph-mon[129280]: pgmap v810: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 764 op/s 2023-07-06T20:35:26.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:25 smithi098 ceph-mon[128968]: pgmap v810: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 764 op/s 2023-07-06T20:35:28.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:27 smithi069 ceph-mon[126399]: pgmap v811: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 768 op/s 2023-07-06T20:35:28.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:27 smithi156 ceph-mon[129280]: pgmap v811: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 768 op/s 2023-07-06T20:35:28.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:27 smithi098 ceph-mon[128968]: pgmap v811: 97 pgs: 97 active+clean; 8.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 768 op/s 2023-07-06T20:35:29.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 820.0 s, 1209.5 tps, lat 87.710 ms stddev 166.054 2023-07-06T20:35:30.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:29 smithi069 ceph-mon[126399]: pgmap v812: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 MiB/s wr, 1.16k op/s 2023-07-06T20:35:30.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:29 smithi156 ceph-mon[129280]: pgmap v812: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 MiB/s wr, 1.16k op/s 2023-07-06T20:35:30.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:29 smithi098 ceph-mon[128968]: pgmap v812: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 MiB/s wr, 1.16k op/s 2023-07-06T20:35:32.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:31 smithi156 ceph-mon[129280]: pgmap v813: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 1.15k op/s 2023-07-06T20:35:32.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:31 smithi098 ceph-mon[128968]: pgmap v813: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 1.15k op/s 2023-07-06T20:35:32.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:31 smithi069 ceph-mon[126399]: pgmap v813: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 1.15k op/s 2023-07-06T20:35:34.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:33 smithi098 ceph-mon[128968]: pgmap v814: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 30 MiB/s wr, 1.67k op/s 2023-07-06T20:35:34.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 825.0 s, 2049.4 tps, lat 48.372 ms stddev 34.991 2023-07-06T20:35:34.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:33 smithi069 ceph-mon[126399]: pgmap v814: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 30 MiB/s wr, 1.67k op/s 2023-07-06T20:35:34.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:33 smithi156 ceph-mon[129280]: pgmap v814: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 30 MiB/s wr, 1.67k op/s 2023-07-06T20:35:36.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:35 smithi069 ceph-mon[126399]: pgmap v815: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 1.46k op/s 2023-07-06T20:35:36.360 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:35 smithi156 ceph-mon[129280]: pgmap v815: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 1.46k op/s 2023-07-06T20:35:36.380 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:35 smithi098 ceph-mon[128968]: pgmap v815: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 1.46k op/s 2023-07-06T20:35:38.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:37 smithi069 ceph-mon[126399]: pgmap v816: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 1.67k op/s 2023-07-06T20:35:38.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:37 smithi156 ceph-mon[129280]: pgmap v816: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 1.67k op/s 2023-07-06T20:35:38.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:37 smithi098 ceph-mon[128968]: pgmap v816: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 1.67k op/s 2023-07-06T20:35:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 830.0 s, 761.5 tps, lat 123.393 ms stddev 176.018 2023-07-06T20:35:40.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:40 smithi156 ceph-mon[129280]: pgmap v817: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 2.18k op/s 2023-07-06T20:35:40.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:40 smithi098 ceph-mon[128968]: pgmap v817: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 2.18k op/s 2023-07-06T20:35:40.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:40 smithi069 ceph-mon[126399]: pgmap v817: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 2.18k op/s 2023-07-06T20:35:42.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:42 smithi156 ceph-mon[129280]: pgmap v818: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 1.74k op/s 2023-07-06T20:35:42.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:42 smithi098 ceph-mon[128968]: pgmap v818: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 1.74k op/s 2023-07-06T20:35:42.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:42 smithi069 ceph-mon[126399]: pgmap v818: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 1.74k op/s 2023-07-06T20:35:44.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 835.0 s, 1696.8 tps, lat 61.951 ms stddev 89.447 2023-07-06T20:35:44.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:44 smithi069 ceph-mon[126399]: pgmap v819: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 39 MiB/s wr, 2.87k op/s 2023-07-06T20:35:44.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:44 smithi156 ceph-mon[129280]: pgmap v819: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 39 MiB/s wr, 2.87k op/s 2023-07-06T20:35:44.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:44 smithi098 ceph-mon[128968]: pgmap v819: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 39 MiB/s wr, 2.87k op/s 2023-07-06T20:35:46.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:46 smithi069 ceph-mon[126399]: pgmap v820: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 2.42k op/s 2023-07-06T20:35:46.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:46 smithi156 ceph-mon[129280]: pgmap v820: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 2.42k op/s 2023-07-06T20:35:46.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:46 smithi098 ceph-mon[128968]: pgmap v820: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 2.42k op/s 2023-07-06T20:35:48.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:48 smithi069 ceph-mon[126399]: pgmap v821: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 2.49k op/s 2023-07-06T20:35:48.606 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:48 smithi156 ceph-mon[129280]: pgmap v821: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 2.49k op/s 2023-07-06T20:35:48.628 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:48 smithi098 ceph-mon[128968]: pgmap v821: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 2.49k op/s 2023-07-06T20:35:49.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 840.0 s, 2141.9 tps, lat 47.076 ms stddev 33.657 2023-07-06T20:35:50.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:50 smithi156 ceph-mon[129280]: pgmap v822: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 2.40k op/s 2023-07-06T20:35:50.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:50 smithi098 ceph-mon[128968]: pgmap v822: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 2.40k op/s 2023-07-06T20:35:50.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:50 smithi069 ceph-mon[126399]: pgmap v822: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 2.40k op/s 2023-07-06T20:35:52.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:52 smithi156 ceph-mon[129280]: pgmap v823: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.75k op/s 2023-07-06T20:35:52.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:52 smithi098 ceph-mon[128968]: pgmap v823: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.75k op/s 2023-07-06T20:35:52.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:52 smithi069 ceph-mon[126399]: pgmap v823: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.75k op/s 2023-07-06T20:35:53.766 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:53 smithi069 ceph-mon[126399]: pgmap v824: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 MiB/s wr, 1.69k op/s 2023-07-06T20:35:53.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:53 smithi156 ceph-mon[129280]: pgmap v824: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 MiB/s wr, 1.69k op/s 2023-07-06T20:35:53.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:53 smithi098 ceph-mon[128968]: pgmap v824: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 MiB/s wr, 1.69k op/s 2023-07-06T20:35:54.231 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 845.0 s, 1422.9 tps, lat 70.678 ms stddev 119.790 2023-07-06T20:35:55.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:54 smithi069 ceph-mon[126399]: pgmap v825: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 21 MiB/s wr, 619 op/s 2023-07-06T20:35:55.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:54 smithi156 ceph-mon[129280]: pgmap v825: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 21 MiB/s wr, 619 op/s 2023-07-06T20:35:55.131 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:54 smithi098 ceph-mon[128968]: pgmap v825: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 21 MiB/s wr, 619 op/s 2023-07-06T20:35:57.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:56 smithi069 ceph-mon[126399]: pgmap v826: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 22 MiB/s wr, 697 op/s 2023-07-06T20:35:57.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:56 smithi156 ceph-mon[129280]: pgmap v826: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 22 MiB/s wr, 697 op/s 2023-07-06T20:35:57.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:56 smithi098 ceph-mon[128968]: pgmap v826: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 22 MiB/s wr, 697 op/s 2023-07-06T20:35:59.112 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:35:58 smithi156 ceph-mon[129280]: pgmap v827: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 23 MiB/s wr, 832 op/s 2023-07-06T20:35:59.130 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:35:58 smithi098 ceph-mon[128968]: pgmap v827: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 23 MiB/s wr, 832 op/s 2023-07-06T20:35:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 850.0 s, 1476.0 tps, lat 67.884 ms stddev 111.064 2023-07-06T20:35:59.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:35:58 smithi069 ceph-mon[126399]: pgmap v827: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 23 MiB/s wr, 832 op/s 2023-07-06T20:36:01.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:00 smithi069 ceph-mon[126399]: pgmap v828: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 21 MiB/s wr, 1.01k op/s 2023-07-06T20:36:01.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:00 smithi156 ceph-mon[129280]: pgmap v828: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 21 MiB/s wr, 1.01k op/s 2023-07-06T20:36:01.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:00 smithi098 ceph-mon[128968]: pgmap v828: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 21 MiB/s wr, 1.01k op/s 2023-07-06T20:36:03.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:02 smithi069 ceph-mon[126399]: pgmap v829: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 27 MiB/s wr, 1.48k op/s 2023-07-06T20:36:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:02 smithi156 ceph-mon[129280]: pgmap v829: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 27 MiB/s wr, 1.48k op/s 2023-07-06T20:36:03.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:02 smithi098 ceph-mon[128968]: pgmap v829: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 27 MiB/s wr, 1.48k op/s 2023-07-06T20:36:04.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 855.0 s, 2021.0 tps, lat 49.428 ms stddev 37.461 2023-07-06T20:36:05.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:04 smithi156 ceph-mon[129280]: pgmap v830: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 1.60k op/s 2023-07-06T20:36:05.130 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:04 smithi098 ceph-mon[128968]: pgmap v830: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 1.60k op/s 2023-07-06T20:36:05.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:04 smithi069 ceph-mon[126399]: pgmap v830: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 1.60k op/s 2023-07-06T20:36:07.113 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:06 smithi156 ceph-mon[129280]: pgmap v831: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 1.66k op/s 2023-07-06T20:36:07.136 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:06 smithi098 ceph-mon[128968]: pgmap v831: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 1.66k op/s 2023-07-06T20:36:07.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:06 smithi069 ceph-mon[126399]: pgmap v831: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 1.66k op/s 2023-07-06T20:36:09.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:08 smithi069 ceph-mon[126399]: pgmap v832: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 2.34k op/s 2023-07-06T20:36:09.115 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:08 smithi156 ceph-mon[129280]: pgmap v832: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 2.34k op/s 2023-07-06T20:36:09.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:08 smithi098 ceph-mon[128968]: pgmap v832: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 2.34k op/s 2023-07-06T20:36:09.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 860.0 s, 483.6 tps, lat 207.264 ms stddev 234.271 2023-07-06T20:36:11.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:10 smithi069 ceph-mon[126399]: pgmap v833: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 2.72k op/s 2023-07-06T20:36:11.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:10 smithi156 ceph-mon[129280]: pgmap v833: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 2.72k op/s 2023-07-06T20:36:11.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:10 smithi098 ceph-mon[128968]: pgmap v833: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 2.72k op/s 2023-07-06T20:36:13.021 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:12 smithi069 ceph-mon[126399]: pgmap v834: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 36 MiB/s wr, 2.93k op/s 2023-07-06T20:36:13.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:12 smithi156 ceph-mon[129280]: pgmap v834: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 36 MiB/s wr, 2.93k op/s 2023-07-06T20:36:13.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:12 smithi098 ceph-mon[128968]: pgmap v834: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 36 MiB/s wr, 2.93k op/s 2023-07-06T20:36:14.235 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 865.0 s, 2023.5 tps, lat 49.456 ms stddev 35.867 2023-07-06T20:36:15.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:14 smithi069 ceph-mon[126399]: pgmap v835: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 MiB/s wr, 2.62k op/s 2023-07-06T20:36:15.122 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:14 smithi156 ceph-mon[129280]: pgmap v835: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 MiB/s wr, 2.62k op/s 2023-07-06T20:36:15.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:14 smithi098 ceph-mon[128968]: pgmap v835: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 MiB/s wr, 2.62k op/s 2023-07-06T20:36:16.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:16 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:36:16.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:16 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:36:16.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:16 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:36:17.762 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:17 smithi069 ceph-mon[126399]: pgmap v836: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 2.43k op/s 2023-07-06T20:36:17.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:17 smithi156 ceph-mon[129280]: pgmap v836: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 2.43k op/s 2023-07-06T20:36:17.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:17 smithi098 ceph-mon[128968]: pgmap v836: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 2.43k op/s 2023-07-06T20:36:18.761 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:18 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:36:18.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:18 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:36:18.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:18 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:36:18.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:18 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:36:18.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:18 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:36:18.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:18 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:36:18.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:18 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:36:18.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:18 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:36:18.876 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:18 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:36:19.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 870.0 s, 1997.7 tps, lat 49.644 ms stddev 35.222 2023-07-06T20:36:19.763 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:19 smithi069 ceph-mon[126399]: pgmap v837: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 MiB/s wr, 2.33k op/s 2023-07-06T20:36:19.857 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:19 smithi156 ceph-mon[129280]: pgmap v837: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 MiB/s wr, 2.33k op/s 2023-07-06T20:36:19.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:19 smithi098 ceph-mon[128968]: pgmap v837: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 MiB/s wr, 2.33k op/s 2023-07-06T20:36:21.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:20 smithi069 ceph-mon[126399]: pgmap v838: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 MiB/s wr, 1.56k op/s 2023-07-06T20:36:21.117 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:20 smithi156 ceph-mon[129280]: pgmap v838: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 MiB/s wr, 1.56k op/s 2023-07-06T20:36:21.136 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:20 smithi098 ceph-mon[128968]: pgmap v838: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 MiB/s wr, 1.56k op/s 2023-07-06T20:36:23.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:22 smithi069 ceph-mon[126399]: pgmap v839: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 1.07k op/s 2023-07-06T20:36:23.111 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:22 smithi156 ceph-mon[129280]: pgmap v839: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 1.07k op/s 2023-07-06T20:36:23.133 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:22 smithi098 ceph-mon[128968]: pgmap v839: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 MiB/s wr, 1.07k op/s 2023-07-06T20:36:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 875.0 s, 1442.8 tps, lat 69.857 ms stddev 79.530 2023-07-06T20:36:25.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:24 smithi069 ceph-mon[126399]: pgmap v840: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 707 op/s 2023-07-06T20:36:25.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:24 smithi156 ceph-mon[129280]: pgmap v840: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 707 op/s 2023-07-06T20:36:25.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:24 smithi098 ceph-mon[128968]: pgmap v840: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 707 op/s 2023-07-06T20:36:27.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:26 smithi069 ceph-mon[126399]: pgmap v841: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 20 MiB/s wr, 635 op/s 2023-07-06T20:36:27.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:26 smithi156 ceph-mon[129280]: pgmap v841: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 20 MiB/s wr, 635 op/s 2023-07-06T20:36:27.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:26 smithi098 ceph-mon[128968]: pgmap v841: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 20 MiB/s wr, 635 op/s 2023-07-06T20:36:29.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:28 smithi156 ceph-mon[129280]: pgmap v842: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 759 op/s 2023-07-06T20:36:29.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:28 smithi098 ceph-mon[128968]: pgmap v842: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 759 op/s 2023-07-06T20:36:29.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 880.0 s, 1793.2 tps, lat 48.733 ms stddev 35.922 2023-07-06T20:36:29.262 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:28 smithi069 ceph-mon[126399]: pgmap v842: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 759 op/s 2023-07-06T20:36:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:30 smithi069 ceph-mon[126399]: pgmap v843: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 930 op/s 2023-07-06T20:36:31.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:30 smithi156 ceph-mon[129280]: pgmap v843: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 930 op/s 2023-07-06T20:36:31.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:30 smithi098 ceph-mon[128968]: pgmap v843: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 930 op/s 2023-07-06T20:36:33.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:32 smithi069 ceph-mon[126399]: pgmap v844: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 MiB/s wr, 1.34k op/s 2023-07-06T20:36:33.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:32 smithi156 ceph-mon[129280]: pgmap v844: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 MiB/s wr, 1.34k op/s 2023-07-06T20:36:33.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:32 smithi098 ceph-mon[128968]: pgmap v844: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 MiB/s wr, 1.34k op/s 2023-07-06T20:36:34.233 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 885.0 s, 1433.0 tps, lat 76.161 ms stddev 142.275 2023-07-06T20:36:35.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:34 smithi156 ceph-mon[129280]: pgmap v845: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 MiB/s wr, 1.64k op/s 2023-07-06T20:36:35.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:34 smithi098 ceph-mon[128968]: pgmap v845: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 MiB/s wr, 1.64k op/s 2023-07-06T20:36:35.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:34 smithi069 ceph-mon[126399]: pgmap v845: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 MiB/s wr, 1.64k op/s 2023-07-06T20:36:37.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:36 smithi069 ceph-mon[126399]: pgmap v846: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 2.01k op/s 2023-07-06T20:36:37.110 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:36 smithi156 ceph-mon[129280]: pgmap v846: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 2.01k op/s 2023-07-06T20:36:37.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:36 smithi098 ceph-mon[128968]: pgmap v846: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 2.01k op/s 2023-07-06T20:36:39.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:38 smithi156 ceph-mon[129280]: pgmap v847: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 2.99k op/s 2023-07-06T20:36:39.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:38 smithi098 ceph-mon[128968]: pgmap v847: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 2.99k op/s 2023-07-06T20:36:39.228 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 890.0 s, 877.0 tps, lat 116.785 ms stddev 162.841 2023-07-06T20:36:39.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:38 smithi069 ceph-mon[126399]: pgmap v847: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 2.99k op/s 2023-07-06T20:36:41.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:40 smithi069 ceph-mon[126399]: pgmap v848: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 MiB/s wr, 3.23k op/s 2023-07-06T20:36:41.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:40 smithi156 ceph-mon[129280]: pgmap v848: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 MiB/s wr, 3.23k op/s 2023-07-06T20:36:41.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:40 smithi098 ceph-mon[128968]: pgmap v848: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 MiB/s wr, 3.23k op/s 2023-07-06T20:36:43.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:42 smithi069 ceph-mon[126399]: pgmap v849: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 36 MiB/s wr, 3.15k op/s 2023-07-06T20:36:43.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:42 smithi156 ceph-mon[129280]: pgmap v849: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 36 MiB/s wr, 3.15k op/s 2023-07-06T20:36:43.130 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:42 smithi098 ceph-mon[128968]: pgmap v849: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 36 MiB/s wr, 3.15k op/s 2023-07-06T20:36:44.230 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 895.0 s, 1977.1 tps, lat 51.143 ms stddev 41.521 2023-07-06T20:36:45.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:44 smithi156 ceph-mon[129280]: pgmap v850: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 MiB/s wr, 2.76k op/s 2023-07-06T20:36:45.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:44 smithi098 ceph-mon[128968]: pgmap v850: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 MiB/s wr, 2.76k op/s 2023-07-06T20:36:45.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:44 smithi069 ceph-mon[126399]: pgmap v850: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 MiB/s wr, 2.76k op/s 2023-07-06T20:36:47.012 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:46 smithi069 ceph-mon[126399]: pgmap v851: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 2.40k op/s 2023-07-06T20:36:47.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:46 smithi156 ceph-mon[129280]: pgmap v851: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 2.40k op/s 2023-07-06T20:36:47.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:46 smithi098 ceph-mon[128968]: pgmap v851: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 2.40k op/s 2023-07-06T20:36:49.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:48 smithi069 ceph-mon[126399]: pgmap v852: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 29 MiB/s wr, 2.01k op/s 2023-07-06T20:36:49.111 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:48 smithi156 ceph-mon[129280]: pgmap v852: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 29 MiB/s wr, 2.01k op/s 2023-07-06T20:36:49.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:48 smithi098 ceph-mon[128968]: pgmap v852: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 29 MiB/s wr, 2.01k op/s 2023-07-06T20:36:49.232 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 900.0 s, 1951.5 tps, lat 51.072 ms stddev 37.732 2023-07-06T20:36:49.487 INFO:teuthology.orchestra.run.smithi069.stdout:transaction type: 2023-07-06T20:36:49.487 INFO:teuthology.orchestra.run.smithi069.stdout:scaling factor: 500 2023-07-06T20:36:49.487 INFO:teuthology.orchestra.run.smithi069.stdout:query mode: simple 2023-07-06T20:36:49.487 INFO:teuthology.orchestra.run.smithi069.stdout:number of clients: 100 2023-07-06T20:36:49.488 INFO:teuthology.orchestra.run.smithi069.stdout:number of threads: 4 2023-07-06T20:36:49.488 INFO:teuthology.orchestra.run.smithi069.stdout:duration: 900 s 2023-07-06T20:36:49.488 INFO:teuthology.orchestra.run.smithi069.stdout:number of transactions actually processed: 1797911 2023-07-06T20:36:49.489 INFO:teuthology.orchestra.run.smithi069.stdout:latency average = 50.053 ms 2023-07-06T20:36:49.489 INFO:teuthology.orchestra.run.smithi069.stdout:latency stddev = 77.278 ms 2023-07-06T20:36:49.489 INFO:teuthology.orchestra.run.smithi069.stdout:tps = 1997.106137 (including connections establishing) 2023-07-06T20:36:49.489 INFO:teuthology.orchestra.run.smithi069.stdout:tps = 1997.123207 (excluding connections establishing) 2023-07-06T20:36:49.497 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-07-06T20:36:51.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:50 smithi098 ceph-mon[128968]: pgmap v853: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 22 MiB/s wr, 974 op/s 2023-07-06T20:36:51.263 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:50 smithi069 ceph-mon[126399]: pgmap v853: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 22 MiB/s wr, 974 op/s 2023-07-06T20:36:51.361 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:50 smithi156 ceph-mon[129280]: pgmap v853: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 22 MiB/s wr, 974 op/s 2023-07-06T20:36:53.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:52 smithi156 ceph-mon[129280]: pgmap v854: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 662 op/s 2023-07-06T20:36:53.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:52 smithi098 ceph-mon[128968]: pgmap v854: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 662 op/s 2023-07-06T20:36:53.268 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:52 smithi069 ceph-mon[126399]: pgmap v854: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 662 op/s 2023-07-06T20:36:55.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:54 smithi156 ceph-mon[129280]: pgmap v855: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 718 op/s 2023-07-06T20:36:55.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:54 smithi098 ceph-mon[128968]: pgmap v855: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 718 op/s 2023-07-06T20:36:55.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:54 smithi069 ceph-mon[126399]: pgmap v855: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 718 op/s 2023-07-06T20:36:57.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:56 smithi156 ceph-mon[129280]: pgmap v856: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 18 MiB/s wr, 844 op/s 2023-07-06T20:36:57.131 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:56 smithi098 ceph-mon[128968]: pgmap v856: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 18 MiB/s wr, 844 op/s 2023-07-06T20:36:57.261 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:56 smithi069 ceph-mon[126399]: pgmap v856: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 18 MiB/s wr, 844 op/s 2023-07-06T20:36:59.264 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:36:58 smithi069 ceph-mon[126399]: pgmap v857: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2023-07-06T20:36:59.366 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:36:58 smithi156 ceph-mon[129280]: pgmap v857: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2023-07-06T20:36:59.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:36:58 smithi098 ceph-mon[128968]: pgmap v857: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2023-07-06T20:37:00.646 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-07-06T20:37:00.803 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/: 2023-07-06T20:37:00.803 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.803 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 . 2023-07-06T20:37:00.803 INFO:teuthology.orchestra.run.smithi069.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 .. 2023-07-06T20:37:00.803 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 data 2023-07-06T20:37:00.804 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.804 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data: 2023-07-06T20:37:00.804 INFO:teuthology.orchestra.run.smithi069.stdout:total 31 2023-07-06T20:37:00.804 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 . 2023-07-06T20:37:00.804 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 .. 2023-07-06T20:37:00.804 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jul 6 20:21 base 2023-07-06T20:37:00.804 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Jul 6 20:19 current_logfiles 2023-07-06T20:37:00.805 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Jul 6 20:19 global 2023-07-06T20:37:00.805 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 log 2023-07-06T20:37:00.805 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_commit_ts 2023-07-06T20:37:00.805 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_dynshmem 2023-07-06T20:37:00.805 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Jul 6 20:19 pg_hba.conf 2023-07-06T20:37:00.805 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jul 6 20:19 pg_ident.conf 2023-07-06T20:37:00.805 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:37 pg_logical 2023-07-06T20:37:00.805 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:19 pg_multixact 2023-07-06T20:37:00.806 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 pg_notify 2023-07-06T20:37:00.806 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_replslot 2023-07-06T20:37:00.806 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_serial 2023-07-06T20:37:00.806 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_snapshots 2023-07-06T20:37:00.806 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:37 pg_stat 2023-07-06T20:37:00.806 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:37 pg_stat_tmp 2023-07-06T20:37:00.806 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:36 pg_subtrans 2023-07-06T20:37:00.807 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_tblspc 2023-07-06T20:37:00.807 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 pg_twophase 2023-07-06T20:37:00.807 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 PG_VERSION 2023-07-06T20:37:00.807 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Jul 6 20:37 pg_wal 2023-07-06T20:37:00.807 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:29 pg_xact 2023-07-06T20:37:00.807 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jul 6 20:19 postgresql.auto.conf 2023-07-06T20:37:00.807 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Jul 6 20:19 postgresql.conf 2023-07-06T20:37:00.807 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Jul 6 20:19 postmaster.opts 2023-07-06T20:37:00.808 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.808 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base: 2023-07-06T20:37:00.808 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.808 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jul 6 20:21 . 2023-07-06T20:37:00.808 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.808 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 6 20:19 1 2023-07-06T20:37:00.808 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 6 20:19 12346 2023-07-06T20:37:00.808 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 315 Jul 6 20:22 12347 2023-07-06T20:37:00.809 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:21 pgsql_tmp 2023-07-06T20:37:00.809 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.809 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/1: 2023-07-06T20:37:00.809 INFO:teuthology.orchestra.run.smithi069.stdout:total 7217 2023-07-06T20:37:00.809 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 6 20:19 . 2023-07-06T20:37:00.809 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jul 6 20:21 .. 2023-07-06T20:37:00.809 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 112 2023-07-06T20:37:00.809 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 113 2023-07-06T20:37:00.810 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 6 20:19 12182 2023-07-06T20:37:00.810 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12182_fsm 2023-07-06T20:37:00.810 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12182_vm 2023-07-06T20:37:00.810 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12184 2023-07-06T20:37:00.810 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12186 2023-07-06T20:37:00.810 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12187 2023-07-06T20:37:00.810 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12187_fsm 2023-07-06T20:37:00.810 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12187_vm 2023-07-06T20:37:00.811 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12189 2023-07-06T20:37:00.811 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12191 2023-07-06T20:37:00.811 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12192 2023-07-06T20:37:00.811 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12192_fsm 2023-07-06T20:37:00.811 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12192_vm 2023-07-06T20:37:00.811 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12194 2023-07-06T20:37:00.811 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12196 2023-07-06T20:37:00.811 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12197 2023-07-06T20:37:00.812 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12197_fsm 2023-07-06T20:37:00.812 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12197_vm 2023-07-06T20:37:00.812 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12199 2023-07-06T20:37:00.812 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12201 2023-07-06T20:37:00.812 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12202 2023-07-06T20:37:00.812 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12202_fsm 2023-07-06T20:37:00.812 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12202_vm 2023-07-06T20:37:00.812 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12204 2023-07-06T20:37:00.813 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12206 2023-07-06T20:37:00.813 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12207 2023-07-06T20:37:00.813 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12207_fsm 2023-07-06T20:37:00.813 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12207_vm 2023-07-06T20:37:00.813 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12209 2023-07-06T20:37:00.813 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12211 2023-07-06T20:37:00.813 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12212 2023-07-06T20:37:00.813 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12214 2023-07-06T20:37:00.814 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12216 2023-07-06T20:37:00.814 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 6 20:19 1247 2023-07-06T20:37:00.814 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1247_fsm 2023-07-06T20:37:00.814 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1247_vm 2023-07-06T20:37:00.814 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 6 20:19 1249 2023-07-06T20:37:00.814 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1249_fsm 2023-07-06T20:37:00.814 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1249_vm 2023-07-06T20:37:00.814 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 6 20:19 1255 2023-07-06T20:37:00.815 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1255_fsm 2023-07-06T20:37:00.815 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1255_vm 2023-07-06T20:37:00.815 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 6 20:19 1259 2023-07-06T20:37:00.815 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1259_fsm 2023-07-06T20:37:00.815 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1259_vm 2023-07-06T20:37:00.815 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1417 2023-07-06T20:37:00.815 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1417_vm 2023-07-06T20:37:00.815 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1418 2023-07-06T20:37:00.816 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1418_vm 2023-07-06T20:37:00.816 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 174 2023-07-06T20:37:00.816 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 175 2023-07-06T20:37:00.816 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2187 2023-07-06T20:37:00.816 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2224 2023-07-06T20:37:00.816 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2224_vm 2023-07-06T20:37:00.816 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2328 2023-07-06T20:37:00.817 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2328_vm 2023-07-06T20:37:00.817 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2336 2023-07-06T20:37:00.817 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2336_vm 2023-07-06T20:37:00.817 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2337 2023-07-06T20:37:00.817 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2600 2023-07-06T20:37:00.817 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2600_fsm 2023-07-06T20:37:00.817 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2600_vm 2023-07-06T20:37:00.817 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2601 2023-07-06T20:37:00.818 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2601_fsm 2023-07-06T20:37:00.818 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2601_vm 2023-07-06T20:37:00.818 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2602 2023-07-06T20:37:00.818 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2602_fsm 2023-07-06T20:37:00.818 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2602_vm 2023-07-06T20:37:00.818 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2603 2023-07-06T20:37:00.818 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2603_fsm 2023-07-06T20:37:00.818 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2603_vm 2023-07-06T20:37:00.819 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2604 2023-07-06T20:37:00.819 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2604_vm 2023-07-06T20:37:00.819 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2605 2023-07-06T20:37:00.819 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2605_fsm 2023-07-06T20:37:00.819 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2605_vm 2023-07-06T20:37:00.819 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2606 2023-07-06T20:37:00.819 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2606_fsm 2023-07-06T20:37:00.819 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2606_vm 2023-07-06T20:37:00.820 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2607 2023-07-06T20:37:00.820 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2607_fsm 2023-07-06T20:37:00.820 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2607_vm 2023-07-06T20:37:00.820 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 6 20:19 2608 2023-07-06T20:37:00.820 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2608_fsm 2023-07-06T20:37:00.820 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2608_vm 2023-07-06T20:37:00.820 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 6 20:19 2609 2023-07-06T20:37:00.820 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2609_fsm 2023-07-06T20:37:00.821 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2609_vm 2023-07-06T20:37:00.821 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2610 2023-07-06T20:37:00.821 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2610_fsm 2023-07-06T20:37:00.821 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2610_vm 2023-07-06T20:37:00.821 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2611 2023-07-06T20:37:00.821 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2611_vm 2023-07-06T20:37:00.821 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2612 2023-07-06T20:37:00.821 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2612_fsm 2023-07-06T20:37:00.822 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2612_vm 2023-07-06T20:37:00.822 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2613 2023-07-06T20:37:00.822 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2613_vm 2023-07-06T20:37:00.822 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2615 2023-07-06T20:37:00.822 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2615_fsm 2023-07-06T20:37:00.822 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2615_vm 2023-07-06T20:37:00.822 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2616 2023-07-06T20:37:00.822 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2616_fsm 2023-07-06T20:37:00.823 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2616_vm 2023-07-06T20:37:00.823 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 6 20:19 2617 2023-07-06T20:37:00.823 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2617_fsm 2023-07-06T20:37:00.823 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2617_vm 2023-07-06T20:37:00.823 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 6 20:19 2618 2023-07-06T20:37:00.823 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2618_fsm 2023-07-06T20:37:00.823 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2618_vm 2023-07-06T20:37:00.823 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 6 20:19 2619 2023-07-06T20:37:00.824 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2619_fsm 2023-07-06T20:37:00.824 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2619_vm 2023-07-06T20:37:00.824 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2620 2023-07-06T20:37:00.824 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2620_vm 2023-07-06T20:37:00.824 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2650 2023-07-06T20:37:00.824 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2651 2023-07-06T20:37:00.824 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2652 2023-07-06T20:37:00.824 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2653 2023-07-06T20:37:00.825 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2654 2023-07-06T20:37:00.825 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2655 2023-07-06T20:37:00.825 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2656 2023-07-06T20:37:00.825 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2657 2023-07-06T20:37:00.825 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 6 20:19 2658 2023-07-06T20:37:00.825 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 6 20:19 2659 2023-07-06T20:37:00.825 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2660 2023-07-06T20:37:00.825 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2661 2023-07-06T20:37:00.826 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2662 2023-07-06T20:37:00.826 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2663 2023-07-06T20:37:00.826 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2664 2023-07-06T20:37:00.826 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2665 2023-07-06T20:37:00.826 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2666 2023-07-06T20:37:00.826 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2667 2023-07-06T20:37:00.826 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2668 2023-07-06T20:37:00.826 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2669 2023-07-06T20:37:00.827 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2670 2023-07-06T20:37:00.827 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 335872 Jul 6 20:19 2673 2023-07-06T20:37:00.827 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Jul 6 20:19 2674 2023-07-06T20:37:00.827 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 6 20:19 2675 2023-07-06T20:37:00.827 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2678 2023-07-06T20:37:00.827 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2679 2023-07-06T20:37:00.827 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2680 2023-07-06T20:37:00.827 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2681 2023-07-06T20:37:00.828 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2682 2023-07-06T20:37:00.828 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2683 2023-07-06T20:37:00.828 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2684 2023-07-06T20:37:00.828 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2685 2023-07-06T20:37:00.828 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2686 2023-07-06T20:37:00.828 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2687 2023-07-06T20:37:00.828 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2688 2023-07-06T20:37:00.828 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2689 2023-07-06T20:37:00.829 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 6 20:19 2690 2023-07-06T20:37:00.829 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 6 20:19 2691 2023-07-06T20:37:00.829 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2692 2023-07-06T20:37:00.829 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2693 2023-07-06T20:37:00.829 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2696 2023-07-06T20:37:00.829 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2699 2023-07-06T20:37:00.829 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2701 2023-07-06T20:37:00.829 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2702 2023-07-06T20:37:00.830 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2703 2023-07-06T20:37:00.830 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2704 2023-07-06T20:37:00.830 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2753 2023-07-06T20:37:00.830 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2753_fsm 2023-07-06T20:37:00.830 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2753_vm 2023-07-06T20:37:00.830 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2754 2023-07-06T20:37:00.830 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2755 2023-07-06T20:37:00.830 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2756 2023-07-06T20:37:00.831 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2757 2023-07-06T20:37:00.831 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2830 2023-07-06T20:37:00.831 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2830_vm 2023-07-06T20:37:00.831 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2831 2023-07-06T20:37:00.831 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2832 2023-07-06T20:37:00.831 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2832_vm 2023-07-06T20:37:00.831 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2833 2023-07-06T20:37:00.831 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2834 2023-07-06T20:37:00.832 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2834_vm 2023-07-06T20:37:00.832 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2835 2023-07-06T20:37:00.832 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2836 2023-07-06T20:37:00.832 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2836_vm 2023-07-06T20:37:00.832 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2837 2023-07-06T20:37:00.832 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 6 20:19 2838 2023-07-06T20:37:00.832 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2838_fsm 2023-07-06T20:37:00.832 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2838_vm 2023-07-06T20:37:00.833 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2839 2023-07-06T20:37:00.833 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2840 2023-07-06T20:37:00.833 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2840_fsm 2023-07-06T20:37:00.833 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2840_vm 2023-07-06T20:37:00.833 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2841 2023-07-06T20:37:00.833 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2995 2023-07-06T20:37:00.833 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2995_vm 2023-07-06T20:37:00.834 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2996 2023-07-06T20:37:00.834 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3079 2023-07-06T20:37:00.834 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3079_fsm 2023-07-06T20:37:00.834 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3079_vm 2023-07-06T20:37:00.834 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3080 2023-07-06T20:37:00.834 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3081 2023-07-06T20:37:00.834 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3085 2023-07-06T20:37:00.834 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3118 2023-07-06T20:37:00.835 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3118_vm 2023-07-06T20:37:00.835 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3119 2023-07-06T20:37:00.835 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3164 2023-07-06T20:37:00.835 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3256 2023-07-06T20:37:00.835 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3256_vm 2023-07-06T20:37:00.835 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3257 2023-07-06T20:37:00.835 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3258 2023-07-06T20:37:00.835 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3350 2023-07-06T20:37:00.835 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3350_vm 2023-07-06T20:37:00.836 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3351 2023-07-06T20:37:00.836 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3379 2023-07-06T20:37:00.836 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3380 2023-07-06T20:37:00.836 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3381 2023-07-06T20:37:00.836 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3381_vm 2023-07-06T20:37:00.836 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3394 2023-07-06T20:37:00.836 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3394_fsm 2023-07-06T20:37:00.837 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3394_vm 2023-07-06T20:37:00.837 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3395 2023-07-06T20:37:00.837 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3439 2023-07-06T20:37:00.837 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3439_vm 2023-07-06T20:37:00.837 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3440 2023-07-06T20:37:00.837 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 3455 2023-07-06T20:37:00.837 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3456 2023-07-06T20:37:00.837 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3456_fsm 2023-07-06T20:37:00.838 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3456_vm 2023-07-06T20:37:00.838 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3466 2023-07-06T20:37:00.838 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3466_vm 2023-07-06T20:37:00.838 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3467 2023-07-06T20:37:00.838 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3468 2023-07-06T20:37:00.838 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3501 2023-07-06T20:37:00.838 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3501_vm 2023-07-06T20:37:00.838 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3502 2023-07-06T20:37:00.839 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3503 2023-07-06T20:37:00.839 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3534 2023-07-06T20:37:00.839 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3541 2023-07-06T20:37:00.839 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3541_fsm 2023-07-06T20:37:00.839 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3541_vm 2023-07-06T20:37:00.839 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3542 2023-07-06T20:37:00.839 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3574 2023-07-06T20:37:00.839 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3575 2023-07-06T20:37:00.840 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3576 2023-07-06T20:37:00.840 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3576_vm 2023-07-06T20:37:00.840 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3596 2023-07-06T20:37:00.840 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3596_vm 2023-07-06T20:37:00.840 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3597 2023-07-06T20:37:00.840 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3598 2023-07-06T20:37:00.840 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3598_vm 2023-07-06T20:37:00.840 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3599 2023-07-06T20:37:00.841 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3600 2023-07-06T20:37:00.841 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3600_fsm 2023-07-06T20:37:00.841 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3600_vm 2023-07-06T20:37:00.841 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3601 2023-07-06T20:37:00.841 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3601_fsm 2023-07-06T20:37:00.841 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3601_vm 2023-07-06T20:37:00.841 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3602 2023-07-06T20:37:00.841 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3602_fsm 2023-07-06T20:37:00.842 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3602_vm 2023-07-06T20:37:00.842 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3603 2023-07-06T20:37:00.842 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3603_fsm 2023-07-06T20:37:00.842 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3603_vm 2023-07-06T20:37:00.843 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3604 2023-07-06T20:37:00.843 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3605 2023-07-06T20:37:00.843 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3606 2023-07-06T20:37:00.844 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3607 2023-07-06T20:37:00.844 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3608 2023-07-06T20:37:00.844 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 3609 2023-07-06T20:37:00.844 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3712 2023-07-06T20:37:00.844 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3764 2023-07-06T20:37:00.844 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3764_fsm 2023-07-06T20:37:00.844 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3764_vm 2023-07-06T20:37:00.844 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3766 2023-07-06T20:37:00.845 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3767 2023-07-06T20:37:00.845 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3997 2023-07-06T20:37:00.845 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 5002 2023-07-06T20:37:00.845 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 548 2023-07-06T20:37:00.845 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 549 2023-07-06T20:37:00.845 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6102 2023-07-06T20:37:00.845 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6102_vm 2023-07-06T20:37:00.845 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6104 2023-07-06T20:37:00.846 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6104_vm 2023-07-06T20:37:00.846 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6106 2023-07-06T20:37:00.846 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6106_vm 2023-07-06T20:37:00.846 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6110 2023-07-06T20:37:00.846 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6111 2023-07-06T20:37:00.846 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6112 2023-07-06T20:37:00.846 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6113 2023-07-06T20:37:00.846 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6117 2023-07-06T20:37:00.847 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 826 2023-07-06T20:37:00.847 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 826_vm 2023-07-06T20:37:00.847 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 827 2023-07-06T20:37:00.847 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 828 2023-07-06T20:37:00.847 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 6 20:19 pg_filenode.map 2023-07-06T20:37:00.847 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 PG_VERSION 2023-07-06T20:37:00.847 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.847 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-07-06T20:37:00.848 INFO:teuthology.orchestra.run.smithi069.stdout:total 7217 2023-07-06T20:37:00.848 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Jul 6 20:19 . 2023-07-06T20:37:00.848 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jul 6 20:21 .. 2023-07-06T20:37:00.848 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 112 2023-07-06T20:37:00.848 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 113 2023-07-06T20:37:00.848 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 6 20:19 12182 2023-07-06T20:37:00.848 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12182_fsm 2023-07-06T20:37:00.849 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12182_vm 2023-07-06T20:37:00.849 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12184 2023-07-06T20:37:00.849 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12186 2023-07-06T20:37:00.849 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12187 2023-07-06T20:37:00.849 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12187_fsm 2023-07-06T20:37:00.849 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12187_vm 2023-07-06T20:37:00.849 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12189 2023-07-06T20:37:00.849 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12191 2023-07-06T20:37:00.850 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12192 2023-07-06T20:37:00.850 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12192_fsm 2023-07-06T20:37:00.850 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12192_vm 2023-07-06T20:37:00.850 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12194 2023-07-06T20:37:00.850 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12196 2023-07-06T20:37:00.850 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12197 2023-07-06T20:37:00.850 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12197_fsm 2023-07-06T20:37:00.850 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12197_vm 2023-07-06T20:37:00.851 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12199 2023-07-06T20:37:00.851 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12201 2023-07-06T20:37:00.851 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12202 2023-07-06T20:37:00.851 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12202_fsm 2023-07-06T20:37:00.851 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12202_vm 2023-07-06T20:37:00.851 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12204 2023-07-06T20:37:00.851 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12206 2023-07-06T20:37:00.851 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12207 2023-07-06T20:37:00.852 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12207_fsm 2023-07-06T20:37:00.852 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12207_vm 2023-07-06T20:37:00.852 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12209 2023-07-06T20:37:00.852 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12211 2023-07-06T20:37:00.852 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12212 2023-07-06T20:37:00.852 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12214 2023-07-06T20:37:00.852 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12216 2023-07-06T20:37:00.852 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 6 20:19 1247 2023-07-06T20:37:00.853 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1247_fsm 2023-07-06T20:37:00.853 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1247_vm 2023-07-06T20:37:00.853 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 6 20:19 1249 2023-07-06T20:37:00.853 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1249_fsm 2023-07-06T20:37:00.853 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1249_vm 2023-07-06T20:37:00.853 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 6 20:19 1255 2023-07-06T20:37:00.853 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1255_fsm 2023-07-06T20:37:00.853 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1255_vm 2023-07-06T20:37:00.854 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 6 20:19 1259 2023-07-06T20:37:00.854 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1259_fsm 2023-07-06T20:37:00.854 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1259_vm 2023-07-06T20:37:00.854 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1417 2023-07-06T20:37:00.854 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1417_vm 2023-07-06T20:37:00.854 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1418 2023-07-06T20:37:00.854 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1418_vm 2023-07-06T20:37:00.854 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 174 2023-07-06T20:37:00.855 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 175 2023-07-06T20:37:00.855 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2187 2023-07-06T20:37:00.855 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2224 2023-07-06T20:37:00.855 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2224_vm 2023-07-06T20:37:00.855 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2328 2023-07-06T20:37:00.855 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2328_vm 2023-07-06T20:37:00.855 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2336 2023-07-06T20:37:00.855 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2336_vm 2023-07-06T20:37:00.856 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2337 2023-07-06T20:37:00.856 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2600 2023-07-06T20:37:00.856 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2600_fsm 2023-07-06T20:37:00.856 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2600_vm 2023-07-06T20:37:00.856 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2601 2023-07-06T20:37:00.856 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2601_fsm 2023-07-06T20:37:00.856 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2601_vm 2023-07-06T20:37:00.857 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2602 2023-07-06T20:37:00.857 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2602_fsm 2023-07-06T20:37:00.857 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2602_vm 2023-07-06T20:37:00.857 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2603 2023-07-06T20:37:00.857 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2603_fsm 2023-07-06T20:37:00.857 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2603_vm 2023-07-06T20:37:00.857 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2604 2023-07-06T20:37:00.857 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2604_vm 2023-07-06T20:37:00.858 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2605 2023-07-06T20:37:00.858 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2605_fsm 2023-07-06T20:37:00.858 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2605_vm 2023-07-06T20:37:00.858 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2606 2023-07-06T20:37:00.858 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2606_fsm 2023-07-06T20:37:00.858 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2606_vm 2023-07-06T20:37:00.858 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2607 2023-07-06T20:37:00.858 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2607_fsm 2023-07-06T20:37:00.859 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2607_vm 2023-07-06T20:37:00.859 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 6 20:19 2608 2023-07-06T20:37:00.859 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2608_fsm 2023-07-06T20:37:00.859 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2608_vm 2023-07-06T20:37:00.859 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 6 20:19 2609 2023-07-06T20:37:00.859 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2609_fsm 2023-07-06T20:37:00.859 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2609_vm 2023-07-06T20:37:00.859 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2610 2023-07-06T20:37:00.860 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2610_fsm 2023-07-06T20:37:00.860 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2610_vm 2023-07-06T20:37:00.860 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2611 2023-07-06T20:37:00.860 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2611_vm 2023-07-06T20:37:00.860 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2612 2023-07-06T20:37:00.860 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2612_fsm 2023-07-06T20:37:00.860 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2612_vm 2023-07-06T20:37:00.860 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2613 2023-07-06T20:37:00.861 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2613_vm 2023-07-06T20:37:00.861 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2615 2023-07-06T20:37:00.861 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2615_fsm 2023-07-06T20:37:00.861 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2615_vm 2023-07-06T20:37:00.861 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2616 2023-07-06T20:37:00.861 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2616_fsm 2023-07-06T20:37:00.861 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2616_vm 2023-07-06T20:37:00.861 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 6 20:19 2617 2023-07-06T20:37:00.862 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2617_fsm 2023-07-06T20:37:00.862 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2617_vm 2023-07-06T20:37:00.862 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 6 20:19 2618 2023-07-06T20:37:00.862 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2618_fsm 2023-07-06T20:37:00.862 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2618_vm 2023-07-06T20:37:00.862 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 6 20:19 2619 2023-07-06T20:37:00.862 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2619_fsm 2023-07-06T20:37:00.862 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2619_vm 2023-07-06T20:37:00.863 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2620 2023-07-06T20:37:00.863 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2620_vm 2023-07-06T20:37:00.863 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2650 2023-07-06T20:37:00.863 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2651 2023-07-06T20:37:00.863 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2652 2023-07-06T20:37:00.863 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2653 2023-07-06T20:37:00.863 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2654 2023-07-06T20:37:00.863 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2655 2023-07-06T20:37:00.864 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2656 2023-07-06T20:37:00.864 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2657 2023-07-06T20:37:00.864 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 6 20:19 2658 2023-07-06T20:37:00.864 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 6 20:19 2659 2023-07-06T20:37:00.864 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2660 2023-07-06T20:37:00.864 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2661 2023-07-06T20:37:00.864 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2662 2023-07-06T20:37:00.864 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2663 2023-07-06T20:37:00.865 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2664 2023-07-06T20:37:00.865 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2665 2023-07-06T20:37:00.865 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2666 2023-07-06T20:37:00.865 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2667 2023-07-06T20:37:00.865 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2668 2023-07-06T20:37:00.865 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2669 2023-07-06T20:37:00.865 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2670 2023-07-06T20:37:00.866 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 335872 Jul 6 20:19 2673 2023-07-06T20:37:00.866 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Jul 6 20:19 2674 2023-07-06T20:37:00.866 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 6 20:19 2675 2023-07-06T20:37:00.866 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2678 2023-07-06T20:37:00.866 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2679 2023-07-06T20:37:00.866 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2680 2023-07-06T20:37:00.866 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2681 2023-07-06T20:37:00.866 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2682 2023-07-06T20:37:00.867 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2683 2023-07-06T20:37:00.867 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2684 2023-07-06T20:37:00.867 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2685 2023-07-06T20:37:00.867 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2686 2023-07-06T20:37:00.867 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2687 2023-07-06T20:37:00.867 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2688 2023-07-06T20:37:00.867 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2689 2023-07-06T20:37:00.867 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 6 20:19 2690 2023-07-06T20:37:00.868 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 6 20:19 2691 2023-07-06T20:37:00.868 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2692 2023-07-06T20:37:00.868 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2693 2023-07-06T20:37:00.868 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2696 2023-07-06T20:37:00.868 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2699 2023-07-06T20:37:00.868 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2701 2023-07-06T20:37:00.868 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2702 2023-07-06T20:37:00.868 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2703 2023-07-06T20:37:00.869 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2704 2023-07-06T20:37:00.869 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2753 2023-07-06T20:37:00.869 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2753_fsm 2023-07-06T20:37:00.869 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2753_vm 2023-07-06T20:37:00.869 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2754 2023-07-06T20:37:00.869 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2755 2023-07-06T20:37:00.869 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2756 2023-07-06T20:37:00.869 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2757 2023-07-06T20:37:00.870 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2830 2023-07-06T20:37:00.870 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2830_vm 2023-07-06T20:37:00.870 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2831 2023-07-06T20:37:00.870 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2832 2023-07-06T20:37:00.870 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2832_vm 2023-07-06T20:37:00.870 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2833 2023-07-06T20:37:00.870 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2834 2023-07-06T20:37:00.870 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2834_vm 2023-07-06T20:37:00.871 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2835 2023-07-06T20:37:00.871 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2836 2023-07-06T20:37:00.871 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2836_vm 2023-07-06T20:37:00.871 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2837 2023-07-06T20:37:00.871 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 6 20:19 2838 2023-07-06T20:37:00.871 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2838_fsm 2023-07-06T20:37:00.871 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2838_vm 2023-07-06T20:37:00.871 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2839 2023-07-06T20:37:00.872 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2840 2023-07-06T20:37:00.872 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2840_fsm 2023-07-06T20:37:00.872 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2840_vm 2023-07-06T20:37:00.872 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2841 2023-07-06T20:37:00.872 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2995 2023-07-06T20:37:00.872 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2995_vm 2023-07-06T20:37:00.872 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2996 2023-07-06T20:37:00.872 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3079 2023-07-06T20:37:00.873 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3079_fsm 2023-07-06T20:37:00.873 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3079_vm 2023-07-06T20:37:00.873 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3080 2023-07-06T20:37:00.873 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3081 2023-07-06T20:37:00.873 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3085 2023-07-06T20:37:00.873 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3118 2023-07-06T20:37:00.873 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3118_vm 2023-07-06T20:37:00.873 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3119 2023-07-06T20:37:00.874 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3164 2023-07-06T20:37:00.874 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3256 2023-07-06T20:37:00.874 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3256_vm 2023-07-06T20:37:00.874 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3257 2023-07-06T20:37:00.874 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3258 2023-07-06T20:37:00.874 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3350 2023-07-06T20:37:00.874 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3350_vm 2023-07-06T20:37:00.874 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3351 2023-07-06T20:37:00.875 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3379 2023-07-06T20:37:00.875 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3380 2023-07-06T20:37:00.875 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3381 2023-07-06T20:37:00.875 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3381_vm 2023-07-06T20:37:00.875 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3394 2023-07-06T20:37:00.875 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3394_fsm 2023-07-06T20:37:00.875 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3394_vm 2023-07-06T20:37:00.875 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3395 2023-07-06T20:37:00.876 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3439 2023-07-06T20:37:00.876 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3439_vm 2023-07-06T20:37:00.876 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3440 2023-07-06T20:37:00.876 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 3455 2023-07-06T20:37:00.876 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3456 2023-07-06T20:37:00.876 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3456_fsm 2023-07-06T20:37:00.876 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3456_vm 2023-07-06T20:37:00.877 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3466 2023-07-06T20:37:00.877 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3466_vm 2023-07-06T20:37:00.877 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3467 2023-07-06T20:37:00.877 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3468 2023-07-06T20:37:00.877 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3501 2023-07-06T20:37:00.877 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3501_vm 2023-07-06T20:37:00.877 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3502 2023-07-06T20:37:00.877 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3503 2023-07-06T20:37:00.878 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3534 2023-07-06T20:37:00.878 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3541 2023-07-06T20:37:00.878 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3541_fsm 2023-07-06T20:37:00.878 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3541_vm 2023-07-06T20:37:00.878 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3542 2023-07-06T20:37:00.878 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3574 2023-07-06T20:37:00.878 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3575 2023-07-06T20:37:00.878 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3576 2023-07-06T20:37:00.879 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3576_vm 2023-07-06T20:37:00.879 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3596 2023-07-06T20:37:00.879 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3596_vm 2023-07-06T20:37:00.879 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3597 2023-07-06T20:37:00.879 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3598 2023-07-06T20:37:00.879 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3598_vm 2023-07-06T20:37:00.879 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3599 2023-07-06T20:37:00.879 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3600 2023-07-06T20:37:00.880 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3600_fsm 2023-07-06T20:37:00.880 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3600_vm 2023-07-06T20:37:00.880 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3601 2023-07-06T20:37:00.880 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3601_fsm 2023-07-06T20:37:00.880 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3601_vm 2023-07-06T20:37:00.880 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3602 2023-07-06T20:37:00.880 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3602_fsm 2023-07-06T20:37:00.880 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3602_vm 2023-07-06T20:37:00.881 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3603 2023-07-06T20:37:00.881 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3603_fsm 2023-07-06T20:37:00.881 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3603_vm 2023-07-06T20:37:00.881 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3604 2023-07-06T20:37:00.881 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3605 2023-07-06T20:37:00.881 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3606 2023-07-06T20:37:00.881 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3607 2023-07-06T20:37:00.881 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3608 2023-07-06T20:37:00.883 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 3609 2023-07-06T20:37:00.883 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3712 2023-07-06T20:37:00.883 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3764 2023-07-06T20:37:00.883 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3764_fsm 2023-07-06T20:37:00.884 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3764_vm 2023-07-06T20:37:00.884 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3766 2023-07-06T20:37:00.884 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3767 2023-07-06T20:37:00.884 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3997 2023-07-06T20:37:00.884 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 5002 2023-07-06T20:37:00.884 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 548 2023-07-06T20:37:00.884 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 549 2023-07-06T20:37:00.885 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6102 2023-07-06T20:37:00.885 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6102_vm 2023-07-06T20:37:00.885 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6104 2023-07-06T20:37:00.885 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6104_vm 2023-07-06T20:37:00.885 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6106 2023-07-06T20:37:00.885 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6106_vm 2023-07-06T20:37:00.885 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6110 2023-07-06T20:37:00.885 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6111 2023-07-06T20:37:00.886 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6112 2023-07-06T20:37:00.886 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6113 2023-07-06T20:37:00.886 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6117 2023-07-06T20:37:00.886 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 826 2023-07-06T20:37:00.886 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 826_vm 2023-07-06T20:37:00.886 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 827 2023-07-06T20:37:00.886 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 828 2023-07-06T20:37:00.886 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 6 20:19 pg_filenode.map 2023-07-06T20:37:00.887 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 PG_VERSION 2023-07-06T20:37:00.887 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.887 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-07-06T20:37:00.887 INFO:teuthology.orchestra.run.smithi069.stdout:total 7857216 2023-07-06T20:37:00.887 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 315 Jul 6 20:22 . 2023-07-06T20:37:00.887 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jul 6 20:21 .. 2023-07-06T20:37:00.887 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 112 2023-07-06T20:37:00.887 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 113 2023-07-06T20:37:00.888 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 6 20:19 12182 2023-07-06T20:37:00.888 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12182_fsm 2023-07-06T20:37:00.888 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12182_vm 2023-07-06T20:37:00.888 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12184 2023-07-06T20:37:00.888 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12186 2023-07-06T20:37:00.888 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12187 2023-07-06T20:37:00.888 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12187_fsm 2023-07-06T20:37:00.889 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12187_vm 2023-07-06T20:37:00.889 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12189 2023-07-06T20:37:00.889 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12191 2023-07-06T20:37:00.889 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12192 2023-07-06T20:37:00.889 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12192_fsm 2023-07-06T20:37:00.889 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12192_vm 2023-07-06T20:37:00.889 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12194 2023-07-06T20:37:00.889 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12196 2023-07-06T20:37:00.890 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12197 2023-07-06T20:37:00.890 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12197_fsm 2023-07-06T20:37:00.890 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12197_vm 2023-07-06T20:37:00.890 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12199 2023-07-06T20:37:00.890 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12201 2023-07-06T20:37:00.890 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12202 2023-07-06T20:37:00.890 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12202_fsm 2023-07-06T20:37:00.890 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12202_vm 2023-07-06T20:37:00.891 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12204 2023-07-06T20:37:00.891 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12206 2023-07-06T20:37:00.891 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12207 2023-07-06T20:37:00.891 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 12207_fsm 2023-07-06T20:37:00.891 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12207_vm 2023-07-06T20:37:00.891 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12209 2023-07-06T20:37:00.891 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12211 2023-07-06T20:37:00.891 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12212 2023-07-06T20:37:00.892 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 12214 2023-07-06T20:37:00.892 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 12216 2023-07-06T20:37:00.892 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 6 20:19 1247 2023-07-06T20:37:00.892 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1247_fsm 2023-07-06T20:37:00.892 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1247_vm 2023-07-06T20:37:00.892 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 6 20:21 1249 2023-07-06T20:37:00.892 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1249_fsm 2023-07-06T20:37:00.892 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1249_vm 2023-07-06T20:37:00.893 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 6 20:19 1255 2023-07-06T20:37:00.893 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1255_fsm 2023-07-06T20:37:00.893 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1255_vm 2023-07-06T20:37:00.893 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 6 20:36 1259 2023-07-06T20:37:00.893 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1259_fsm 2023-07-06T20:37:00.893 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1259_vm 2023-07-06T20:37:00.893 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1417 2023-07-06T20:37:00.893 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1417_vm 2023-07-06T20:37:00.894 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1418 2023-07-06T20:37:00.894 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 1418_vm 2023-07-06T20:37:00.894 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1744896 Jul 6 20:36 16387 2023-07-06T20:37:00.894 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:36 16387_fsm 2023-07-06T20:37:00.894 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:36 16387_vm 2023-07-06T20:37:00.894 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1490944 Jul 6 20:36 16393 2023-07-06T20:37:00.894 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:36 16393_fsm 2023-07-06T20:37:00.894 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:36 16393_vm 2023-07-06T20:37:00.895 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 6 20:36 16396 2023-07-06T20:37:00.895 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 6 20:36 16396.1 2023-07-06T20:37:00.895 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 6 20:36 16396.2 2023-07-06T20:37:00.895 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 6 20:36 16396.3 2023-07-06T20:37:00.895 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 6 20:36 16396.4 2023-07-06T20:37:00.895 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 6 20:36 16396.5 2023-07-06T20:37:00.895 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 372203520 Jul 6 20:36 16396.6 2023-07-06T20:37:00.895 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1695744 Jul 6 20:36 16396_fsm 2023-07-06T20:37:00.896 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 Jul 6 20:36 16396_vm 2023-07-06T20:37:00.896 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 6 20:36 16397 2023-07-06T20:37:00.896 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Jul 6 20:36 16399 2023-07-06T20:37:00.896 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 6 20:36 16401 2023-07-06T20:37:00.896 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49373184 Jul 6 20:36 16401.1 2023-07-06T20:37:00.896 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 94789632 Jul 6 20:36 16403 2023-07-06T20:37:00.896 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:36 16403_fsm 2023-07-06T20:37:00.897 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 174 2023-07-06T20:37:00.897 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 175 2023-07-06T20:37:00.897 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2187 2023-07-06T20:37:00.897 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2224 2023-07-06T20:37:00.897 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2224_vm 2023-07-06T20:37:00.897 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2328 2023-07-06T20:37:00.897 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2328_vm 2023-07-06T20:37:00.897 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2336 2023-07-06T20:37:00.898 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2336_vm 2023-07-06T20:37:00.898 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2337 2023-07-06T20:37:00.898 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2600 2023-07-06T20:37:00.898 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2600_fsm 2023-07-06T20:37:00.898 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2600_vm 2023-07-06T20:37:00.898 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2601 2023-07-06T20:37:00.898 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2601_fsm 2023-07-06T20:37:00.898 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2601_vm 2023-07-06T20:37:00.899 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2602 2023-07-06T20:37:00.899 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2602_fsm 2023-07-06T20:37:00.899 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2602_vm 2023-07-06T20:37:00.899 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2603 2023-07-06T20:37:00.899 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2603_fsm 2023-07-06T20:37:00.899 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2603_vm 2023-07-06T20:37:00.899 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2604 2023-07-06T20:37:00.899 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2604_vm 2023-07-06T20:37:00.900 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2605 2023-07-06T20:37:00.900 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2605_fsm 2023-07-06T20:37:00.900 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2605_vm 2023-07-06T20:37:00.900 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:21 2606 2023-07-06T20:37:00.900 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2606_fsm 2023-07-06T20:37:00.900 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:21 2606_vm 2023-07-06T20:37:00.900 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2607 2023-07-06T20:37:00.900 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2607_fsm 2023-07-06T20:37:00.901 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2607_vm 2023-07-06T20:37:00.901 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Jul 6 20:21 2608 2023-07-06T20:37:00.901 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:21 2608_fsm 2023-07-06T20:37:00.901 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:21 2608_vm 2023-07-06T20:37:00.901 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 6 20:19 2609 2023-07-06T20:37:00.901 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2609_fsm 2023-07-06T20:37:00.901 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2609_vm 2023-07-06T20:37:00.901 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:21 2610 2023-07-06T20:37:00.902 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:21 2610_fsm 2023-07-06T20:37:00.902 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:21 2610_vm 2023-07-06T20:37:00.902 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2611 2023-07-06T20:37:00.902 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2611_vm 2023-07-06T20:37:00.902 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2612 2023-07-06T20:37:00.902 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2612_fsm 2023-07-06T20:37:00.902 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2612_vm 2023-07-06T20:37:00.902 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2613 2023-07-06T20:37:00.903 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2613_vm 2023-07-06T20:37:00.903 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2615 2023-07-06T20:37:00.903 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2615_fsm 2023-07-06T20:37:00.903 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2615_vm 2023-07-06T20:37:00.903 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2616 2023-07-06T20:37:00.903 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2616_fsm 2023-07-06T20:37:00.903 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2616_vm 2023-07-06T20:37:00.903 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 6 20:19 2617 2023-07-06T20:37:00.904 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2617_fsm 2023-07-06T20:37:00.904 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2617_vm 2023-07-06T20:37:00.904 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 6 20:19 2618 2023-07-06T20:37:00.904 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2618_fsm 2023-07-06T20:37:00.904 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2618_vm 2023-07-06T20:37:00.904 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Jul 6 20:36 2619 2023-07-06T20:37:00.904 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:22 2619_fsm 2023-07-06T20:37:00.904 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:21 2619_vm 2023-07-06T20:37:00.905 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2620 2023-07-06T20:37:00.905 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2620_vm 2023-07-06T20:37:00.905 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2650 2023-07-06T20:37:00.905 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2651 2023-07-06T20:37:00.905 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2652 2023-07-06T20:37:00.905 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2653 2023-07-06T20:37:00.905 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2654 2023-07-06T20:37:00.905 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2655 2023-07-06T20:37:00.906 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2656 2023-07-06T20:37:00.906 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2657 2023-07-06T20:37:00.906 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jul 6 20:21 2658 2023-07-06T20:37:00.906 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 6 20:21 2659 2023-07-06T20:37:00.906 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2660 2023-07-06T20:37:00.906 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2661 2023-07-06T20:37:00.906 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:21 2662 2023-07-06T20:37:00.906 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:21 2663 2023-07-06T20:37:00.907 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:21 2664 2023-07-06T20:37:00.907 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:21 2665 2023-07-06T20:37:00.907 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:21 2666 2023-07-06T20:37:00.907 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:21 2667 2023-07-06T20:37:00.907 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2668 2023-07-06T20:37:00.907 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2669 2023-07-06T20:37:00.907 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2670 2023-07-06T20:37:00.907 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 335872 Jul 6 20:21 2673 2023-07-06T20:37:00.908 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Jul 6 20:21 2674 2023-07-06T20:37:00.908 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 6 20:19 2675 2023-07-06T20:37:00.908 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:21 2678 2023-07-06T20:37:00.908 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:21 2679 2023-07-06T20:37:00.908 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2680 2023-07-06T20:37:00.908 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2681 2023-07-06T20:37:00.908 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2682 2023-07-06T20:37:00.909 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2683 2023-07-06T20:37:00.909 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2684 2023-07-06T20:37:00.909 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2685 2023-07-06T20:37:00.909 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2686 2023-07-06T20:37:00.909 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2687 2023-07-06T20:37:00.909 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2688 2023-07-06T20:37:00.909 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 6 20:19 2689 2023-07-06T20:37:00.909 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 6 20:19 2690 2023-07-06T20:37:00.910 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 6 20:19 2691 2023-07-06T20:37:00.910 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2692 2023-07-06T20:37:00.910 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2693 2023-07-06T20:37:00.910 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:23 2696 2023-07-06T20:37:00.910 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2699 2023-07-06T20:37:00.910 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2701 2023-07-06T20:37:00.910 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2702 2023-07-06T20:37:00.910 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2703 2023-07-06T20:37:00.911 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:19 2704 2023-07-06T20:37:00.911 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2753 2023-07-06T20:37:00.911 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2753_fsm 2023-07-06T20:37:00.911 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2753_vm 2023-07-06T20:37:00.911 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2754 2023-07-06T20:37:00.911 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2755 2023-07-06T20:37:00.911 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2756 2023-07-06T20:37:00.911 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 2757 2023-07-06T20:37:00.912 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2830 2023-07-06T20:37:00.912 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2830_vm 2023-07-06T20:37:00.912 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2831 2023-07-06T20:37:00.912 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2832 2023-07-06T20:37:00.912 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2832_vm 2023-07-06T20:37:00.912 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2833 2023-07-06T20:37:00.912 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2834 2023-07-06T20:37:00.912 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2834_vm 2023-07-06T20:37:00.913 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2835 2023-07-06T20:37:00.913 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2836 2023-07-06T20:37:00.913 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2836_vm 2023-07-06T20:37:00.913 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2837 2023-07-06T20:37:00.913 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 6 20:19 2838 2023-07-06T20:37:00.913 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2838_fsm 2023-07-06T20:37:00.913 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2838_vm 2023-07-06T20:37:00.913 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2839 2023-07-06T20:37:00.914 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2840 2023-07-06T20:37:00.914 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2840_fsm 2023-07-06T20:37:00.914 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2840_vm 2023-07-06T20:37:00.914 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2841 2023-07-06T20:37:00.914 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2995 2023-07-06T20:37:00.914 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2995_vm 2023-07-06T20:37:00.914 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2996 2023-07-06T20:37:00.914 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3079 2023-07-06T20:37:00.915 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3079_fsm 2023-07-06T20:37:00.915 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3079_vm 2023-07-06T20:37:00.915 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3080 2023-07-06T20:37:00.915 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3081 2023-07-06T20:37:00.915 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3085 2023-07-06T20:37:00.915 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3118 2023-07-06T20:37:00.915 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3118_vm 2023-07-06T20:37:00.915 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3119 2023-07-06T20:37:00.916 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3164 2023-07-06T20:37:00.916 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3256 2023-07-06T20:37:00.916 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3256_vm 2023-07-06T20:37:00.916 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3257 2023-07-06T20:37:00.916 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3258 2023-07-06T20:37:00.916 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3350 2023-07-06T20:37:00.916 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3350_vm 2023-07-06T20:37:00.916 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3351 2023-07-06T20:37:00.917 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3379 2023-07-06T20:37:00.917 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3380 2023-07-06T20:37:00.917 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3381 2023-07-06T20:37:00.917 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3381_vm 2023-07-06T20:37:00.917 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3394 2023-07-06T20:37:00.917 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3394_fsm 2023-07-06T20:37:00.917 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3394_vm 2023-07-06T20:37:00.917 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3395 2023-07-06T20:37:00.918 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3439 2023-07-06T20:37:00.918 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3439_vm 2023-07-06T20:37:00.918 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3440 2023-07-06T20:37:00.918 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 6 20:21 3455 2023-07-06T20:37:00.918 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3456 2023-07-06T20:37:00.918 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3456_fsm 2023-07-06T20:37:00.918 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3456_vm 2023-07-06T20:37:00.918 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3466 2023-07-06T20:37:00.919 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3466_vm 2023-07-06T20:37:00.919 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3467 2023-07-06T20:37:00.919 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3468 2023-07-06T20:37:00.919 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3501 2023-07-06T20:37:00.919 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3501_vm 2023-07-06T20:37:00.919 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3502 2023-07-06T20:37:00.919 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3503 2023-07-06T20:37:00.919 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3534 2023-07-06T20:37:00.920 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3541 2023-07-06T20:37:00.920 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3541_fsm 2023-07-06T20:37:00.920 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3541_vm 2023-07-06T20:37:00.920 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3542 2023-07-06T20:37:00.920 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3574 2023-07-06T20:37:00.920 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3575 2023-07-06T20:37:00.920 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3576 2023-07-06T20:37:00.920 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3576_vm 2023-07-06T20:37:00.921 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3596 2023-07-06T20:37:00.921 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3596_vm 2023-07-06T20:37:00.921 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3597 2023-07-06T20:37:00.921 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3598 2023-07-06T20:37:00.921 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3598_vm 2023-07-06T20:37:00.921 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3599 2023-07-06T20:37:00.921 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3600 2023-07-06T20:37:00.921 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3600_fsm 2023-07-06T20:37:00.922 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3600_vm 2023-07-06T20:37:00.922 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3601 2023-07-06T20:37:00.922 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3601_fsm 2023-07-06T20:37:00.922 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3601_vm 2023-07-06T20:37:00.922 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3602 2023-07-06T20:37:00.922 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3602_fsm 2023-07-06T20:37:00.922 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3602_vm 2023-07-06T20:37:00.922 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3603 2023-07-06T20:37:00.923 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3603_fsm 2023-07-06T20:37:00.923 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3603_vm 2023-07-06T20:37:00.923 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3604 2023-07-06T20:37:00.923 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3605 2023-07-06T20:37:00.923 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3606 2023-07-06T20:37:00.923 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3607 2023-07-06T20:37:00.923 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3608 2023-07-06T20:37:00.923 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 6 20:19 3609 2023-07-06T20:37:00.924 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3712 2023-07-06T20:37:00.924 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3764 2023-07-06T20:37:00.924 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 3764_fsm 2023-07-06T20:37:00.924 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3764_vm 2023-07-06T20:37:00.924 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3766 2023-07-06T20:37:00.924 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 3767 2023-07-06T20:37:00.924 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3997 2023-07-06T20:37:00.924 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 5002 2023-07-06T20:37:00.925 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 548 2023-07-06T20:37:00.925 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 549 2023-07-06T20:37:00.925 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6102 2023-07-06T20:37:00.925 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6102_vm 2023-07-06T20:37:00.925 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6104 2023-07-06T20:37:00.925 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6104_vm 2023-07-06T20:37:00.925 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6106 2023-07-06T20:37:00.926 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6106_vm 2023-07-06T20:37:00.926 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6110 2023-07-06T20:37:00.926 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6111 2023-07-06T20:37:00.926 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6112 2023-07-06T20:37:00.926 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6113 2023-07-06T20:37:00.926 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6117 2023-07-06T20:37:00.926 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 826 2023-07-06T20:37:00.926 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 826_vm 2023-07-06T20:37:00.927 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 827 2023-07-06T20:37:00.927 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 828 2023-07-06T20:37:00.927 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 6 20:19 pg_filenode.map 2023-07-06T20:37:00.927 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Jul 6 20:19 pg_internal.init 2023-07-06T20:37:00.927 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:19 PG_VERSION 2023-07-06T20:37:00.927 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.927 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-07-06T20:37:00.927 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.928 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:21 . 2023-07-06T20:37:00.928 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jul 6 20:21 .. 2023-07-06T20:37:00.928 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.928 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/global: 2023-07-06T20:37:00.928 INFO:teuthology.orchestra.run.smithi069.stdout:total 574 2023-07-06T20:37:00.928 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Jul 6 20:19 . 2023-07-06T20:37:00.928 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.928 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1136 2023-07-06T20:37:00.929 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1136_fsm 2023-07-06T20:37:00.929 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1136_vm 2023-07-06T20:37:00.929 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 1137 2023-07-06T20:37:00.929 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1213 2023-07-06T20:37:00.929 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1213_fsm 2023-07-06T20:37:00.929 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1213_vm 2023-07-06T20:37:00.929 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1214 2023-07-06T20:37:00.929 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1214_fsm 2023-07-06T20:37:00.930 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1214_vm 2023-07-06T20:37:00.930 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 1232 2023-07-06T20:37:00.930 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 1233 2023-07-06T20:37:00.930 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1260 2023-07-06T20:37:00.930 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1260_fsm 2023-07-06T20:37:00.930 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1260_vm 2023-07-06T20:37:00.930 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1261 2023-07-06T20:37:00.930 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1261_fsm 2023-07-06T20:37:00.931 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1261_vm 2023-07-06T20:37:00.931 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1262 2023-07-06T20:37:00.931 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 1262_fsm 2023-07-06T20:37:00.931 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 1262_vm 2023-07-06T20:37:00.931 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2396 2023-07-06T20:37:00.931 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 6 20:19 2396_fsm 2023-07-06T20:37:00.931 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2396_vm 2023-07-06T20:37:00.931 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2397 2023-07-06T20:37:00.932 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2671 2023-07-06T20:37:00.932 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2672 2023-07-06T20:37:00.932 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2676 2023-07-06T20:37:00.932 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2677 2023-07-06T20:37:00.932 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2694 2023-07-06T20:37:00.932 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2695 2023-07-06T20:37:00.932 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2697 2023-07-06T20:37:00.932 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 6 20:19 2698 2023-07-06T20:37:00.933 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2846 2023-07-06T20:37:00.933 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2846_vm 2023-07-06T20:37:00.933 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2847 2023-07-06T20:37:00.933 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2964 2023-07-06T20:37:00.933 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2964_vm 2023-07-06T20:37:00.933 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2965 2023-07-06T20:37:00.933 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2966 2023-07-06T20:37:00.933 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 2966_vm 2023-07-06T20:37:00.934 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 2967 2023-07-06T20:37:00.934 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3592 2023-07-06T20:37:00.934 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 3592_vm 2023-07-06T20:37:00.934 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 3593 2023-07-06T20:37:00.934 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 4060 2023-07-06T20:37:00.934 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 4060_vm 2023-07-06T20:37:00.934 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 4061 2023-07-06T20:37:00.934 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6000 2023-07-06T20:37:00.935 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6000_vm 2023-07-06T20:37:00.935 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6001 2023-07-06T20:37:00.935 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6002 2023-07-06T20:37:00.935 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6100 2023-07-06T20:37:00.935 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 6100_vm 2023-07-06T20:37:00.935 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6114 2023-07-06T20:37:00.935 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 6115 2023-07-06T20:37:00.935 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:37 pg_control 2023-07-06T20:37:00.936 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 6 20:19 pg_filenode.map 2023-07-06T20:37:00.936 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Jul 6 20:19 pg_internal.init 2023-07-06T20:37:00.936 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.936 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/log: 2023-07-06T20:37:00.936 INFO:teuthology.orchestra.run.smithi069.stdout:total 14 2023-07-06T20:37:00.936 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 . 2023-07-06T20:37:00.936 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.937 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 13875 Jul 6 20:37 postgresql-Thu.log 2023-07-06T20:37:00.937 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.937 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-07-06T20:37:00.937 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.937 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:37:00.937 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.937 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.937 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-07-06T20:37:00.938 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.938 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:37:00.938 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.938 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.938 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-07-06T20:37:00.938 INFO:teuthology.orchestra.run.smithi069.stdout:total 1 2023-07-06T20:37:00.938 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:37 . 2023-07-06T20:37:00.938 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.939 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 mappings 2023-07-06T20:37:00.939 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 6 20:37 replorigin_checkpoint 2023-07-06T20:37:00.939 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 snapshots 2023-07-06T20:37:00.939 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.939 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-07-06T20:37:00.939 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.939 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:37:00.939 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:37 .. 2023-07-06T20:37:00.940 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.940 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-07-06T20:37:00.940 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.940 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:37:00.940 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:37 .. 2023-07-06T20:37:00.940 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.940 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-07-06T20:37:00.941 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.941 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:19 . 2023-07-06T20:37:00.941 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.941 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 members 2023-07-06T20:37:00.941 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 offsets 2023-07-06T20:37:00.941 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.941 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-07-06T20:37:00.941 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2023-07-06T20:37:00.942 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 . 2023-07-06T20:37:00.942 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:19 .. 2023-07-06T20:37:00.942 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 0000 2023-07-06T20:37:00.942 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.942 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-07-06T20:37:00.942 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2023-07-06T20:37:00.942 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 . 2023-07-06T20:37:00.942 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:19 .. 2023-07-06T20:37:00.943 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 0000 2023-07-06T20:37:00.943 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.943 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-07-06T20:37:00.943 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2023-07-06T20:37:00.943 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:19 . 2023-07-06T20:37:00.943 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.943 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 6 20:19 0000 2023-07-06T20:37:00.943 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.944 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-07-06T20:37:00.944 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.944 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:37:00.944 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.944 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.944 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-07-06T20:37:00.944 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.944 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:37:00.945 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.945 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-07-06T20:37:00.945 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.945 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:37:00.945 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.945 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-07-06T20:37:00.946 INFO:teuthology.orchestra.run.smithi069.stdout:total 13 2023-07-06T20:37:00.946 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 6 20:37 . 2023-07-06T20:37:00.946 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.946 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Jul 6 20:37 db_0.stat 2023-07-06T20:37:00.946 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Jul 6 20:37 db_12347.stat 2023-07-06T20:37:00.946 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Jul 6 20:37 global.stat 2023-07-06T20:37:00.946 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.946 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-07-06T20:37:00.947 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.947 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:37 . 2023-07-06T20:37:00.947 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.947 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.947 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-07-06T20:37:00.947 INFO:teuthology.orchestra.run.smithi069.stdout:total 120 2023-07-06T20:37:00.947 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jul 6 20:36 . 2023-07-06T20:37:00.947 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.948 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 6 20:36 001B 2023-07-06T20:37:00.948 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.948 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-07-06T20:37:00.948 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.948 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:37:00.948 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.948 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.948 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-07-06T20:37:00.949 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.949 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:37:00.949 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.949 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.949 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-07-06T20:37:00.949 INFO:teuthology.orchestra.run.smithi069.stdout:total 1048576 2023-07-06T20:37:00.949 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Jul 6 20:37 . 2023-07-06T20:37:00.949 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.950 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000041 2023-07-06T20:37:00.950 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:37 000000010000000600000042 2023-07-06T20:37:00.950 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 000000010000000600000043 2023-07-06T20:37:00.950 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 000000010000000600000044 2023-07-06T20:37:00.950 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 000000010000000600000045 2023-07-06T20:37:00.950 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 000000010000000600000046 2023-07-06T20:37:00.950 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 000000010000000600000047 2023-07-06T20:37:00.950 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 000000010000000600000048 2023-07-06T20:37:00.951 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 000000010000000600000049 2023-07-06T20:37:00.951 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 00000001000000060000004A 2023-07-06T20:37:00.951 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 00000001000000060000004B 2023-07-06T20:37:00.951 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 00000001000000060000004C 2023-07-06T20:37:00.951 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 00000001000000060000004D 2023-07-06T20:37:00.951 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 00000001000000060000004E 2023-07-06T20:37:00.951 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 00000001000000060000004F 2023-07-06T20:37:00.952 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000050 2023-07-06T20:37:00.952 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000051 2023-07-06T20:37:00.952 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 000000010000000600000052 2023-07-06T20:37:00.952 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000053 2023-07-06T20:37:00.952 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000054 2023-07-06T20:37:00.952 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000055 2023-07-06T20:37:00.952 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 000000010000000600000056 2023-07-06T20:37:00.952 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 000000010000000600000057 2023-07-06T20:37:00.953 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000058 2023-07-06T20:37:00.953 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000059 2023-07-06T20:37:00.953 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000005A 2023-07-06T20:37:00.953 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000005B 2023-07-06T20:37:00.953 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000005C 2023-07-06T20:37:00.953 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000005D 2023-07-06T20:37:00.953 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000005E 2023-07-06T20:37:00.953 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 00000001000000060000005F 2023-07-06T20:37:00.954 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000060 2023-07-06T20:37:00.954 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 000000010000000600000061 2023-07-06T20:37:00.954 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000062 2023-07-06T20:37:00.954 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000063 2023-07-06T20:37:00.954 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:35 000000010000000600000064 2023-07-06T20:37:00.954 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000065 2023-07-06T20:37:00.954 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000066 2023-07-06T20:37:00.954 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000067 2023-07-06T20:37:00.955 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000068 2023-07-06T20:37:00.955 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000069 2023-07-06T20:37:00.955 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000006A 2023-07-06T20:37:00.955 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000006B 2023-07-06T20:37:00.955 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000006C 2023-07-06T20:37:00.955 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000006D 2023-07-06T20:37:00.955 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000006E 2023-07-06T20:37:00.955 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000006F 2023-07-06T20:37:00.956 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000070 2023-07-06T20:37:00.956 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000071 2023-07-06T20:37:00.956 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000072 2023-07-06T20:37:00.956 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000073 2023-07-06T20:37:00.956 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000074 2023-07-06T20:37:00.956 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000075 2023-07-06T20:37:00.956 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000076 2023-07-06T20:37:00.956 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000077 2023-07-06T20:37:00.957 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000078 2023-07-06T20:37:00.957 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000079 2023-07-06T20:37:00.957 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000007A 2023-07-06T20:37:00.957 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000007B 2023-07-06T20:37:00.957 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000007C 2023-07-06T20:37:00.957 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000007D 2023-07-06T20:37:00.957 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000007E 2023-07-06T20:37:00.957 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 00000001000000060000007F 2023-07-06T20:37:00.958 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 6 20:36 000000010000000600000080 2023-07-06T20:37:00.958 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 archive_status 2023-07-06T20:37:00.958 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.958 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-07-06T20:37:00.958 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2023-07-06T20:37:00.958 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 6 20:19 . 2023-07-06T20:37:00.958 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Jul 6 20:37 .. 2023-07-06T20:37:00.958 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:00.959 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-07-06T20:37:00.959 INFO:teuthology.orchestra.run.smithi069.stdout:total 440 2023-07-06T20:37:00.959 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jul 6 20:29 . 2023-07-06T20:37:00.959 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jul 6 20:37 .. 2023-07-06T20:37:00.959 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Jul 6 20:29 0000 2023-07-06T20:37:00.959 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 Jul 6 20:36 0001 2023-07-06T20:37:00.960 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:00 smithi069 ceph-mon[126399]: pgmap v858: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 2.29k op/s 2023-07-06T20:37:00.961 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-06T20:37:00.970 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-06T20:37:00.970 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-07-06T20:37:00.971 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-06T20:37:00.996 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:37:01.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:00 smithi156 ceph-mon[129280]: pgmap v858: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 2.29k op/s 2023-07-06T20:37:01.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:00 smithi098 ceph-mon[128968]: pgmap v858: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 2.29k op/s 2023-07-06T20:37:01.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.193+0000 7fd1f5bff700 1 -- 172.21.15.69:0/167310648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f00feb00 msgr2=0x7fd1f00fef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:01.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.193+0000 7fd1f5bff700 1 --2- 172.21.15.69:0/167310648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f00feb00 0x7fd1f00fef10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009990 tx=0x7fd1e0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.193+0000 7fd1f5bff700 1 -- 172.21.15.69:0/167310648 shutdown_connections 2023-07-06T20:37:01.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.193+0000 7fd1f5bff700 1 --2- 172.21.15.69:0/167310648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1f0100680 0x7fd1f0106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.196 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.193+0000 7fd1f5bff700 1 --2- 172.21.15.69:0/167310648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f00feb00 0x7fd1f00fef10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.193+0000 7fd1f5bff700 1 --2- 172.21.15.69:0/167310648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f00ffcf0 0x7fd1f0100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.193+0000 7fd1f5bff700 1 -- 172.21.15.69:0/167310648 >> 172.21.15.69:0/167310648 conn(0x7fd1f00fa100 msgr2=0x7fd1f00fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.193+0000 7fd1f5bff700 1 -- 172.21.15.69:0/167310648 shutdown_connections 2023-07-06T20:37:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.193+0000 7fd1f5bff700 1 -- 172.21.15.69:0/167310648 wait complete. 2023-07-06T20:37:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.194+0000 7fd1f5bff700 1 Processor -- start 2023-07-06T20:37:01.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.195+0000 7fd1f5bff700 1 -- start start 2023-07-06T20:37:01.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.195+0000 7fd1f5bff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f00feb00 0x7fd1f0102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:01.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.195+0000 7fd1f5bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f00ffcf0 0x7fd1f01010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:01.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.195+0000 7fd1f5bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1f0100680 0x7fd1f0101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:01.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.195+0000 7fd1f5bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0101db0 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.195+0000 7fd1f5bff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0101f20 con 0x7fd1f00feb00 2023-07-06T20:37:01.199 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.195+0000 7fd1f5bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f019f9d0 con 0x7fd1f0100680 2023-07-06T20:37:01.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.195+0000 7fd1ef7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f00feb00 0x7fd1f0102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:01.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.195+0000 7fd1ef7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f00feb00 0x7fd1f0102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:46972/0 (socket says 172.21.15.69:46972) 2023-07-06T20:37:01.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.195+0000 7fd1ef7fe700 1 -- 172.21.15.69:0/1191999162 learned_addr learned my addr 172.21.15.69:0/1191999162 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:01.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.195+0000 7fd1eeffd700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f00ffcf0 0x7fd1f01010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:01.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.195+0000 7fd1effff700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1f0100680 0x7fd1f0101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:01.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.196+0000 7fd1eeffd700 1 -- 172.21.15.69:0/1191999162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1f0100680 msgr2=0x7fd1f0101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:01.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.196+0000 7fd1eeffd700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1f0100680 0x7fd1f0101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.196+0000 7fd1eeffd700 1 -- 172.21.15.69:0/1191999162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f00feb00 msgr2=0x7fd1f0102a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:01.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.196+0000 7fd1eeffd700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f00feb00 0x7fd1f0102a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.196+0000 7fd1eeffd700 1 -- 172.21.15.69:0/1191999162 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e4009520 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.196+0000 7fd1ef7fe700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f00feb00 0x7fd1f0102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:37:01.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.196+0000 7fd1eeffd700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f00ffcf0 0x7fd1f01010d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400e9f0 tx=0x7fd1e400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:01.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.196+0000 7fd1ecff9700 1 -- 172.21.15.69:0/1191999162 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400cb70 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.196+0000 7fd1ecff9700 1 -- 172.21.15.69:0/1191999162 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e400ee90 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.196+0000 7fd1ecff9700 1 -- 172.21.15.69:0/1191999162 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4003cf0 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.196+0000 7fd1f5bff700 1 -- 172.21.15.69:0/1191999162 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e00095f0 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.196+0000 7fd1f5bff700 1 -- 172.21.15.69:0/1191999162 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f019ffe0 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.197+0000 7fd1f5bff700 1 -- 172.21.15.69:0/1191999162 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f010a320 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.202 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.199+0000 7fd1ecff9700 1 -- 172.21.15.69:0/1191999162 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd1e400ccd0 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.201+0000 7fd1ecff9700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd1d80a11a0 0x7fd1d80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:01.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.201+0000 7fd1ecff9700 1 -- 172.21.15.69:0/1191999162 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fd1e4014070 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.201+0000 7fd1ef7fe700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd1d80a11a0 0x7fd1d80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:01.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.201+0000 7fd1ef7fe700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd1d80a11a0 0x7fd1d80a3650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0005b20 tx=0x7fd1e0005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:01.207 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.205+0000 7fd1ecff9700 1 -- 172.21.15.69:0/1191999162 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd1e4096310 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.434 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.432+0000 7fd1f5bff700 1 -- 172.21.15.69:0/1191999162 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1f01a0650 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.436 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.434+0000 7fd1ecff9700 1 -- 172.21.15.69:0/1191999162 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7fd1e4099930 con 0x7fd1f00ffcf0 2023-07-06T20:37:01.436 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:01.437 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":87,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:18:59.638711+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:37:01.439 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.436+0000 7fd1f5bff700 1 -- 172.21.15.69:0/1191999162 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd1d80a11a0 msgr2=0x7fd1d80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:01.439 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.436+0000 7fd1f5bff700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd1d80a11a0 0x7fd1d80a3650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0005b20 tx=0x7fd1e0005a90 comp rx=0 tx=0).stop 2023-07-06T20:37:01.439 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.436+0000 7fd1f5bff700 1 -- 172.21.15.69:0/1191999162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f00ffcf0 msgr2=0x7fd1f01010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:01.439 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.436+0000 7fd1f5bff700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f00ffcf0 0x7fd1f01010d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400e9f0 tx=0x7fd1e400ed00 comp rx=0 tx=0).stop 2023-07-06T20:37:01.439 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.436+0000 7fd1f5bff700 1 -- 172.21.15.69:0/1191999162 shutdown_connections 2023-07-06T20:37:01.439 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.436+0000 7fd1f5bff700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fd1d80a11a0 0x7fd1d80a3650 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.439 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.436+0000 7fd1f5bff700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1f0100680 0x7fd1f0101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.440 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.436+0000 7fd1f5bff700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f00feb00 0x7fd1f0102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.440 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.436+0000 7fd1f5bff700 1 --2- 172.21.15.69:0/1191999162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1f00ffcf0 0x7fd1f01010d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.440 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.436+0000 7fd1f5bff700 1 -- 172.21.15.69:0/1191999162 >> 172.21.15.69:0/1191999162 conn(0x7fd1f00fa100 msgr2=0x7fd1f0067840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:01.440 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.437+0000 7fd1f5bff700 1 -- 172.21.15.69:0/1191999162 shutdown_connections 2023-07-06T20:37:01.440 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.437+0000 7fd1f5bff700 1 -- 172.21.15.69:0/1191999162 wait complete. 2023-07-06T20:37:01.458 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:14.022799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:17:26.523183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:17:27.543386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:37:01.459 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:37:01.661 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.658+0000 7f6ff13ff700 1 -- 172.21.15.69:0/3224931538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fec074680 msgr2=0x7f6fec072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:01.661 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.658+0000 7f6ff13ff700 1 --2- 172.21.15.69:0/3224931538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fec074680 0x7f6fec072ad0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009990 tx=0x7f6fdc009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.661 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.659+0000 7f6ff13ff700 1 -- 172.21.15.69:0/3224931538 shutdown_connections 2023-07-06T20:37:01.662 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.659+0000 7f6ff13ff700 1 --2- 172.21.15.69:0/3224931538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fec10c1f0 0x7f6fec10e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.662 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.659+0000 7f6ff13ff700 1 --2- 172.21.15.69:0/3224931538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fec074680 0x7f6fec072ad0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.662 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.659+0000 7f6ff13ff700 1 --2- 172.21.15.69:0/3224931538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fec0730a0 0x7f6fec073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.662 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.659+0000 7f6ff13ff700 1 -- 172.21.15.69:0/3224931538 >> 172.21.15.69:0/3224931538 conn(0x7f6fec0fb260 msgr2=0x7f6fec0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:01.662 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.659+0000 7f6ff13ff700 1 -- 172.21.15.69:0/3224931538 shutdown_connections 2023-07-06T20:37:01.662 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.659+0000 7f6ff13ff700 1 -- 172.21.15.69:0/3224931538 wait complete. 2023-07-06T20:37:01.663 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.660+0000 7f6ff13ff700 1 Processor -- start 2023-07-06T20:37:01.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.660+0000 7f6ff13ff700 1 -- start start 2023-07-06T20:37:01.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6ff13ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fec0730a0 0x7f6fec19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:01.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6ff13ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fec074680 0x7f6fec19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:01.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6ff13ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fec10c1f0 0x7f6fec1a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:01.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6ff13ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec1a3b10 con 0x7f6fec074680 2023-07-06T20:37:01.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6ff13ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec1a3c50 con 0x7f6fec0730a0 2023-07-06T20:37:01.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6ff13ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec1a3de0 con 0x7f6fec10c1f0 2023-07-06T20:37:01.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6feaffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fec0730a0 0x7f6fec19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:01.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6feaffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fec0730a0 0x7f6fec19bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:46998/0 (socket says 172.21.15.69:46998) 2023-07-06T20:37:01.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6feaffd700 1 -- 172.21.15.69:0/1680559426 learned_addr learned my addr 172.21.15.69:0/1680559426 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:01.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6fea7fc700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fec074680 0x7f6fec19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:01.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6feb7fe700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fec10c1f0 0x7f6fec1a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:01.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6feaffd700 1 -- 172.21.15.69:0/1680559426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fec10c1f0 msgr2=0x7f6fec1a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:01.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6feaffd700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fec10c1f0 0x7f6fec1a3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6feaffd700 1 -- 172.21.15.69:0/1680559426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fec074680 msgr2=0x7f6fec19c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:01.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6feaffd700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fec074680 0x7f6fec19c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.661+0000 7f6feaffd700 1 -- 172.21.15.69:0/1680559426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe4009c40 con 0x7f6fec0730a0 2023-07-06T20:37:01.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.662+0000 7f6feaffd700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fec0730a0 0x7f6fec19bf90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc000c00 tx=0x7f6fdc005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:01.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.662+0000 7f6fd3fff700 1 -- 172.21.15.69:0/1680559426 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc01d070 con 0x7f6fec0730a0 2023-07-06T20:37:01.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.662+0000 7f6fd3fff700 1 -- 172.21.15.69:0/1680559426 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fdc022470 con 0x7f6fec0730a0 2023-07-06T20:37:01.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.662+0000 7f6ff13ff700 1 -- 172.21.15.69:0/1680559426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc0095f0 con 0x7f6fec0730a0 2023-07-06T20:37:01.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.662+0000 7f6fd3fff700 1 -- 172.21.15.69:0/1680559426 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc017650 con 0x7f6fec0730a0 2023-07-06T20:37:01.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.662+0000 7f6ff13ff700 1 -- 172.21.15.69:0/1680559426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec1a4350 con 0x7f6fec0730a0 2023-07-06T20:37:01.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.663+0000 7f6ff13ff700 1 -- 172.21.15.69:0/1680559426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fec101d40 con 0x7f6fec0730a0 2023-07-06T20:37:01.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.665+0000 7f6fd3fff700 1 -- 172.21.15.69:0/1680559426 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6fdc0225e0 con 0x7f6fec0730a0 2023-07-06T20:37:01.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.667+0000 7f6fd3fff700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6fd40a11d0 0x7f6fd40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:01.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.667+0000 7f6fd3fff700 1 -- 172.21.15.69:0/1680559426 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f6fdc0ce020 con 0x7f6fec0730a0 2023-07-06T20:37:01.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.667+0000 7f6fea7fc700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6fd40a11d0 0x7f6fd40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:01.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.667+0000 7f6fea7fc700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6fd40a11d0 0x7f6fd40a3680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0743a0 tx=0x7f6fe0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.671+0000 7f6fd3fff700 1 -- 172.21.15.69:0/1680559426 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6fdc0977b0 con 0x7f6fec0730a0 2023-07-06T20:37:01.886 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.883+0000 7f6ff13ff700 1 -- 172.21.15.69:0/1680559426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6fec02ca70 con 0x7f6fec0730a0 2023-07-06T20:37:01.886 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.884+0000 7f6fd3fff700 1 -- 172.21.15.69:0/1680559426 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6fdc09add0 con 0x7f6fec0730a0 2023-07-06T20:37:01.886 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2023-07-06T20:37:01.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.886+0000 7f6ff13ff700 1 -- 172.21.15.69:0/1680559426 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6fd40a11d0 msgr2=0x7f6fd40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:01.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.886+0000 7f6ff13ff700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6fd40a11d0 0x7f6fd40a3680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0743a0 tx=0x7f6fe0009260 comp rx=0 tx=0).stop 2023-07-06T20:37:01.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.886+0000 7f6ff13ff700 1 -- 172.21.15.69:0/1680559426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fec0730a0 msgr2=0x7f6fec19bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:01.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.886+0000 7f6ff13ff700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fec0730a0 0x7f6fec19bf90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc000c00 tx=0x7f6fdc005720 comp rx=0 tx=0).stop 2023-07-06T20:37:01.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.886+0000 7f6ff13ff700 1 -- 172.21.15.69:0/1680559426 shutdown_connections 2023-07-06T20:37:01.889 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.886+0000 7f6ff13ff700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f6fd40a11d0 0x7f6fd40a3680 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.886+0000 7f6ff13ff700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fec10c1f0 0x7f6fec1a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.886+0000 7f6ff13ff700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fec0730a0 0x7f6fec19bf90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.886+0000 7f6ff13ff700 1 --2- 172.21.15.69:0/1680559426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fec074680 0x7f6fec19c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:01.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.886+0000 7f6ff13ff700 1 -- 172.21.15.69:0/1680559426 >> 172.21.15.69:0/1680559426 conn(0x7f6fec0fb260 msgr2=0x7f6fec0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:01.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.887+0000 7f6ff13ff700 1 -- 172.21.15.69:0/1680559426 shutdown_connections 2023-07-06T20:37:01.890 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:01.887+0000 7f6ff13ff700 1 -- 172.21.15.69:0/1680559426 wait complete. 2023-07-06T20:37:01.908 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:37:02.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:01 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1191999162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:37:02.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.110+0000 7f31c75ff700 1 -- 172.21.15.69:0/727815277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31c0100e00 msgr2=0x7f31c0101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:02.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.110+0000 7f31c75ff700 1 --2- 172.21.15.69:0/727815277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31c0100e00 0x7f31c0101250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009990 tx=0x7f31b0009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.110+0000 7f31c75ff700 1 -- 172.21.15.69:0/727815277 shutdown_connections 2023-07-06T20:37:02.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.110+0000 7f31c75ff700 1 --2- 172.21.15.69:0/727815277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31c0101790 0x7f31c0108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.113 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.110+0000 7f31c75ff700 1 --2- 172.21.15.69:0/727815277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31c00ffc10 0x7f31c0100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.110+0000 7f31c75ff700 1 --2- 172.21.15.69:0/727815277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31c0100e00 0x7f31c0101250 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.110+0000 7f31c75ff700 1 -- 172.21.15.69:0/727815277 >> 172.21.15.69:0/727815277 conn(0x7f31c00fb210 msgr2=0x7f31c00fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:02.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.110+0000 7f31c75ff700 1 -- 172.21.15.69:0/727815277 shutdown_connections 2023-07-06T20:37:02.114 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.110+0000 7f31c75ff700 1 -- 172.21.15.69:0/727815277 wait complete. 2023-07-06T20:37:02.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.111+0000 7f31c75ff700 1 Processor -- start 2023-07-06T20:37:02.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.111+0000 7f31c75ff700 1 -- start start 2023-07-06T20:37:02.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c75ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31c00ffc10 0x7f31c0195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:02.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c75ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31c0100e00 0x7f31c0195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:02.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c75ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31c0101790 0x7f31c019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:02.116 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c75ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c019d4f0 con 0x7f31c0101790 2023-07-06T20:37:02.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c75ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c019d630 con 0x7f31c00ffc10 2023-07-06T20:37:02.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c75ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c019d7c0 con 0x7f31c0100e00 2023-07-06T20:37:02.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c539b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31c00ffc10 0x7f31c0195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:02.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c539b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31c00ffc10 0x7f31c0195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:47018/0 (socket says 172.21.15.69:47018) 2023-07-06T20:37:02.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c539b700 1 -- 172.21.15.69:0/162359339 learned_addr learned my addr 172.21.15.69:0/162359339 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:02.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c5b9c700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31c0101790 0x7f31c019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:02.117 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c4b9a700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31c0100e00 0x7f31c0195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:02.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c539b700 1 -- 172.21.15.69:0/162359339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31c0100e00 msgr2=0x7f31c0195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:02.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c539b700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31c0100e00 0x7f31c0195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c539b700 1 -- 172.21.15.69:0/162359339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31c0101790 msgr2=0x7f31c019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:02.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c539b700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31c0101790 0x7f31c019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c539b700 1 -- 172.21.15.69:0/162359339 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b8009c40 con 0x7f31c00ffc10 2023-07-06T20:37:02.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.112+0000 7f31c5b9c700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31c0101790 0x7f31c019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:37:02.118 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.113+0000 7f31c539b700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31c00ffc10 0x7f31c0195970 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00e970 tx=0x7f31bc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:02.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.113+0000 7f31b67fc700 1 -- 172.21.15.69:0/162359339 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc009790 con 0x7f31c00ffc10 2023-07-06T20:37:02.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.113+0000 7f31b67fc700 1 -- 172.21.15.69:0/162359339 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31bc00c7c0 con 0x7f31c00ffc10 2023-07-06T20:37:02.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.113+0000 7f31c75ff700 1 -- 172.21.15.69:0/162359339 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b00095f0 con 0x7f31c00ffc10 2023-07-06T20:37:02.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.113+0000 7f31b67fc700 1 -- 172.21.15.69:0/162359339 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc003c20 con 0x7f31c00ffc10 2023-07-06T20:37:02.119 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.113+0000 7f31c75ff700 1 -- 172.21.15.69:0/162359339 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c019de20 con 0x7f31c00ffc10 2023-07-06T20:37:02.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.114+0000 7f31c75ff700 1 -- 172.21.15.69:0/162359339 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c010b3e0 con 0x7f31c00ffc10 2023-07-06T20:37:02.120 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.116+0000 7f31b67fc700 1 -- 172.21.15.69:0/162359339 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f31bc00c930 con 0x7f31c00ffc10 2023-07-06T20:37:02.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.117+0000 7f31b67fc700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f31ac0a1150 0x7f31ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:02.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.117+0000 7f31b67fc700 1 -- 172.21.15.69:0/162359339 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f31bc014070 con 0x7f31c00ffc10 2023-07-06T20:37:02.121 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.118+0000 7f31c4b9a700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f31ac0a1150 0x7f31ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:02.122 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.118+0000 7f31c4b9a700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f31ac0a1150 0x7f31ac0a3600 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f31b000b2e0 tx=0x7f31b0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:02.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:01 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1191999162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:37:02.126 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.123+0000 7f31b67fc700 1 -- 172.21.15.69:0/162359339 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f31bc096370 con 0x7f31c00ffc10 2023-07-06T20:37:02.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:01 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1191999162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:37:02.334 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.332+0000 7f31c75ff700 1 -- 172.21.15.69:0/162359339 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31c0106b60 con 0x7f31c00ffc10 2023-07-06T20:37:02.335 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.332+0000 7f31b67fc700 1 -- 172.21.15.69:0/162359339 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f31bc01f340 con 0x7f31c00ffc10 2023-07-06T20:37:02.335 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2023-07-06T20:37:02.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.335+0000 7f31c75ff700 1 -- 172.21.15.69:0/162359339 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f31ac0a1150 msgr2=0x7f31ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:02.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.335+0000 7f31c75ff700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f31ac0a1150 0x7f31ac0a3600 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f31b000b2e0 tx=0x7f31b0005960 comp rx=0 tx=0).stop 2023-07-06T20:37:02.337 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.335+0000 7f31c75ff700 1 -- 172.21.15.69:0/162359339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31c00ffc10 msgr2=0x7f31c0195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:02.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.335+0000 7f31c75ff700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31c00ffc10 0x7f31c0195970 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00e970 tx=0x7f31bc00ec80 comp rx=0 tx=0).stop 2023-07-06T20:37:02.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.335+0000 7f31c75ff700 1 -- 172.21.15.69:0/162359339 shutdown_connections 2023-07-06T20:37:02.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.335+0000 7f31c75ff700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f31ac0a1150 0x7f31ac0a3600 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.335+0000 7f31c75ff700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31c0101790 0x7f31c019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.335+0000 7f31c75ff700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31c00ffc10 0x7f31c0195970 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.335+0000 7f31c75ff700 1 --2- 172.21.15.69:0/162359339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31c0100e00 0x7f31c0195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.338 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.335+0000 7f31c75ff700 1 -- 172.21.15.69:0/162359339 >> 172.21.15.69:0/162359339 conn(0x7f31c00fb210 msgr2=0x7f31c00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:02.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.335+0000 7f31c75ff700 1 -- 172.21.15.69:0/162359339 shutdown_connections 2023-07-06T20:37:02.339 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.335+0000 7f31c75ff700 1 -- 172.21.15.69:0/162359339 wait complete. 2023-07-06T20:37:02.356 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:37:02.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.553+0000 7face07ff700 1 -- 172.21.15.69:0/2325412163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facd8100e10 msgr2=0x7facd8101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:02.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.553+0000 7face07ff700 1 --2- 172.21.15.69:0/2325412163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facd8100e10 0x7facd8101260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7facc8009990 tx=0x7facc8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.553+0000 7face07ff700 1 -- 172.21.15.69:0/2325412163 shutdown_connections 2023-07-06T20:37:02.556 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.553+0000 7face07ff700 1 --2- 172.21.15.69:0/2325412163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd81017a0 0x7facd8108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.553+0000 7face07ff700 1 --2- 172.21.15.69:0/2325412163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facd80ffc20 0x7facd8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.553+0000 7face07ff700 1 --2- 172.21.15.69:0/2325412163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facd8100e10 0x7facd8101260 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.553+0000 7face07ff700 1 -- 172.21.15.69:0/2325412163 >> 172.21.15.69:0/2325412163 conn(0x7facd80fb260 msgr2=0x7facd80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:02.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.554+0000 7face07ff700 1 -- 172.21.15.69:0/2325412163 shutdown_connections 2023-07-06T20:37:02.557 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.554+0000 7face07ff700 1 -- 172.21.15.69:0/2325412163 wait complete. 2023-07-06T20:37:02.558 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.554+0000 7face07ff700 1 Processor -- start 2023-07-06T20:37:02.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.555+0000 7face07ff700 1 -- start start 2023-07-06T20:37:02.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.555+0000 7face07ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facd80ffc20 0x7facd80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:02.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.555+0000 7face07ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facd8100e10 0x7facd8072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:02.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.555+0000 7face07ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd81017a0 0x7facd8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:02.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.555+0000 7face07ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd8073710 con 0x7facd80ffc20 2023-07-06T20:37:02.559 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.555+0000 7face07ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd8073880 con 0x7facd81017a0 2023-07-06T20:37:02.560 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.555+0000 7face07ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd819f980 con 0x7facd8100e10 2023-07-06T20:37:02.560 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.555+0000 7facded9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd81017a0 0x7facd8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:02.560 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.555+0000 7facde59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facd80ffc20 0x7facd80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:02.560 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.555+0000 7facded9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd81017a0 0x7facd8072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:47032/0 (socket says 172.21.15.69:47032) 2023-07-06T20:37:02.560 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.555+0000 7facded9c700 1 -- 172.21.15.69:0/2895249444 learned_addr learned my addr 172.21.15.69:0/2895249444 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:02.560 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.555+0000 7facddd9a700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facd8100e10 0x7facd8072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:02.560 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.556+0000 7facded9c700 1 -- 172.21.15.69:0/2895249444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facd8100e10 msgr2=0x7facd8072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:02.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.556+0000 7facded9c700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facd8100e10 0x7facd8072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.556+0000 7facded9c700 1 -- 172.21.15.69:0/2895249444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facd80ffc20 msgr2=0x7facd80743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:02.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.556+0000 7facded9c700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facd80ffc20 0x7facd80743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.556+0000 7facded9c700 1 -- 172.21.15.69:0/2895249444 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd0009c40 con 0x7facd81017a0 2023-07-06T20:37:02.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.556+0000 7facded9c700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd81017a0 0x7facd8072f70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7facd000c370 tx=0x7facd000c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:02.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.556+0000 7faccf7fe700 1 -- 172.21.15.69:0/2895249444 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd0017070 con 0x7facd81017a0 2023-07-06T20:37:02.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.556+0000 7faccf7fe700 1 -- 172.21.15.69:0/2895249444 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facd000f040 con 0x7facd81017a0 2023-07-06T20:37:02.561 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.556+0000 7face07ff700 1 -- 172.21.15.69:0/2895249444 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc80095f0 con 0x7facd81017a0 2023-07-06T20:37:02.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.556+0000 7faccf7fe700 1 -- 172.21.15.69:0/2895249444 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd000e050 con 0x7facd81017a0 2023-07-06T20:37:02.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.556+0000 7face07ff700 1 -- 172.21.15.69:0/2895249444 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd819ffd0 con 0x7facd81017a0 2023-07-06T20:37:02.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.557+0000 7face07ff700 1 -- 172.21.15.69:0/2895249444 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd81071c0 con 0x7facd81017a0 2023-07-06T20:37:02.562 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.559+0000 7faccf7fe700 1 -- 172.21.15.69:0/2895249444 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7facd000cb60 con 0x7facd81017a0 2023-07-06T20:37:02.563 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.561+0000 7faccf7fe700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7facc40a11d0 0x7facc40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:02.563 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.561+0000 7faccf7fe700 1 -- 172.21.15.69:0/2895249444 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7facd00cc910 con 0x7facd81017a0 2023-07-06T20:37:02.563 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.561+0000 7facde59b700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7facc40a11d0 0x7facc40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:02.564 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.561+0000 7facde59b700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7facc40a11d0 0x7facc40a3680 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7facd8100c70 tx=0x7facd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:02.566 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.564+0000 7faccf7fe700 1 -- 172.21.15.69:0/2895249444 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7facd0095fd0 con 0x7facd81017a0 2023-07-06T20:37:02.769 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.767+0000 7face07ff700 1 -- 172.21.15.69:0/2895249444 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7facd802caf0 con 0x7facd81017a0 2023-07-06T20:37:02.770 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.768+0000 7faccf7fe700 1 -- 172.21.15.69:0/2895249444 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7facd00995f0 con 0x7facd81017a0 2023-07-06T20:37:02.770 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2023-07-06T20:37:02.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.770+0000 7face07ff700 1 -- 172.21.15.69:0/2895249444 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7facc40a11d0 msgr2=0x7facc40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:02.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.770+0000 7face07ff700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7facc40a11d0 0x7facc40a3680 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7facd8100c70 tx=0x7facd4009260 comp rx=0 tx=0).stop 2023-07-06T20:37:02.773 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.770+0000 7face07ff700 1 -- 172.21.15.69:0/2895249444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd81017a0 msgr2=0x7facd8072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:02.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.770+0000 7face07ff700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd81017a0 0x7facd8072f70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7facd000c370 tx=0x7facd000c680 comp rx=0 tx=0).stop 2023-07-06T20:37:02.775 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.770+0000 7face07ff700 1 -- 172.21.15.69:0/2895249444 shutdown_connections 2023-07-06T20:37:02.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.770+0000 7face07ff700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7facc40a11d0 0x7facc40a3680 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.770+0000 7face07ff700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd81017a0 0x7facd8072f70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.770+0000 7face07ff700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facd80ffc20 0x7facd80743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.770+0000 7face07ff700 1 --2- 172.21.15.69:0/2895249444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facd8100e10 0x7facd8072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.770+0000 7face07ff700 1 -- 172.21.15.69:0/2895249444 >> 172.21.15.69:0/2895249444 conn(0x7facd80fb260 msgr2=0x7facd8104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:02.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.770+0000 7face07ff700 1 -- 172.21.15.69:0/2895249444 shutdown_connections 2023-07-06T20:37:02.776 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.771+0000 7face07ff700 1 -- 172.21.15.69:0/2895249444 wait complete. 2023-07-06T20:37:02.792 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-06T20:37:02.990 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.987+0000 7f7b62bff700 1 -- 172.21.15.69:0/2798333631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c100e20 msgr2=0x7f7b5c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.987+0000 7f7b62bff700 1 --2- 172.21.15.69:0/2798333631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c100e20 0x7f7b5c101270 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009990 tx=0x7f7b48009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.987+0000 7f7b62bff700 1 -- 172.21.15.69:0/2798333631 shutdown_connections 2023-07-06T20:37:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.987+0000 7f7b62bff700 1 --2- 172.21.15.69:0/2798333631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b5c1017b0 0x7f7b5c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.987+0000 7f7b62bff700 1 --2- 172.21.15.69:0/2798333631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5c0ffc30 0x7f7b5c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.987+0000 7f7b62bff700 1 --2- 172.21.15.69:0/2798333631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c100e20 0x7f7b5c101270 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:02.991 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.987+0000 7f7b62bff700 1 -- 172.21.15.69:0/2798333631 >> 172.21.15.69:0/2798333631 conn(0x7f7b5c0fb210 msgr2=0x7f7b5c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:02.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.987+0000 7f7b62bff700 1 -- 172.21.15.69:0/2798333631 shutdown_connections 2023-07-06T20:37:02.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.987+0000 7f7b62bff700 1 -- 172.21.15.69:0/2798333631 wait complete. 2023-07-06T20:37:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.988+0000 7f7b62bff700 1 Processor -- start 2023-07-06T20:37:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.988+0000 7f7b62bff700 1 -- start start 2023-07-06T20:37:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b62bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b5c0ffc30 0x7f7b5c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:02.998 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b62bff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5c100e20 0x7f7b5c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:02.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b62bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c1017b0 0x7f7b5c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:02.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b62bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c19f6c0 con 0x7f7b5c1017b0 2023-07-06T20:37:02.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b62bff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c19f800 con 0x7f7b5c100e20 2023-07-06T20:37:02.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b62bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c19f990 con 0x7f7b5c0ffc30 2023-07-06T20:37:02.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5c100e20 0x7f7b5c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:02.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5c100e20 0x7f7b5c198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:33598/0 (socket says 172.21.15.69:33598) 2023-07-06T20:37:02.999 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b5bfff700 1 -- 172.21.15.69:0/1227748546 learned_addr learned my addr 172.21.15.69:0/1227748546 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:03.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b6119c700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c1017b0 0x7f7b5c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:03.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b6099b700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b5c0ffc30 0x7f7b5c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:03.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b5bfff700 1 -- 172.21.15.69:0/1227748546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b5c0ffc30 msgr2=0x7f7b5c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:03.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b5bfff700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b5c0ffc30 0x7f7b5c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:03.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b5bfff700 1 -- 172.21.15.69:0/1227748546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c1017b0 msgr2=0x7f7b5c19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:03.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b5bfff700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c1017b0 0x7f7b5c19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:03.000 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.989+0000 7f7b5bfff700 1 -- 172.21.15.69:0/1227748546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b480095f0 con 0x7f7b5c100e20 2023-07-06T20:37:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.990+0000 7f7b5bfff700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5c100e20 0x7f7b5c198080 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7b48004760 tx=0x7f7b48004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.990+0000 7f7b59ffb700 1 -- 172.21.15.69:0/1227748546 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4801d070 con 0x7f7b5c100e20 2023-07-06T20:37:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.990+0000 7f7b59ffb700 1 -- 172.21.15.69:0/1227748546 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b48004830 con 0x7f7b5c100e20 2023-07-06T20:37:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.990+0000 7f7b62bff700 1 -- 172.21.15.69:0/1227748546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5c19fc10 con 0x7f7b5c100e20 2023-07-06T20:37:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.990+0000 7f7b59ffb700 1 -- 172.21.15.69:0/1227748546 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48005350 con 0x7f7b5c100e20 2023-07-06T20:37:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.990+0000 7f7b62bff700 1 -- 172.21.15.69:0/1227748546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5c1a0030 con 0x7f7b5c100e20 2023-07-06T20:37:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.991+0000 7f7b62bff700 1 -- 172.21.15.69:0/1227748546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5c10b4c0 con 0x7f7b5c100e20 2023-07-06T20:37:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.993+0000 7f7b59ffb700 1 -- 172.21.15.69:0/1227748546 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7b480054d0 con 0x7f7b5c100e20 2023-07-06T20:37:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.995+0000 7f7b59ffb700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7b4c0a1220 0x7f7b4c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.995+0000 7f7b59ffb700 1 -- 172.21.15.69:0/1227748546 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f7b480cdf20 con 0x7f7b5c100e20 2023-07-06T20:37:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.995+0000 7f7b6099b700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7b4c0a1220 0x7f7b4c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:02.996+0000 7f7b6099b700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7b4c0a1220 0x7f7b4c0a36d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c100c80 tx=0x7f7b50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.000+0000 7f7b59ffb700 1 -- 172.21.15.69:0/1227748546 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7b48097670 con 0x7f7b5c100e20 2023-07-06T20:37:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:02 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1680559426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:37:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:02 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/162359339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:37:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:02 smithi156 ceph-mon[129280]: pgmap v859: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 2.38k op/s 2023-07-06T20:37:03.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:02 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2895249444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:37:03.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:02 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1680559426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:37:03.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:02 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/162359339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:37:03.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:02 smithi098 ceph-mon[128968]: pgmap v859: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 2.38k op/s 2023-07-06T20:37:03.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:02 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2895249444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:37:03.257 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.255+0000 7f7b62bff700 1 -- 172.21.15.69:0/1227748546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f7b5c1a0640 con 0x7f7b5c100e20 2023-07-06T20:37:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:02 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1680559426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:37:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:02 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/162359339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:37:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:02 smithi069 ceph-mon[126399]: pgmap v859: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 2.38k op/s 2023-07-06T20:37:03.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:02 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2895249444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:37:03.269 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.267+0000 7f7b59ffb700 1 -- 172.21.15.69:0/1227748546 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733244 (secure 0 0 0) 0x7f7b4809ac90 con 0x7f7b5c100e20 2023-07-06T20:37:03.277 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:03.277 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:37:03.277 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 18, 2023-07-06T20:37:03.277 INFO:teuthology.orchestra.run.smithi069.stdout: "active_gid": 14152, 2023-07-06T20:37:03.278 INFO:teuthology.orchestra.run.smithi069.stdout: "active_name": "x", 2023-07-06T20:37:03.278 INFO:teuthology.orchestra.run.smithi069.stdout: "active_addrs": { 2023-07-06T20:37:03.278 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2023-07-06T20:37:03.278 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.278 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2023-07-06T20:37:03.278 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:6800", 2023-07-06T20:37:03.278 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 4215258217 2023-07-06T20:37:03.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.279 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.279 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2023-07-06T20:37:03.279 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:6801", 2023-07-06T20:37:03.279 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 4215258217 2023-07-06T20:37:03.279 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.279 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2023-07-06T20:37:03.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.280 INFO:teuthology.orchestra.run.smithi069.stdout: "active_addr": "172.21.15.69:6801/4215258217", 2023-07-06T20:37:03.280 INFO:teuthology.orchestra.run.smithi069.stdout: "active_change": "2023-07-06T20:10:57.432872+0000", 2023-07-06T20:37:03.280 INFO:teuthology.orchestra.run.smithi069.stdout: "active_mgr_features": 4540138322906710015, 2023-07-06T20:37:03.280 INFO:teuthology.orchestra.run.smithi069.stdout: "available": true, 2023-07-06T20:37:03.280 INFO:teuthology.orchestra.run.smithi069.stdout: "standbys": [ 2023-07-06T20:37:03.280 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.280 INFO:teuthology.orchestra.run.smithi069.stdout: "gid": 24109, 2023-07-06T20:37:03.281 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "z", 2023-07-06T20:37:03.281 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr_features": 4540138322906710015, 2023-07-06T20:37:03.281 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2023-07-06T20:37:03.281 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.281 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2023-07-06T20:37:03.281 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.281 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.281 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.282 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:03.282 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:03.282 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.282 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.282 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.282 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:03.282 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.282 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.283 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.283 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2023-07-06T20:37:03.283 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.283 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.283 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.283 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.283 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.284 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.284 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.284 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.284 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.284 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.284 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.284 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.284 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.285 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.285 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.285 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.285 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.285 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.285 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.285 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.285 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.286 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.286 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.286 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.286 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.286 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.286 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.286 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.287 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.287 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.287 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.287 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.287 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.287 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.287 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.287 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.288 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.288 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.288 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.288 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.288 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.288 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.288 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.289 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.289 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.289 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.289 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.289 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.289 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.289 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.289 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.290 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.290 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.290 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.290 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.290 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.290 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.290 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.290 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.291 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.291 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.291 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.291 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.291 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.291 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.291 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.291 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.292 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.292 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.292 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.292 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.292 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.292 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2023-07-06T20:37:03.293 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2023-07-06T20:37:03.293 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.293 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.293 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.293 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.293 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.293 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.293 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.294 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2023-07-06T20:37:03.294 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.294 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.294 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.294 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.294 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2023-07-06T20:37:03.294 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2023-07-06T20:37:03.294 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.295 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.295 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.295 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2023-07-06T20:37:03.295 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.295 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.295 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.295 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2023-07-06T20:37:03.295 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.296 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.296 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.296 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.296 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2023-07-06T20:37:03.296 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2023-07-06T20:37:03.296 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.296 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.297 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.297 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.297 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.297 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.297 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2023-07-06T20:37:03.297 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.297 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.297 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.298 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.298 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2023-07-06T20:37:03.298 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2023-07-06T20:37:03.298 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.298 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.298 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.298 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.298 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.299 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.299 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.299 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2023-07-06T20:37:03.299 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.299 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.299 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.299 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.300 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2023-07-06T20:37:03.300 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2023-07-06T20:37:03.300 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.300 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.300 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.300 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2023-07-06T20:37:03.300 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.300 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.301 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.301 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2023-07-06T20:37:03.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.301 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.301 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.301 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2023-07-06T20:37:03.301 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2023-07-06T20:37:03.302 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.302 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.302 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.302 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.302 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.302 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.302 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.302 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2023-07-06T20:37:03.303 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.303 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.303 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.303 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.303 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2023-07-06T20:37:03.303 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2023-07-06T20:37:03.303 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.304 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.304 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.304 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.304 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.304 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.304 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.304 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-06T20:37:03.305 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.305 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.305 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.305 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.305 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2023-07-06T20:37:03.305 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2023-07-06T20:37:03.305 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.305 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.306 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.306 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.306 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.306 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.306 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.306 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2023-07-06T20:37:03.306 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.306 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.307 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.307 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.307 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.307 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.307 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2023-07-06T20:37:03.307 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.308 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.308 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.308 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2023-07-06T20:37:03.308 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2023-07-06T20:37:03.308 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.308 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.308 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.308 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.309 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.309 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.309 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.309 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2023-07-06T20:37:03.309 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.309 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.309 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.309 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.310 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2023-07-06T20:37:03.310 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2023-07-06T20:37:03.310 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.310 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.310 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.310 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2023-07-06T20:37:03.310 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.311 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.311 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.311 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2023-07-06T20:37:03.311 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-06T20:37:03.311 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.311 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.311 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2023-07-06T20:37:03.312 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2023-07-06T20:37:03.312 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:03.312 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.312 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.312 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:03.312 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2023-07-06T20:37:03.312 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2023-07-06T20:37:03.312 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.313 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-06T20:37:03.313 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-06T20:37:03.313 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.313 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.313 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.313 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2023-07-06T20:37:03.313 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2023-07-06T20:37:03.314 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:03.314 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.314 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.314 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2023-07-06T20:37:03.314 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2023-07-06T20:37:03.314 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2023-07-06T20:37:03.314 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.314 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-06T20:37:03.315 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.315 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.315 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.315 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.315 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2023-07-06T20:37:03.315 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2023-07-06T20:37:03.315 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.315 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.316 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.316 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2023-07-06T20:37:03.316 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.316 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.316 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.316 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-06T20:37:03.316 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-06T20:37:03.316 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.317 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.317 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.317 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2023-07-06T20:37:03.317 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2023-07-06T20:37:03.317 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:03.317 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.317 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.317 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2023-07-06T20:37:03.318 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2023-07-06T20:37:03.318 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2023-07-06T20:37:03.318 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.318 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2023-07-06T20:37:03.318 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-06T20:37:03.318 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.318 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.318 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.319 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2023-07-06T20:37:03.319 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2023-07-06T20:37:03.319 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.319 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.319 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.319 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2023-07-06T20:37:03.319 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.319 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.320 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.320 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2023-07-06T20:37:03.320 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-06T20:37:03.320 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.320 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.320 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.320 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2023-07-06T20:37:03.321 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2023-07-06T20:37:03.321 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:03.321 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.321 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.321 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:03.321 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2023-07-06T20:37:03.321 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2023-07-06T20:37:03.321 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.322 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-06T20:37:03.322 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-06T20:37:03.322 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.322 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.322 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.322 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.323 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.323 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.323 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.323 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.323 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.323 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.323 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.324 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.324 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.324 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.324 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.324 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.324 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.324 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.324 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.325 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.325 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.325 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.325 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.325 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.325 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.325 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.325 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.326 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.326 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.326 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.326 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.326 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.326 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.326 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.327 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.327 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.327 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.327 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.327 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.327 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.327 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.328 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.328 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.328 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.328 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.328 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.328 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.328 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.329 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.329 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.329 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.329 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.329 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.329 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.329 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.329 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.330 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.330 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.330 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.330 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.330 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.330 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.330 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.330 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.331 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.331 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.331 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.331 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.331 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.331 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.331 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2023-07-06T20:37:03.331 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2023-07-06T20:37:03.332 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:03.332 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.332 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.332 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:03.332 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.332 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.332 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.333 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-06T20:37:03.333 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.333 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.333 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.333 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.333 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2023-07-06T20:37:03.333 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2023-07-06T20:37:03.333 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.334 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.334 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.334 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2023-07-06T20:37:03.334 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.334 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.334 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.334 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2023-07-06T20:37:03.334 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2023-07-06T20:37:03.335 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap" 2023-07-06T20:37:03.335 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.335 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2023-07-06T20:37:03.335 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.335 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.335 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.335 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.336 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2023-07-06T20:37:03.336 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2023-07-06T20:37:03.336 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.336 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.336 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.336 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.336 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.336 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.337 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.337 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-06T20:37:03.337 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.337 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.337 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.337 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:03.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:03.338 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.338 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.338 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.338 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:03.338 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.338 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.338 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.338 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-06T20:37:03.339 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.339 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.339 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.339 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2023-07-06T20:37:03.339 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2023-07-06T20:37:03.339 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.340 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.340 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.340 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2023-07-06T20:37:03.340 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2023-07-06T20:37:03.340 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.340 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.340 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-06T20:37:03.340 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-06T20:37:03.341 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.341 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.341 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.341 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2023-07-06T20:37:03.341 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2023-07-06T20:37:03.341 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:03.341 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.341 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.342 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:03.342 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.342 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.342 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.342 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-06T20:37:03.342 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.342 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.342 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.343 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.343 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.343 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.343 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.343 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2023-07-06T20:37:03.343 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.343 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.343 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.344 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2023-07-06T20:37:03.344 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2023-07-06T20:37:03.344 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:03.344 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.344 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.344 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2023-07-06T20:37:03.344 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.345 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.345 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.345 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-06T20:37:03.345 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.345 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.345 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.345 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.345 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2023-07-06T20:37:03.346 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2023-07-06T20:37:03.346 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.346 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.346 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.346 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2023-07-06T20:37:03.346 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.346 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.346 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.347 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-06T20:37:03.347 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.347 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.347 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.347 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.347 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2023-07-06T20:37:03.347 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2023-07-06T20:37:03.347 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.348 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.348 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.348 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2023-07-06T20:37:03.348 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.348 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.348 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.348 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-06T20:37:03.348 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.349 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.349 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.349 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.349 INFO:teuthology.orchestra.run.smithi069.stdout: "alertmanager_web_password": { 2023-07-06T20:37:03.349 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alertmanager_web_password", 2023-07-06T20:37:03.349 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.349 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.350 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.350 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:03.350 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.350 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.350 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.350 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Alertmanager web password", 2023-07-06T20:37:03.350 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.351 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.351 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.351 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.351 INFO:teuthology.orchestra.run.smithi069.stdout: "alertmanager_web_user": { 2023-07-06T20:37:03.351 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alertmanager_web_user", 2023-07-06T20:37:03.351 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.351 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.351 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.352 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:03.352 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.352 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.352 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.352 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Alertmanager web user", 2023-07-06T20:37:03.352 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.352 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.352 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.353 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.353 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2023-07-06T20:37:03.353 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2023-07-06T20:37:03.353 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.353 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.353 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.353 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.354 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.354 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.354 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.354 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-06T20:37:03.354 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-06T20:37:03.354 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.354 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.355 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.355 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2023-07-06T20:37:03.355 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2023-07-06T20:37:03.355 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.355 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.355 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.355 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:03.355 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.356 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.356 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.356 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-06T20:37:03.356 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.356 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.356 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.356 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.356 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2023-07-06T20:37:03.357 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2023-07-06T20:37:03.357 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:03.357 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.357 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.357 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2023-07-06T20:37:03.357 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.357 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.357 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.358 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-06T20:37:03.358 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.358 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.358 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.358 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.358 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2023-07-06T20:37:03.358 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2023-07-06T20:37:03.358 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.359 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.359 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.359 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.359 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.359 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.359 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.359 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-06T20:37:03.360 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.360 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.360 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.360 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.360 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2023-07-06T20:37:03.360 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2023-07-06T20:37:03.360 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.360 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.361 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.361 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.361 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.361 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.361 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.361 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-06T20:37:03.361 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.361 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.362 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.362 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.362 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2023-07-06T20:37:03.362 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2023-07-06T20:37:03.362 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.362 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.362 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.362 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.363 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.363 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.363 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.363 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-06T20:37:03.363 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.363 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.363 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.363 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.364 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2023-07-06T20:37:03.364 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2023-07-06T20:37:03.364 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.364 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.364 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.364 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-06T20:37:03.364 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.365 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.365 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.365 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2023-07-06T20:37:03.365 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.365 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.365 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.365 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.365 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2023-07-06T20:37:03.366 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2023-07-06T20:37:03.366 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.366 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.366 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.366 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-06T20:37:03.366 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.366 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.366 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.367 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2023-07-06T20:37:03.367 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.367 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.367 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.367 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.367 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2023-07-06T20:37:03.367 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2023-07-06T20:37:03.367 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.368 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.368 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.368 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-06T20:37:03.368 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.368 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.368 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.368 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2023-07-06T20:37:03.368 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.369 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.369 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.369 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.369 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2023-07-06T20:37:03.369 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2023-07-06T20:37:03.369 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.369 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.370 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.370 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-06T20:37:03.370 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.370 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.370 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.370 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2023-07-06T20:37:03.370 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.370 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.371 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.371 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.371 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2023-07-06T20:37:03.371 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2023-07-06T20:37:03.371 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.371 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.371 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.372 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-06T20:37:03.372 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.372 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.372 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.372 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2023-07-06T20:37:03.372 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.372 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.373 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.373 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.373 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2023-07-06T20:37:03.373 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2023-07-06T20:37:03.373 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.373 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.373 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.373 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-06T20:37:03.374 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.374 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.374 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.374 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2023-07-06T20:37:03.374 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.374 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.374 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.375 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.375 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2023-07-06T20:37:03.375 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2023-07-06T20:37:03.375 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.375 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.375 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.375 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-06T20:37:03.375 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.376 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.376 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.376 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2023-07-06T20:37:03.376 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.376 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.376 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.376 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.376 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2023-07-06T20:37:03.377 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2023-07-06T20:37:03.377 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.377 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.377 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.377 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-06T20:37:03.377 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.377 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.378 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.378 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2023-07-06T20:37:03.378 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.378 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.378 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.378 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.378 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2023-07-06T20:37:03.378 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2023-07-06T20:37:03.379 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.379 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.379 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.379 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-06T20:37:03.379 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.379 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.379 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.379 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2023-07-06T20:37:03.380 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.380 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.380 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.380 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2023-07-06T20:37:03.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2023-07-06T20:37:03.380 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.380 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.381 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.381 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-06T20:37:03.381 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.381 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.381 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.381 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2023-07-06T20:37:03.381 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.382 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.382 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.382 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.382 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2023-07-06T20:37:03.382 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2023-07-06T20:37:03.382 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.382 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.382 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.383 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-06T20:37:03.383 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.383 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.383 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.383 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2023-07-06T20:37:03.383 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.383 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.383 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.384 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.384 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2023-07-06T20:37:03.384 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2023-07-06T20:37:03.384 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.384 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.384 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.384 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-06T20:37:03.384 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.385 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.385 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.385 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2023-07-06T20:37:03.385 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.385 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.385 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.385 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.385 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2023-07-06T20:37:03.386 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2023-07-06T20:37:03.386 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.386 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.386 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.386 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-06T20:37:03.386 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.386 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.386 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.387 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2023-07-06T20:37:03.387 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.387 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.387 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.387 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.387 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2023-07-06T20:37:03.387 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2023-07-06T20:37:03.387 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.388 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.388 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.388 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-06T20:37:03.388 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.388 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.388 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.388 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2023-07-06T20:37:03.388 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.389 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.389 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.389 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.389 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2023-07-06T20:37:03.389 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2023-07-06T20:37:03.389 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.389 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.389 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.390 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.390 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.390 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.390 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.390 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2023-07-06T20:37:03.390 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.390 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.391 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.391 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.391 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2023-07-06T20:37:03.391 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2023-07-06T20:37:03.391 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.391 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.391 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.391 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:03.392 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.392 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.392 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.392 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-06T20:37:03.392 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.392 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.392 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.392 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.393 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2023-07-06T20:37:03.393 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2023-07-06T20:37:03.393 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.393 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.393 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.393 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2023-07-06T20:37:03.393 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.394 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.394 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.394 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-06T20:37:03.394 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.394 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.394 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.394 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.394 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2023-07-06T20:37:03.395 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2023-07-06T20:37:03.395 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.395 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.395 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.395 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2023-07-06T20:37:03.395 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.395 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.395 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.396 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-06T20:37:03.396 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.396 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.396 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.396 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.396 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2023-07-06T20:37:03.396 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2023-07-06T20:37:03.397 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.397 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.397 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.397 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2023-07-06T20:37:03.397 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.397 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.397 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.397 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2023-07-06T20:37:03.398 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.398 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.398 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.398 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.398 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2023-07-06T20:37:03.398 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2023-07-06T20:37:03.398 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.398 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.399 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.399 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.399 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.399 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.399 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.399 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-06T20:37:03.399 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.399 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.400 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.400 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.400 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2023-07-06T20:37:03.400 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2023-07-06T20:37:03.400 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.400 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.400 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.400 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:03.401 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.401 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.401 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.401 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2023-07-06T20:37:03.401 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.401 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.401 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.401 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.402 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2023-07-06T20:37:03.402 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2023-07-06T20:37:03.402 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.402 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.402 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.402 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:03.402 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.402 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.403 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.403 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2023-07-06T20:37:03.403 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.403 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.403 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.403 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.403 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.404 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.404 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.404 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.404 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.404 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.404 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.404 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.404 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.405 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.405 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.405 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.405 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.405 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.405 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.405 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.405 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.406 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.406 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.406 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.406 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.406 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2023-07-06T20:37:03.406 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2023-07-06T20:37:03.406 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.406 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.407 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.407 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.407 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.407 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.407 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.407 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-06T20:37:03.407 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.407 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.408 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.408 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.408 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.408 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.408 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.408 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.408 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.408 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.409 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.409 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.409 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.409 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-06T20:37:03.409 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.409 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.409 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.410 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.410 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.410 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.410 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.410 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.410 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.410 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.410 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.411 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.411 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.411 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.411 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.411 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.411 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.411 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.411 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.412 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.412 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.412 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.412 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.412 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.412 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.412 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.412 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.413 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.413 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.413 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.413 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.413 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.413 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.413 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.413 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.414 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.414 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.414 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.414 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-06T20:37:03.414 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-06T20:37:03.414 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.414 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.415 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.415 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.415 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.415 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.415 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.415 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-06T20:37:03.416 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.416 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.416 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.416 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.416 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-06T20:37:03.416 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-06T20:37:03.416 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.416 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.417 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.417 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2023-07-06T20:37:03.417 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.417 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.417 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.417 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-06T20:37:03.417 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.417 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.418 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.418 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.418 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2023-07-06T20:37:03.418 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2023-07-06T20:37:03.418 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.418 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.418 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.418 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:03.419 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.419 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.419 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.419 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2023-07-06T20:37:03.419 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.419 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.419 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.420 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2023-07-06T20:37:03.420 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2023-07-06T20:37:03.420 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.420 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.420 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:03.420 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.421 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.421 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.421 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-06T20:37:03.421 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.421 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.421 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.422 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2023-07-06T20:37:03.422 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2023-07-06T20:37:03.422 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.422 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.422 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.422 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.422 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.423 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.423 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2023-07-06T20:37:03.423 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.423 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.423 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.423 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.423 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2023-07-06T20:37:03.423 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2023-07-06T20:37:03.424 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.424 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.424 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.424 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2023-07-06T20:37:03.424 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.424 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.424 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.425 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2023-07-06T20:37:03.425 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2023-07-06T20:37:03.425 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.425 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2023-07-06T20:37:03.425 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.425 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.425 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.425 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.426 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2023-07-06T20:37:03.426 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2023-07-06T20:37:03.426 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.426 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.426 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.426 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-06T20:37:03.426 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.426 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.427 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.427 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-06T20:37:03.427 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.427 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.427 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.427 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.427 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_web_password": { 2023-07-06T20:37:03.427 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_web_password", 2023-07-06T20:37:03.428 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.428 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.428 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.428 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:03.428 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.428 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.428 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.429 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus web password", 2023-07-06T20:37:03.429 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.429 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.429 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.429 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.429 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_web_user": { 2023-07-06T20:37:03.429 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_web_user", 2023-07-06T20:37:03.429 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.430 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.430 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.430 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:03.430 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.430 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.430 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.430 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus web user", 2023-07-06T20:37:03.430 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.431 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.431 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.431 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2023-07-06T20:37:03.431 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2023-07-06T20:37:03.431 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.431 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.431 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.432 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.432 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.432 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.432 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.432 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-06T20:37:03.432 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.432 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.432 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.433 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.433 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2023-07-06T20:37:03.433 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2023-07-06T20:37:03.433 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.433 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.433 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.433 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.433 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.434 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.434 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.434 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-06T20:37:03.434 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.434 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.434 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.434 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.435 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2023-07-06T20:37:03.435 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2023-07-06T20:37:03.435 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.435 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.435 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.435 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.435 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.435 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.436 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-06T20:37:03.436 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.436 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.436 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.436 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2023-07-06T20:37:03.436 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2023-07-06T20:37:03.437 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.437 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.437 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.437 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.437 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.437 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.438 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-06T20:37:03.438 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.438 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.438 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.438 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.438 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2023-07-06T20:37:03.438 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2023-07-06T20:37:03.439 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.439 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.439 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.439 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.439 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.439 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.439 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.439 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-06T20:37:03.440 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.440 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.440 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.440 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.440 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2023-07-06T20:37:03.440 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2023-07-06T20:37:03.440 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.440 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.441 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.441 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2023-07-06T20:37:03.441 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.441 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.441 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.441 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2023-07-06T20:37:03.441 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.442 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.442 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.442 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.442 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2023-07-06T20:37:03.442 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2023-07-06T20:37:03.442 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.442 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.442 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.443 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.443 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.443 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.443 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.443 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-06T20:37:03.443 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.443 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.443 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.444 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.444 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2023-07-06T20:37:03.444 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2023-07-06T20:37:03.444 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.444 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.444 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.445 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.445 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.445 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.445 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-06T20:37:03.445 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.445 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.445 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.445 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.446 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2023-07-06T20:37:03.446 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2023-07-06T20:37:03.446 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.446 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.446 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.446 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.446 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.446 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.447 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.447 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-06T20:37:03.447 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.447 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.447 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.447 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.447 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2023-07-06T20:37:03.447 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2023-07-06T20:37:03.448 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.448 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.448 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.448 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.448 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.448 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.448 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.448 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2023-07-06T20:37:03.449 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.449 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.449 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.449 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.449 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2023-07-06T20:37:03.449 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2023-07-06T20:37:03.449 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.450 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.450 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.450 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.450 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.450 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.450 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-06T20:37:03.450 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.451 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.451 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.451 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.451 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2023-07-06T20:37:03.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2023-07-06T20:37:03.451 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.451 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.451 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.452 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.452 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.452 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.452 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-06T20:37:03.452 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.452 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.452 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.453 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.453 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.453 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.453 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.453 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2023-07-06T20:37:03.453 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.453 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.454 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.454 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.454 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.454 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.454 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.454 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.454 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.454 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.455 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.455 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.455 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.455 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.455 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.455 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.455 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.455 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.456 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.456 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.456 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.456 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.456 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.456 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.456 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.456 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.457 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.457 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.457 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.457 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.457 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.457 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.457 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.457 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.458 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.458 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.458 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.458 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.458 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.458 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.458 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.459 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.459 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.459 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.459 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.459 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.459 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.459 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.459 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.460 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.460 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.460 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.460 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.460 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.460 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.460 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.460 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.461 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.461 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.461 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.461 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.461 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.462 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.462 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.462 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.462 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.462 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.462 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.463 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.463 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2023-07-06T20:37:03.463 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2023-07-06T20:37:03.463 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.463 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.463 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.463 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2023-07-06T20:37:03.463 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.464 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.464 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.464 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-06T20:37:03.464 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.464 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.464 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.464 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.464 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2023-07-06T20:37:03.465 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2023-07-06T20:37:03.465 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.465 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.465 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.465 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2023-07-06T20:37:03.465 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.465 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.466 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.466 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-06T20:37:03.466 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.466 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.466 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.466 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.466 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.466 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.467 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2023-07-06T20:37:03.467 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.467 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.467 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.467 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-06T20:37:03.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-06T20:37:03.467 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.468 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.468 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:03.468 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.468 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.468 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.468 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.469 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.469 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.469 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.469 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2023-07-06T20:37:03.469 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-06T20:37:03.469 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.469 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.470 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.470 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.470 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.470 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.470 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.470 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.470 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.470 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.471 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.471 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.471 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-06T20:37:03.471 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-06T20:37:03.471 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.471 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.471 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.471 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.472 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.472 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.472 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.472 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.472 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.472 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.472 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.472 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.473 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2023-07-06T20:37:03.473 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2023-07-06T20:37:03.473 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.473 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.473 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.473 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.473 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.474 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.474 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.474 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.474 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.474 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.474 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.474 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.474 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-06T20:37:03.475 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-06T20:37:03.475 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.475 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.475 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.475 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.475 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.475 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.475 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.476 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.476 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.476 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.476 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2023-07-06T20:37:03.476 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-06T20:37:03.476 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.477 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.477 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.477 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.477 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.477 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.477 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.477 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.477 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.478 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.478 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.478 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-06T20:37:03.478 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-06T20:37:03.478 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.478 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.479 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.479 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.479 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.479 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.479 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.479 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.479 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.479 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.480 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.480 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-06T20:37:03.480 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-06T20:37:03.480 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.480 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.480 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.480 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.481 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.481 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.481 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.481 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.481 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.481 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.481 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.481 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.482 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-06T20:37:03.482 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-06T20:37:03.482 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.482 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.482 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.482 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.482 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.482 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.483 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.483 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.483 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.483 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.483 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.483 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-06T20:37:03.483 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-06T20:37:03.484 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.484 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.484 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.484 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.484 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.484 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.484 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.485 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.485 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.485 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.485 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-06T20:37:03.485 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-06T20:37:03.485 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.486 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.486 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.486 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.486 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.486 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.486 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.486 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.486 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.487 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.487 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.487 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.487 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-06T20:37:03.487 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-06T20:37:03.487 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.487 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.488 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.488 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.488 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.488 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.488 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.488 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.488 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.488 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.489 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.489 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.489 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-06T20:37:03.489 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-06T20:37:03.489 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.489 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.489 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.489 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.490 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.490 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.490 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.490 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.490 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.490 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.490 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.490 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.491 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-06T20:37:03.491 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-06T20:37:03.491 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.491 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.491 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.491 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.491 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.491 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.492 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.492 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.492 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.492 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.492 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.492 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.492 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2023-07-06T20:37:03.492 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-06T20:37:03.493 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.493 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.493 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.493 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:03.493 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.493 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.493 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.493 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.494 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.494 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.494 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.494 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.494 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-06T20:37:03.494 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-06T20:37:03.494 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.494 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.495 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.495 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.495 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.495 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.495 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.495 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.495 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.495 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.496 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.496 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.496 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2023-07-06T20:37:03.496 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2023-07-06T20:37:03.496 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.496 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.496 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.496 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.497 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.497 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.497 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.497 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.497 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.497 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.497 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.497 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.498 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2023-07-06T20:37:03.498 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-06T20:37:03.498 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.498 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.498 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.498 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:03.498 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.498 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.499 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.499 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.499 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.499 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.499 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.499 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.499 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-06T20:37:03.499 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-06T20:37:03.500 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.500 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.500 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.500 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.500 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.500 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.500 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.500 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.501 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.501 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.501 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.501 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.501 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-06T20:37:03.501 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-06T20:37:03.501 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.502 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.502 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.502 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.502 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.502 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.502 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.502 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.502 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.503 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.503 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.503 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.503 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-06T20:37:03.503 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-06T20:37:03.503 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.503 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.503 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.504 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.504 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.504 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.504 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.504 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.504 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.504 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.505 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.505 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-06T20:37:03.505 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-06T20:37:03.505 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.505 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.505 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.505 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.505 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.506 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.506 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.506 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.506 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.506 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.506 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.506 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.506 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2023-07-06T20:37:03.507 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-06T20:37:03.507 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.507 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.507 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.507 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.507 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.507 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.507 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.508 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.508 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.508 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.508 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.508 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-06T20:37:03.508 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-06T20:37:03.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.509 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.509 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.509 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.509 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.509 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.509 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.509 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.510 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.510 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.510 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.510 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.510 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-06T20:37:03.510 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-06T20:37:03.510 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.510 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.511 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.511 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.511 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.511 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.511 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.511 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.511 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.512 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.512 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.512 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.512 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-06T20:37:03.512 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-06T20:37:03.512 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.512 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.512 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.513 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.513 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.513 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.513 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.513 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.513 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.513 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.513 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.514 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.514 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-06T20:37:03.514 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-06T20:37:03.514 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.514 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.514 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.514 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.514 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.515 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.515 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.515 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.515 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.515 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.515 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.515 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-06T20:37:03.516 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-06T20:37:03.516 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.516 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.516 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.516 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.516 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.516 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.517 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.517 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.517 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.517 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.517 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.517 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.517 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-06T20:37:03.517 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-06T20:37:03.518 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.518 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.518 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.518 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.518 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.518 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.518 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.519 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.519 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.519 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.519 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.519 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-06T20:37:03.519 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-06T20:37:03.519 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.519 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.520 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.520 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.520 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.520 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.520 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.520 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.520 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.521 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.521 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.521 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.521 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-06T20:37:03.521 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-06T20:37:03.521 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.521 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.521 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.522 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.522 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.522 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.522 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.522 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.522 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.522 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.522 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.523 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.523 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2023-07-06T20:37:03.523 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-06T20:37:03.523 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.523 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.523 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.523 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.523 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.524 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.524 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.524 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.524 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.524 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.524 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.524 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.525 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-06T20:37:03.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-06T20:37:03.525 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.525 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.525 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.525 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-06T20:37:03.525 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.525 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.526 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.526 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.526 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.526 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.526 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.526 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.526 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-06T20:37:03.526 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-06T20:37:03.527 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.527 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.527 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.527 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:03.527 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.527 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.527 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.527 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.528 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.528 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.528 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.528 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.528 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-06T20:37:03.528 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-06T20:37:03.528 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.528 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.529 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.529 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2023-07-06T20:37:03.529 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.529 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.529 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.529 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.529 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.529 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.530 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.530 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.530 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-06T20:37:03.530 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-06T20:37:03.530 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.530 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.530 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2023-07-06T20:37:03.531 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.531 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.531 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.531 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.531 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.531 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.531 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.532 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.532 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2023-07-06T20:37:03.532 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-06T20:37:03.532 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.532 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.532 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.532 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.532 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.533 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.533 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.533 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.533 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.533 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.533 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.533 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.533 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-06T20:37:03.534 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-06T20:37:03.534 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.534 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.534 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.534 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:03.534 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.534 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.534 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.535 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.535 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.535 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.535 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.535 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.535 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2023-07-06T20:37:03.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-06T20:37:03.535 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.536 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.536 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.536 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.536 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.536 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.536 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.536 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.537 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.537 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.537 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.537 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.537 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2023-07-06T20:37:03.537 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-06T20:37:03.537 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.537 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.538 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.538 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.538 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.538 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.538 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.538 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.538 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.539 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.539 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-06T20:37:03.539 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-06T20:37:03.539 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.539 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.539 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.539 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:03.540 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.540 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.540 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.540 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.540 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.540 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.540 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.540 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.541 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-06T20:37:03.541 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-06T20:37:03.541 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.541 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.541 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.541 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:03.541 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.541 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.542 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.542 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.542 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.542 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.542 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.542 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.542 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-06T20:37:03.542 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-06T20:37:03.543 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.543 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.543 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.543 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2023-07-06T20:37:03.543 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.543 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.543 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.543 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.544 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.544 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.544 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.544 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.544 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2023-07-06T20:37:03.544 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2023-07-06T20:37:03.544 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.544 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.545 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.545 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.545 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.545 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.545 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.545 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.545 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.546 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.546 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.546 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.546 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2023-07-06T20:37:03.546 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2023-07-06T20:37:03.546 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.546 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.546 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.547 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.547 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.547 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.547 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.547 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.547 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.547 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.548 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2023-07-06T20:37:03.548 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2023-07-06T20:37:03.548 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.548 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.548 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.548 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.549 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.549 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.549 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.549 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2023-07-06T20:37:03.549 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.549 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.549 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.550 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2023-07-06T20:37:03.550 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2023-07-06T20:37:03.550 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.550 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.550 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.550 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2023-07-06T20:37:03.550 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.550 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.551 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.551 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.551 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.551 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.551 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.551 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.551 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2023-07-06T20:37:03.551 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2023-07-06T20:37:03.552 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.552 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.552 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.552 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.552 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.552 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.552 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.552 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.553 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.553 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.553 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.553 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.553 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.553 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.553 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.553 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.554 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.554 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.554 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.554 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.554 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.554 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.554 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.555 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.555 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.555 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.555 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.555 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.555 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.555 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.556 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.556 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.556 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.556 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.556 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.556 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.556 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.557 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.557 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.557 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.557 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.557 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.557 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.557 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.557 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.558 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.558 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.558 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.558 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.558 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.558 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.558 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.559 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.559 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.559 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.559 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.559 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.559 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.559 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.559 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.560 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.560 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.560 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.560 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.560 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.560 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.560 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.561 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.561 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.561 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.561 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.561 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.561 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.561 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.561 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.562 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.562 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.562 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.562 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.562 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.562 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.562 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2023-07-06T20:37:03.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2023-07-06T20:37:03.563 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.563 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.563 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.563 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.563 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.563 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.563 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.564 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2023-07-06T20:37:03.564 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.564 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.564 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.564 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.564 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2023-07-06T20:37:03.564 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2023-07-06T20:37:03.564 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.565 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.565 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.565 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.565 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.565 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.565 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.565 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.565 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.566 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.566 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.566 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.566 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2023-07-06T20:37:03.566 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2023-07-06T20:37:03.566 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.566 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.566 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.567 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2023-07-06T20:37:03.567 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.567 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.567 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.567 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.567 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.567 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.568 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.568 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2023-07-06T20:37:03.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2023-07-06T20:37:03.568 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.568 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.568 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.568 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2023-07-06T20:37:03.569 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.569 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.569 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.569 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.569 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.569 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.569 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.570 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2023-07-06T20:37:03.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2023-07-06T20:37:03.570 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.570 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.570 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.570 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.570 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.571 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.571 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.571 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.571 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.571 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.571 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.571 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2023-07-06T20:37:03.572 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2023-07-06T20:37:03.572 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.572 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.572 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.572 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2023-07-06T20:37:03.572 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.572 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.572 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.573 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.573 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.573 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.573 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.573 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2023-07-06T20:37:03.573 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2023-07-06T20:37:03.573 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.574 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.574 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.574 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2023-07-06T20:37:03.574 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.574 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.574 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.574 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.574 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2023-07-06T20:37:03.575 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.575 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.575 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.575 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.575 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.575 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.575 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2023-07-06T20:37:03.575 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2023-07-06T20:37:03.576 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.576 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.576 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.576 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2023-07-06T20:37:03.576 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2023-07-06T20:37:03.576 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2023-07-06T20:37:03.576 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.577 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.577 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.577 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.577 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.577 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.577 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2023-07-06T20:37:03.577 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2023-07-06T20:37:03.577 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.578 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.578 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.578 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.578 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.578 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.578 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.578 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.578 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.579 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.579 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.579 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.579 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.579 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.579 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.579 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2023-07-06T20:37:03.579 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.580 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.580 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.580 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2023-07-06T20:37:03.580 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2023-07-06T20:37:03.580 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.580 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.580 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.581 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.581 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.581 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.581 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.581 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2023-07-06T20:37:03.581 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.581 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.581 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.582 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.582 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.582 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.582 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.582 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.582 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.582 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.583 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.583 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.583 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.583 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.583 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.583 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.583 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.583 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.584 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.584 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.584 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.584 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.584 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.584 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.584 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.584 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.585 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.585 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.585 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.585 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.585 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.585 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.585 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.585 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.586 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.586 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.586 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.586 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.586 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.586 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.586 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.587 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.587 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.587 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.587 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.587 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.587 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.587 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.587 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.588 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.588 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.588 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.588 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.588 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.588 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.588 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.588 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.589 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.589 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.589 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.589 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.589 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.589 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.589 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.589 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.590 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.590 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.590 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.590 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.590 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.590 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.590 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.591 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.591 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.591 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2023-07-06T20:37:03.591 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2023-07-06T20:37:03.591 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.591 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.591 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.591 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2023-07-06T20:37:03.592 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.592 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.592 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.592 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-06T20:37:03.592 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.592 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.592 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.592 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.593 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2023-07-06T20:37:03.593 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2023-07-06T20:37:03.593 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.593 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.593 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.593 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2023-07-06T20:37:03.593 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.593 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.594 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.594 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-06T20:37:03.594 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.594 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.594 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.594 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.594 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2023-07-06T20:37:03.594 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2023-07-06T20:37:03.595 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.595 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.595 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.595 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2023-07-06T20:37:03.595 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.595 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.595 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.596 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2023-07-06T20:37:03.596 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.596 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.596 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.596 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.596 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2023-07-06T20:37:03.596 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2023-07-06T20:37:03.596 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.597 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.597 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.597 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2023-07-06T20:37:03.597 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.597 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.597 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.597 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-06T20:37:03.597 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.598 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.598 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.598 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.598 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2023-07-06T20:37:03.598 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2023-07-06T20:37:03.598 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.598 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.599 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.599 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.599 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.599 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.599 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.599 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-06T20:37:03.599 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.599 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.600 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.600 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.600 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:03.600 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:03.600 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.600 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.600 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.600 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:03.601 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.601 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.601 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.601 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2023-07-06T20:37:03.601 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.601 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.601 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.602 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.602 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2023-07-06T20:37:03.602 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2023-07-06T20:37:03.602 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.602 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.602 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.602 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2023-07-06T20:37:03.602 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.603 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.603 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.603 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-06T20:37:03.603 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.603 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.603 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.603 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.603 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.604 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.604 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.604 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2023-07-06T20:37:03.604 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.604 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.604 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.604 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.605 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.605 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.605 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.605 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.605 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.605 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.605 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.605 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.606 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.606 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.606 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.606 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.606 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.606 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.606 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.607 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.607 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.607 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.607 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.607 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.607 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.607 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.608 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.608 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.608 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.608 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.608 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.608 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.608 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.608 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.609 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.609 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.609 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.609 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.609 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.609 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.609 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.610 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.610 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.610 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.610 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.610 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.610 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.610 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.610 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.611 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.611 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.611 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.611 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.611 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.611 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.611 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.612 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.612 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.612 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.612 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.612 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.612 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.613 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.613 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.613 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.613 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.613 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.613 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.613 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.613 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.614 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.614 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.614 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2023-07-06T20:37:03.614 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2023-07-06T20:37:03.614 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.614 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.614 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.615 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2023-07-06T20:37:03.615 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.615 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.615 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.615 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.615 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.615 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.616 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.616 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.616 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2023-07-06T20:37:03.616 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2023-07-06T20:37:03.616 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.616 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.616 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.616 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2023-07-06T20:37:03.617 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.617 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.617 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.617 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.617 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.617 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.617 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.618 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.618 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:03.618 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:03.618 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.618 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.618 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.618 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:03.618 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.619 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.619 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.619 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.619 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.619 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.619 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.619 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.619 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.620 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.620 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.620 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2023-07-06T20:37:03.620 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2023-07-06T20:37:03.620 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2023-07-06T20:37:03.620 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.620 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2023-07-06T20:37:03.621 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2023-07-06T20:37:03.621 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.621 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.621 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.621 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2023-07-06T20:37:03.621 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.621 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.621 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.622 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-06T20:37:03.622 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.622 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.622 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.622 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.622 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2023-07-06T20:37:03.622 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2023-07-06T20:37:03.622 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.623 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.623 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.623 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2023-07-06T20:37:03.623 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.623 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.623 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.623 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-06T20:37:03.623 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.624 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.624 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.624 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.624 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2023-07-06T20:37:03.624 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2023-07-06T20:37:03.624 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.624 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.625 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.625 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.625 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.625 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.625 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2023-07-06T20:37:03.625 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.625 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.625 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.626 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.626 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:03.626 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:03.626 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.626 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.626 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.626 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2023-07-06T20:37:03.627 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2023-07-06T20:37:03.627 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.627 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.627 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-06T20:37:03.627 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.627 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.627 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.627 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.628 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.628 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.628 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.628 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.628 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.628 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.628 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.629 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.629 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.629 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.629 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.629 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.629 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.629 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.630 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.630 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.630 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.630 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.630 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.630 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.630 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.630 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.631 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.631 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.631 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.631 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.631 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.631 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.631 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.631 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.632 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.632 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.632 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.632 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.632 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.632 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.632 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.632 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.633 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.633 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.633 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.633 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.633 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.633 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.633 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.634 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.634 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.634 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.634 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.634 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.634 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.634 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.634 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.635 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.635 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.635 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.635 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.635 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.635 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.635 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.635 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.636 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.636 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.636 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.636 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.636 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.636 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.636 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.637 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.637 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.637 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2023-07-06T20:37:03.637 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2023-07-06T20:37:03.637 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.637 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.637 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.637 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.638 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.638 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.638 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.638 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2023-07-06T20:37:03.638 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.638 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.638 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.638 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.639 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2023-07-06T20:37:03.639 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2023-07-06T20:37:03.639 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.639 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.639 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.639 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2023-07-06T20:37:03.639 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.639 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.640 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.640 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2023-07-06T20:37:03.640 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.640 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.640 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.640 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.640 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2023-07-06T20:37:03.640 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2023-07-06T20:37:03.641 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.641 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.641 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.641 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2023-07-06T20:37:03.641 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.641 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.641 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.641 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-06T20:37:03.642 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.642 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.642 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.642 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.642 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2023-07-06T20:37:03.642 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2023-07-06T20:37:03.642 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.642 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.643 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.643 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2023-07-06T20:37:03.643 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2023-07-06T20:37:03.643 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2023-07-06T20:37:03.643 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.643 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-06T20:37:03.643 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.643 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.644 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.644 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.644 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2023-07-06T20:37:03.644 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2023-07-06T20:37:03.644 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.644 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.644 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.645 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.645 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.645 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.645 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.645 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2023-07-06T20:37:03.645 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.645 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.645 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.646 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.646 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2023-07-06T20:37:03.646 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2023-07-06T20:37:03.646 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.646 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.646 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.646 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2023-07-06T20:37:03.646 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.647 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.647 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.647 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-06T20:37:03.647 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.647 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.647 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.647 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.647 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.648 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.648 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.648 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2023-07-06T20:37:03.648 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.648 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.648 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.648 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.649 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.649 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.649 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.649 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.649 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.649 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.649 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.649 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.650 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.650 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.650 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.650 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.650 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.650 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.650 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.650 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.651 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.651 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.651 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.651 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.651 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.651 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.651 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.651 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.652 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.652 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.652 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.652 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.652 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.652 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.652 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.652 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.653 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.653 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.653 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.653 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.653 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.653 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.653 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.654 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.654 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.654 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.654 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.654 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.654 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.654 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.654 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.655 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.655 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.655 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.655 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.655 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.655 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.655 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.655 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.656 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.656 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.656 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.656 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.656 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.656 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.656 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.656 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.657 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.657 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.657 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.657 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.657 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.657 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.657 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.658 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.658 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.658 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2023-07-06T20:37:03.658 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.658 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.658 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.658 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.658 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.659 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.659 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.659 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.659 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.659 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.659 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.659 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.659 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.660 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.660 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.660 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.660 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.660 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.660 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.660 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.661 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.661 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.661 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.661 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.661 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.661 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.661 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.661 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.662 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.662 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.662 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.662 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.662 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.662 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.662 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.663 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.663 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.663 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.663 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.663 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.663 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.663 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.663 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.664 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.664 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.664 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.664 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.664 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.664 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.664 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.664 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.665 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.665 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.665 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.665 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.665 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.665 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.665 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.665 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.666 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.666 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.666 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.666 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.666 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.666 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.666 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.666 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.667 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.667 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.667 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.667 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.667 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.667 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.667 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.668 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.668 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.668 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2023-07-06T20:37:03.668 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.668 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.668 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.668 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2023-07-06T20:37:03.668 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2023-07-06T20:37:03.669 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.669 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.669 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.669 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2023-07-06T20:37:03.669 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.669 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.669 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.669 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-06T20:37:03.670 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.670 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.670 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.670 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.670 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2023-07-06T20:37:03.670 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2023-07-06T20:37:03.670 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.670 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.671 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.671 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:03.671 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2023-07-06T20:37:03.671 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.671 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.671 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-06T20:37:03.671 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.671 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.672 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.672 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.672 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.672 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.672 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.672 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.672 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.673 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.673 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.673 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.673 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.673 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.673 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.673 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.673 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.674 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.674 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.674 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.674 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.674 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.674 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.674 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.674 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.675 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.675 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.675 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.675 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.675 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.675 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.675 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.675 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.676 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.677 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.678 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.679 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.680 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.681 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.681 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.681 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.681 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.681 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.681 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.681 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2023-07-06T20:37:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2023-07-06T20:37:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2023-07-06T20:37:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.682 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2023-07-06T20:37:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2023-07-06T20:37:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-06T20:37:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.683 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.684 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.685 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.686 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.687 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.688 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.689 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.689 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.689 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.689 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.689 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.689 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.689 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.689 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.690 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.691 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.692 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2023-07-06T20:37:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2023-07-06T20:37:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.693 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.694 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.694 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2023-07-06T20:37:03.694 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-06T20:37:03.694 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.694 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.694 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.694 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2023-07-06T20:37:03.695 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2023-07-06T20:37:03.695 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.695 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.695 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.695 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2023-07-06T20:37:03.695 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.695 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.695 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.696 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2023-07-06T20:37:03.696 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.696 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.696 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.696 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.696 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2023-07-06T20:37:03.696 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2023-07-06T20:37:03.696 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.697 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.697 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.697 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2023-07-06T20:37:03.697 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.697 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.697 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.697 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2023-07-06T20:37:03.697 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.698 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.698 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.698 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.698 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2023-07-06T20:37:03.698 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2023-07-06T20:37:03.698 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.698 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.698 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.699 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.699 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.699 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.699 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.699 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2023-07-06T20:37:03.699 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.699 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.699 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.700 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.700 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2023-07-06T20:37:03.700 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2023-07-06T20:37:03.700 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.700 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.700 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.700 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2023-07-06T20:37:03.700 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.701 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.701 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.701 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-06T20:37:03.701 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-06T20:37:03.701 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.701 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.701 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.701 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.702 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.702 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.702 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2023-07-06T20:37:03.702 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.702 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.702 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.702 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.703 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.703 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.703 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.703 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.703 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.703 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.703 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.703 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.704 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.704 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.704 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.704 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.704 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.704 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.704 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.705 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.705 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.705 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.705 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.705 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.705 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.705 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.705 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.706 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.706 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.706 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.706 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.706 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.706 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.706 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.706 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.707 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.707 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.707 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.707 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.707 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.707 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.707 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.707 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.708 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.708 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.708 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.708 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.708 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.708 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.708 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.708 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.709 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.709 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.709 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.709 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.709 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.709 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.709 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.709 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.710 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.710 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.710 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.710 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.710 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.710 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.710 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.710 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.711 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.711 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.711 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.711 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.711 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.711 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.711 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.711 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.712 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.712 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2023-07-06T20:37:03.712 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.712 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.712 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.712 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.712 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.713 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.713 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.713 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.713 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.713 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.713 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.713 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.713 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.714 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.714 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.714 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.714 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.714 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.714 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.714 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.714 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.715 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.715 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.715 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.715 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.715 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.715 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.715 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.715 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.716 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.716 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.716 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.716 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.716 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.716 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.716 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.716 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.717 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.717 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.717 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.717 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.717 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.717 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.717 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.717 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.718 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.718 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.718 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.718 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.718 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.718 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.718 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.718 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.719 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.719 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.719 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.719 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.719 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.719 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.719 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.719 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.720 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.720 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.720 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.720 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.720 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.720 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.720 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.721 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.721 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.721 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.721 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.721 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.721 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.721 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.721 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.722 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2023-07-06T20:37:03.722 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.722 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.722 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.722 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.722 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.722 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.722 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.723 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.723 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.723 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.723 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.723 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.723 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.723 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.723 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.724 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.724 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.724 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.724 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.724 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.724 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.724 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.724 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.725 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.725 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.725 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.725 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.725 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.725 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.725 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.726 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.726 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.726 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.726 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.726 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.726 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.726 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.726 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.727 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.727 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.727 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.727 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.727 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.727 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.727 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.728 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.728 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.728 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.728 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.728 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.728 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.728 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.729 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.729 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.729 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.729 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.729 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.729 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.729 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.729 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.730 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.730 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.730 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.730 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.730 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.730 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.730 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.730 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.731 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.731 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.731 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.731 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.731 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.731 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.731 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.732 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.732 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2023-07-06T20:37:03.732 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.732 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.732 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.732 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2023-07-06T20:37:03.732 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2023-07-06T20:37:03.732 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.733 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.733 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.733 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.733 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.733 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.733 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.733 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-06T20:37:03.733 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.734 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.734 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.734 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.734 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.734 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.734 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.734 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.734 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.735 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.735 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.735 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.735 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.735 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.735 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.735 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.735 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.736 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.736 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.736 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.736 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.736 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.736 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.736 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.737 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.737 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.737 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.737 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.737 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.737 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.737 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.737 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.738 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.738 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.738 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.738 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.738 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.738 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.738 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.738 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.739 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.739 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.739 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.739 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.739 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.739 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.739 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.740 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.740 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.740 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.740 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.740 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.740 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.740 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.740 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.741 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.741 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.741 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.741 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.741 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.741 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.741 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.742 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.742 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.742 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.742 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.742 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.742 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.742 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.742 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.743 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.743 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.743 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.743 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2023-07-06T20:37:03.743 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2023-07-06T20:37:03.743 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.743 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.743 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.744 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.744 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.744 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.744 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.744 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2023-07-06T20:37:03.744 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2023-07-06T20:37:03.744 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2023-07-06T20:37:03.744 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.745 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2023-07-06T20:37:03.745 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.745 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.745 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.745 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.745 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.745 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.745 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.746 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2023-07-06T20:37:03.746 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.746 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.746 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.746 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.746 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.746 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.746 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.747 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.747 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.747 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.747 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.747 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.747 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.747 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.748 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.748 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.748 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.748 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.748 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.748 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.748 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.748 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.749 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.749 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.749 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.749 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.749 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.749 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.749 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.749 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.750 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.750 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.750 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.750 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.750 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.750 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.750 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.750 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.751 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.751 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.751 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.751 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.751 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.751 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.751 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.751 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.752 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.752 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.752 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.752 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.752 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.752 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.752 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.752 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.753 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.753 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.753 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.753 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.753 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.753 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.754 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.754 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.754 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.754 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.754 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.754 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.754 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.755 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.755 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.755 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.755 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.755 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.755 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.755 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.755 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.756 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2023-07-06T20:37:03.756 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.756 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.756 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.756 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.756 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.756 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.756 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.757 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.757 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.757 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.757 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.757 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.757 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.757 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.757 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.758 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.758 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.758 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.758 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.758 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.758 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.758 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.759 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.759 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.759 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.759 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.759 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.759 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.759 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.759 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.760 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.760 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.760 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.760 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.760 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.760 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.760 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.761 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.761 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.761 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.761 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.761 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.761 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.761 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.762 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.762 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.762 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.762 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.763 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.763 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.763 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.763 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.763 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.763 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.763 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.764 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.764 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.764 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.764 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.764 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.764 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.764 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.765 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.765 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.765 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.765 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.765 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.765 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.765 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.766 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.766 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.766 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.766 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.766 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.766 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.766 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2023-07-06T20:37:03.766 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.767 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.767 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.767 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.767 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.767 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.767 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.767 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.767 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.768 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.768 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.768 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.768 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.768 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.768 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.768 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.769 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.769 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.769 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.769 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.769 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.769 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.769 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.769 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.771 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.771 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.771 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.771 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.771 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.771 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.771 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.772 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.772 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.772 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.772 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.772 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.772 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.772 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.773 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.773 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.773 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.773 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.773 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.773 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.773 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.773 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.774 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.774 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.774 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.774 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.774 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.774 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.774 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.774 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.775 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.775 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.775 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.775 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.775 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.775 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.775 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.775 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.776 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.776 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.776 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.776 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.776 INFO:teuthology.orchestra.run.smithi069.stdout: "noautoscale": { 2023-07-06T20:37:03.776 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "noautoscale", 2023-07-06T20:37:03.776 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.777 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.777 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.777 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.777 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.777 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.777 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.777 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "global autoscale flag", 2023-07-06T20:37:03.778 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-06T20:37:03.778 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.778 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.778 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.778 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:03.778 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:03.778 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.778 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.779 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.779 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:03.779 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.779 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.779 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.779 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.779 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.780 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.780 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.780 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.780 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2023-07-06T20:37:03.780 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2023-07-06T20:37:03.780 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:03.781 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.781 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.781 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2023-07-06T20:37:03.781 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2023-07-06T20:37:03.781 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.781 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.781 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2023-07-06T20:37:03.782 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-06T20:37:03.782 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.782 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.782 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.782 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.782 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.782 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.782 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2023-07-06T20:37:03.783 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.783 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.783 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.783 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2023-07-06T20:37:03.783 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2023-07-06T20:37:03.783 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.783 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.784 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.784 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.784 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.784 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.784 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.784 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-06T20:37:03.784 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.784 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.785 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.785 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.785 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2023-07-06T20:37:03.785 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2023-07-06T20:37:03.785 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.785 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.785 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.786 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.786 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.786 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.786 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.786 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.786 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.786 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.786 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.787 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.787 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.787 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.787 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.787 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.787 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.787 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.788 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.788 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.788 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.788 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.788 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.788 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.788 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.788 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.789 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.789 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.789 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.789 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.789 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.789 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.789 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.789 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.790 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.790 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.790 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.790 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.790 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.790 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.790 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.790 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.791 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.791 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.791 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.791 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.791 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.791 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.791 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.792 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.792 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.792 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.792 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.792 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.792 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.792 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.793 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.793 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.793 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.793 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.793 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.793 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.793 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.793 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.794 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.794 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.794 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.794 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.794 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.794 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.794 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.794 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.795 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.795 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.795 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.795 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.795 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.795 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.795 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.795 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.796 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.796 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.796 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2023-07-06T20:37:03.796 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2023-07-06T20:37:03.796 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.796 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.796 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.797 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2023-07-06T20:37:03.797 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.797 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.797 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.797 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2023-07-06T20:37:03.797 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.797 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.797 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.798 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.798 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:03.798 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:03.798 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.798 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.798 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.798 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2023-07-06T20:37:03.799 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.799 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.799 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.799 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2023-07-06T20:37:03.799 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.799 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.799 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.800 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.800 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.800 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.800 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.800 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2023-07-06T20:37:03.800 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.800 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.801 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.801 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2023-07-06T20:37:03.801 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2023-07-06T20:37:03.801 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.801 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.801 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.801 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.801 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.802 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.802 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.802 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.802 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.802 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.802 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.802 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.803 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2023-07-06T20:37:03.803 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2023-07-06T20:37:03.803 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.803 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.803 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.803 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.803 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.804 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.804 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.804 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-06T20:37:03.804 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-06T20:37:03.804 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.804 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.805 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.805 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.805 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.805 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.805 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.805 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.805 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.806 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.806 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.806 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.806 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.806 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.806 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.806 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.807 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.807 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.807 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.807 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.807 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.807 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.807 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.807 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.808 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.808 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.808 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.808 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.808 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.808 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.809 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.809 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.809 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.809 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.809 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.809 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.809 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.810 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.810 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.810 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.810 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.810 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.810 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.810 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.810 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.811 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.811 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.811 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.811 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.811 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.811 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.811 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.811 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.812 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.812 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.812 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.812 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.812 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.812 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.812 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.812 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.813 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.813 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.813 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.813 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.813 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.813 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.813 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.814 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.814 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.814 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.814 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.814 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2023-07-06T20:37:03.814 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2023-07-06T20:37:03.814 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.814 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.815 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.815 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.815 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.815 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.815 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.815 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.815 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.815 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.816 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.816 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.816 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-06T20:37:03.816 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-06T20:37:03.816 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.816 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.816 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.817 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2023-07-06T20:37:03.817 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.817 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.817 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.817 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.817 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.817 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.817 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.818 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.818 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2023-07-06T20:37:03.818 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2023-07-06T20:37:03.818 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:03.818 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.818 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.818 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2023-07-06T20:37:03.818 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.819 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.819 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.819 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.819 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.819 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.819 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.819 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.819 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2023-07-06T20:37:03.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2023-07-06T20:37:03.820 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.820 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.820 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.820 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2023-07-06T20:37:03.820 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.820 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.821 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.821 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-06T20:37:03.821 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.821 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.821 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.821 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.821 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2023-07-06T20:37:03.821 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2023-07-06T20:37:03.822 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.822 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.822 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.822 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2023-07-06T20:37:03.822 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.822 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.822 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.822 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-06T20:37:03.823 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.823 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.823 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.823 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.823 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2023-07-06T20:37:03.823 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2023-07-06T20:37:03.823 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.823 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.824 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.824 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2023-07-06T20:37:03.824 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.824 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.824 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.824 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.824 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.824 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.825 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.825 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.825 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2023-07-06T20:37:03.825 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2023-07-06T20:37:03.825 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.825 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.825 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.826 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2023-07-06T20:37:03.826 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.826 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.826 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.826 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2023-07-06T20:37:03.826 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2023-07-06T20:37:03.826 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.826 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.827 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.827 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.827 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.827 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.827 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2023-07-06T20:37:03.827 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2023-07-06T20:37:03.827 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.828 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.828 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.828 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2023-07-06T20:37:03.828 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2023-07-06T20:37:03.828 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2023-07-06T20:37:03.828 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.828 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.828 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.829 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.829 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.829 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.829 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.829 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.829 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.830 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2023-07-06T20:37:03.830 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.830 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.830 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.830 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.830 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.830 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.830 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.831 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.831 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.831 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.831 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.831 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.831 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.831 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.831 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.832 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.832 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.832 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.832 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.832 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.832 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.832 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.833 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.833 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.833 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.833 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.833 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.833 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.833 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.833 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.834 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.834 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.834 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.834 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.834 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.834 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.834 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.834 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.835 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.835 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.835 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.835 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.835 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.835 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.835 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.836 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.836 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.836 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.836 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.836 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.836 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.836 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.836 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.837 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.837 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.837 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.837 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.837 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.837 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.837 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.838 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.838 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.838 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.838 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.838 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.838 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.838 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.838 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.839 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.839 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.839 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.839 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.839 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.839 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2023-07-06T20:37:03.839 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2023-07-06T20:37:03.839 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.840 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.840 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.840 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:03.840 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.840 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.840 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.840 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.841 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.841 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.841 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.841 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.841 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2023-07-06T20:37:03.841 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2023-07-06T20:37:03.841 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.841 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.842 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.842 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.842 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.842 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.842 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.842 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.842 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.842 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.843 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.843 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.843 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2023-07-06T20:37:03.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2023-07-06T20:37:03.843 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.843 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.843 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.844 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.844 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.844 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.844 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.844 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.844 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.844 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.845 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.845 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.845 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.845 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.845 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2023-07-06T20:37:03.845 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.846 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.846 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.846 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2023-07-06T20:37:03.846 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2023-07-06T20:37:03.846 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.846 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.847 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.847 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.847 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.847 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.847 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.847 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.848 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.848 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.848 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.848 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.848 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2023-07-06T20:37:03.848 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2023-07-06T20:37:03.849 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.849 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.849 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.849 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.849 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.850 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.850 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.850 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.850 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.851 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.851 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.851 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.851 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.852 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.852 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.852 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.852 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.852 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.852 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.852 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.853 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.853 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.853 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.853 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.853 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.853 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.853 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.854 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.854 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.854 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.854 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.854 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.854 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.854 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.855 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.855 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.855 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.856 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.856 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.856 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.857 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.857 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.857 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.857 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.858 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.858 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.858 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.858 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.858 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.858 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.858 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.859 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.859 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.859 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.859 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.859 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.860 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.860 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.860 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.860 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.860 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.860 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.860 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.861 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.861 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.861 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.861 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.861 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.861 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.861 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.861 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.862 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.862 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.862 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.862 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.862 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.862 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.862 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.862 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.863 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.863 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.863 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.863 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2023-07-06T20:37:03.863 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2023-07-06T20:37:03.863 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.863 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.863 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.864 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.864 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.864 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.864 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.864 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.865 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.865 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.865 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.865 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2023-07-06T20:37:03.865 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2023-07-06T20:37:03.865 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.865 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.865 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.866 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.866 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.866 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.866 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.866 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.866 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.866 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.867 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.867 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.867 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.867 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.867 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.867 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2023-07-06T20:37:03.867 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.867 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.868 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.868 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.868 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.868 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.868 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.868 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.868 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.868 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.869 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.869 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.869 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.869 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.869 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.869 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.869 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.870 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.870 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.870 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.870 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.870 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.870 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.870 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.870 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.871 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.871 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.871 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.871 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.871 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.871 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.871 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.871 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.872 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.872 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.872 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.872 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.872 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.872 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.872 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.873 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.873 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.873 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.873 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.873 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.873 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.874 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.874 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.874 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.874 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.874 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.875 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.875 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.875 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.875 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.875 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.875 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.876 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.876 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.876 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.876 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.876 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.876 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.877 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.877 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.877 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.877 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.877 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.878 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.878 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.878 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.878 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.878 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.879 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.879 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.879 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.879 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2023-07-06T20:37:03.879 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.880 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.880 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.880 INFO:teuthology.orchestra.run.smithi069.stdout: "drive_group_interval": { 2023-07-06T20:37:03.881 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "drive_group_interval", 2023-07-06T20:37:03.881 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:03.881 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.881 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.881 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300.0", 2023-07-06T20:37:03.882 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.882 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.882 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.882 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-06T20:37:03.882 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.883 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.883 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.883 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.883 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.883 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.883 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.884 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.884 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.884 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.884 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.884 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.884 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.884 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.884 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.885 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.885 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.885 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.885 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.885 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.885 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.885 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.885 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.886 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.886 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.886 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.886 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.886 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.886 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.886 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.887 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.888 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.888 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.888 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.888 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.888 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.888 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.888 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.889 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.889 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.889 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.889 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.889 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.889 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.890 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.890 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.890 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.890 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.890 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.890 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.890 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.890 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.891 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.891 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.891 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.891 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.891 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.891 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.891 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.891 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.892 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.892 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.892 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.892 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.892 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.892 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.892 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.893 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.893 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.893 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.893 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.893 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.893 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.893 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2023-07-06T20:37:03.893 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2023-07-06T20:37:03.894 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.894 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.894 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.894 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2023-07-06T20:37:03.894 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.894 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.894 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.894 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-06T20:37:03.895 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.895 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.895 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.895 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.895 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.895 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.895 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.895 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2023-07-06T20:37:03.896 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.896 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.896 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.896 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.896 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.896 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.896 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.896 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.897 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.897 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.897 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.897 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.897 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.897 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.897 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.897 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.898 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.898 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.898 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.898 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.898 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.898 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.898 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.898 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.899 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.899 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.899 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.899 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.899 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.899 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.900 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.900 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.900 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.900 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.900 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.900 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.900 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.901 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.901 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.901 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.901 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.901 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.902 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.902 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.902 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.902 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.902 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.902 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.902 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.903 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.903 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.903 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.903 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.903 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.903 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.904 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.904 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.904 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.904 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.904 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.904 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.905 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.905 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.905 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.905 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.905 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.905 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.906 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.906 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.906 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.906 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2023-07-06T20:37:03.906 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2023-07-06T20:37:03.906 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.907 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.907 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.907 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.907 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.907 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.908 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.908 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.908 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.908 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.908 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.908 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2023-07-06T20:37:03.909 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2023-07-06T20:37:03.909 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.909 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.909 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.909 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2023-07-06T20:37:03.909 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.909 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.909 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.910 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.910 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.910 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.910 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.910 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.910 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2023-07-06T20:37:03.911 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2023-07-06T20:37:03.911 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.911 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.911 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.911 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.911 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.912 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.912 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.912 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.912 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.912 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.912 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.912 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2023-07-06T20:37:03.913 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2023-07-06T20:37:03.913 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.913 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.913 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.913 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.913 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.913 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.914 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.914 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.914 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.914 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.914 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.914 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.914 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2023-07-06T20:37:03.914 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2023-07-06T20:37:03.915 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:03.915 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.915 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.915 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.915 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.915 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.915 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.915 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.916 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.916 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.916 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.916 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.916 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2023-07-06T20:37:03.916 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2023-07-06T20:37:03.916 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.917 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.917 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.917 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.917 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.917 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.917 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.917 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.918 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.918 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.918 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.918 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.918 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2023-07-06T20:37:03.918 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2023-07-06T20:37:03.918 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.918 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.919 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.919 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.919 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.919 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.919 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.919 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.919 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.919 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.920 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.920 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2023-07-06T20:37:03.920 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2023-07-06T20:37:03.920 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.920 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.920 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.920 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.921 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.921 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.921 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.921 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.921 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.921 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.921 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.921 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.922 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2023-07-06T20:37:03.922 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2023-07-06T20:37:03.922 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:03.922 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.922 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.922 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.922 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2023-07-06T20:37:03.922 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2023-07-06T20:37:03.923 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.923 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.923 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.923 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.923 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.923 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2023-07-06T20:37:03.923 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2023-07-06T20:37:03.924 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.924 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.924 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.924 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.924 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.924 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.924 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.925 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.925 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.925 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.925 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.925 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2023-07-06T20:37:03.925 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2023-07-06T20:37:03.925 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.926 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.926 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.926 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.926 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.926 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.926 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.926 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.926 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.927 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.927 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.927 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2023-07-06T20:37:03.927 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2023-07-06T20:37:03.927 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.927 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.927 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.928 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.928 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.928 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.928 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.928 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.928 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.929 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.929 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.929 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.929 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.929 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.929 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.929 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2023-07-06T20:37:03.929 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.930 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.930 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.930 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2023-07-06T20:37:03.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2023-07-06T20:37:03.930 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.930 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.931 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.931 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.931 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.931 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.931 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.931 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2023-07-06T20:37:03.931 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.932 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.932 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.932 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2023-07-06T20:37:03.932 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2023-07-06T20:37:03.932 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.932 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.933 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.933 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.933 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.933 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.933 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.934 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2023-07-06T20:37:03.934 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.934 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.934 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.934 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.934 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.935 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.935 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.935 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.935 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.935 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.935 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.936 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.936 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.936 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.936 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.936 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.937 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.937 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.937 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.937 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.937 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.937 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.938 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.938 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.938 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.938 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.938 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.938 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.938 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.939 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.939 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.939 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.939 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.939 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.939 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.939 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.939 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.940 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.940 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.940 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.940 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.940 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.940 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.941 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.941 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.941 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.941 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.941 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.941 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.941 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.941 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.942 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.942 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.942 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.942 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.942 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.942 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.942 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.942 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.943 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.943 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.943 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.943 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.943 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.943 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.943 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.944 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.944 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.944 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.944 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.944 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.944 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.944 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.945 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.945 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.945 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.945 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2023-07-06T20:37:03.945 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.945 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.945 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.945 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.946 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.946 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.946 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.946 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.946 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.946 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.946 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.946 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.947 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.947 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.947 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.947 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.947 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.947 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.947 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.947 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.948 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.948 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.948 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.948 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.948 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.948 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.948 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.948 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.949 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.949 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.949 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.949 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.949 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.949 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.949 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.949 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.950 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.950 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.950 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.950 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.950 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.950 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.950 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.950 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.951 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.951 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.951 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.951 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.951 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.951 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.951 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.952 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.952 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.952 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.952 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.952 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.952 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.952 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.952 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.953 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.953 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.953 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.953 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.953 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.954 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.954 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.954 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.954 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.954 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.954 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.955 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.955 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.955 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.955 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.955 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.955 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2023-07-06T20:37:03.956 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.956 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.956 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.956 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.956 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.956 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.956 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.957 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.957 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.957 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.957 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.957 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.957 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.958 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.958 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.958 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.958 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.958 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.958 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.958 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.959 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.959 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.959 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.959 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.959 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.959 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.960 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.960 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.960 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.960 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.960 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.961 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.961 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.961 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.961 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.961 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.961 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.962 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.962 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.962 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.962 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.962 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.962 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.963 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.963 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.963 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.963 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.963 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.963 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.963 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.964 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.964 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.964 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.964 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.964 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.964 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.964 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.965 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.965 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.965 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.965 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.965 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.965 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.965 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.965 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.966 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.966 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.966 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.966 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.966 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.966 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.967 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.967 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.967 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.967 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.967 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.967 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2023-07-06T20:37:03.967 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.968 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.968 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.968 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2023-07-06T20:37:03.968 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2023-07-06T20:37:03.968 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.968 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.968 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.969 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-06T20:37:03.969 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.969 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.969 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.969 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.969 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.969 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.969 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.970 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.970 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:03.970 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:03.970 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:03.970 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.970 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.970 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2023-07-06T20:37:03.970 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.971 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.971 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.971 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.971 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.971 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.971 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.971 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.971 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:03.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:03.972 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.972 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.972 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.972 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.972 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.973 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.973 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.973 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.973 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.973 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.973 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.973 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.973 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.974 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.974 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.974 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.974 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.974 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.974 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.974 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:03.974 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:03.975 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.975 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.975 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.975 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.975 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.975 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.975 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.975 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.976 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.976 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.976 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.976 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.976 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:03.976 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:03.976 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.977 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.977 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.977 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:03.977 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.977 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.977 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:03.977 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:03.977 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:03.978 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:03.978 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:03.978 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:03.978 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:03.978 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:03.978 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.978 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.978 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.979 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.979 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.979 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:03.979 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:03.979 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.979 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.979 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:03.980 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.980 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.980 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.980 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.980 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.980 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.980 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.980 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.981 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.981 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:03.981 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.981 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:03.981 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2023-07-06T20:37:03.981 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:03.981 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:03.981 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:03.982 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2023-07-06T20:37:03.982 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2023-07-06T20:37:03.982 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.982 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.982 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.982 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.982 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.982 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.983 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.983 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-06T20:37:03.983 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.983 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.983 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.983 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.983 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2023-07-06T20:37:03.984 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2023-07-06T20:37:03.984 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.984 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.984 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.984 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.984 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.984 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.985 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.985 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-06T20:37:03.985 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.985 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.985 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.985 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.985 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2023-07-06T20:37:03.986 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2023-07-06T20:37:03.986 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.986 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.986 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.986 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:03.986 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.986 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.987 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.987 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-06T20:37:03.987 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.987 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.987 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.987 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.988 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2023-07-06T20:37:03.988 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2023-07-06T20:37:03.988 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.988 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.988 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.988 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.988 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.989 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.989 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.989 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-06T20:37:03.989 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.989 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.989 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.990 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.990 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2023-07-06T20:37:03.990 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2023-07-06T20:37:03.990 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.990 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.990 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.991 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.991 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.991 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.991 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.991 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-06T20:37:03.991 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.992 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.992 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.992 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.992 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2023-07-06T20:37:03.992 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2023-07-06T20:37:03.993 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.993 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.993 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.993 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.993 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.993 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.994 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.994 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.994 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.994 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.994 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.994 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.994 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2023-07-06T20:37:03.994 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2023-07-06T20:37:03.995 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.995 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.995 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.995 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:03.995 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.995 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.995 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.996 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.996 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.996 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.996 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.996 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.996 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2023-07-06T20:37:03.996 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2023-07-06T20:37:03.996 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:03.997 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.997 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.997 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-06T20:37:03.997 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.997 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.997 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.997 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.997 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.998 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:03.998 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:03.998 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:03.998 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2023-07-06T20:37:03.998 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2023-07-06T20:37:03.998 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:03.998 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:03.999 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:03.999 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:03.999 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:03.999 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:03.999 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:03.999 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:03.999 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:03.999 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.000 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.000 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.000 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:04.000 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:04.000 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.000 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.000 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.001 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2023-07-06T20:37:04.001 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2023-07-06T20:37:04.001 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.001 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.001 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.001 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.001 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.001 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.002 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2023-07-06T20:37:04.002 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2023-07-06T20:37:04.002 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.002 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.002 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.002 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2023-07-06T20:37:04.002 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.003 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.003 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.003 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.003 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.003 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.003 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.003 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.003 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2023-07-06T20:37:04.004 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2023-07-06T20:37:04.004 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.004 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.004 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.004 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.004 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.004 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.005 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.005 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.005 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.005 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.005 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.005 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.005 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2023-07-06T20:37:04.005 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2023-07-06T20:37:04.006 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.006 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.006 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.006 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.006 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.006 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.006 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.006 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.007 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.007 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.007 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.007 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.007 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.007 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.007 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.008 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.008 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.008 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.008 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.008 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.008 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.008 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.008 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.009 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.009 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.009 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.009 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.009 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.009 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.009 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.010 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.010 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.010 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.010 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.010 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.010 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.010 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.011 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.011 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.011 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.011 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.011 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.011 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.011 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.012 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.012 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.012 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.012 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.012 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.012 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.013 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.013 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.013 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.013 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.013 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.013 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.014 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.014 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.014 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.014 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.014 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.014 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.014 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.014 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.015 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.015 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.015 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.015 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.015 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.016 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.016 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.016 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.016 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.016 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.017 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.017 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.017 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.017 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.017 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.017 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.018 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.018 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.018 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2023-07-06T20:37:04.018 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2023-07-06T20:37:04.019 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.019 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.019 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.019 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.019 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.019 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.020 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.020 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.020 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.020 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.020 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.020 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.020 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2023-07-06T20:37:04.021 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2023-07-06T20:37:04.021 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.021 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.021 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.021 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.021 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.021 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.022 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.022 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.022 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.022 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.022 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.022 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.022 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2023-07-06T20:37:04.022 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2023-07-06T20:37:04.023 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.023 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.023 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.023 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-06T20:37:04.023 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.023 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.024 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.024 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.024 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.024 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.024 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.025 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.025 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.025 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.025 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.025 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2023-07-06T20:37:04.026 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.026 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.026 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.026 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.026 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.026 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.026 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.027 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.027 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.027 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.027 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.027 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.027 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.028 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.028 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.028 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.028 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.028 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.028 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.028 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.028 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.029 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.029 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.029 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.029 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.029 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.029 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.029 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.029 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.030 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.030 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.030 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.030 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.030 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.030 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.030 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.030 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.031 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.031 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.031 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.031 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.031 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.031 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.031 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.031 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.032 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.032 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.032 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.032 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.032 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.032 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.032 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.033 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.033 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.033 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.033 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.033 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.033 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.033 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.033 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.034 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.034 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.034 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.034 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.034 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.034 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.034 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.035 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.035 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.035 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.035 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.035 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.035 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.035 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.036 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.036 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.036 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2023-07-06T20:37:04.036 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.036 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.036 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.036 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.037 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.037 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.037 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.037 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.037 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.037 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.037 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.037 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.038 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.038 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.038 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.038 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.038 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.038 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.038 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.038 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.039 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.039 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.039 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.039 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.039 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.039 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.039 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.040 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.040 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.040 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.040 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.040 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.040 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.040 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.041 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.041 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.041 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.041 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.041 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.041 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.041 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.041 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.042 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.042 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.042 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.042 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.042 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.042 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.042 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.043 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.043 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.043 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.043 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.043 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.043 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.044 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.044 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.044 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.044 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.044 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.044 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.044 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.045 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.045 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.045 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.045 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.045 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.046 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.046 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.046 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.046 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.046 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.047 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2023-07-06T20:37:04.047 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2023-07-06T20:37:04.047 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.047 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.047 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.048 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2023-07-06T20:37:04.048 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.048 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.048 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.048 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-06T20:37:04.048 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.048 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.049 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.049 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.049 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2023-07-06T20:37:04.049 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2023-07-06T20:37:04.049 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.049 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.049 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.049 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:04.050 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.050 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.050 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.050 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-06T20:37:04.050 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.050 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.050 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.051 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.051 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.051 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2023-07-06T20:37:04.051 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.051 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.051 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.052 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2023-07-06T20:37:04.052 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2023-07-06T20:37:04.052 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:04.052 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.052 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.052 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2023-07-06T20:37:04.052 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.052 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.053 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.053 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.053 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.053 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.053 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.053 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2023-07-06T20:37:04.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2023-07-06T20:37:04.054 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.054 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.054 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.054 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.054 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.054 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.054 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.055 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.055 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.055 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.055 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.055 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.055 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:04.055 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:04.055 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.056 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.056 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.056 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:04.056 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.056 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.056 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.056 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.057 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.057 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.057 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.057 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.057 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.057 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.057 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.057 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.058 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.058 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.058 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.058 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.058 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.058 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.058 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.058 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.059 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.059 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.059 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.059 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.059 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.059 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.059 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.059 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.060 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.060 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.060 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.060 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.060 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.060 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.060 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.060 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.061 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.061 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.061 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.061 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.061 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.061 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.061 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.061 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.062 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.062 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.062 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.062 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.062 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.062 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.062 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.062 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.063 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.063 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.063 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.063 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.063 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.063 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.063 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.063 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.064 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.064 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.064 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.064 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.064 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.064 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.065 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.065 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.065 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.065 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.065 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.065 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.065 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.066 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.066 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.066 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.066 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.066 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.066 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2023-07-06T20:37:04.067 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2023-07-06T20:37:04.067 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.067 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.067 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.067 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.067 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.067 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.067 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2023-07-06T20:37:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2023-07-06T20:37:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2023-07-06T20:37:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.070 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.070 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.070 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.070 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.070 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.071 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2023-07-06T20:37:04.071 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2023-07-06T20:37:04.071 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.071 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.071 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.071 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-06T20:37:04.072 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.072 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.072 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.072 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.072 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.072 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.073 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.073 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.073 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.073 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.073 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2023-07-06T20:37:04.074 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.074 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.074 INFO:teuthology.orchestra.run.smithi069.stdout: "gid": 24116, 2023-07-06T20:37:04.074 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "y", 2023-07-06T20:37:04.074 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr_features": 4540138322906710015, 2023-07-06T20:37:04.074 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2023-07-06T20:37:04.075 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.075 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2023-07-06T20:37:04.075 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.075 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.075 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.075 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:04.075 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:04.075 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.076 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.076 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.076 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:04.076 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.076 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.076 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.076 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2023-07-06T20:37:04.077 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.077 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.077 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.077 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.077 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.077 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.077 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.077 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.078 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.078 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.078 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.078 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.078 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.078 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.078 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.078 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.079 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.079 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.079 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.079 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.079 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.079 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.079 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.080 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.081 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.081 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.081 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.082 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.082 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.082 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.082 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.082 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.082 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.082 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.083 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.083 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.083 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.083 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.083 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.083 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.083 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.084 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.084 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.084 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.084 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.084 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.084 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.084 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.084 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.085 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.085 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.085 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.085 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.085 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.085 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.085 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.085 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.086 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.086 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.086 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.086 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.086 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.086 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.086 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.086 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.087 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.087 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.087 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2023-07-06T20:37:04.087 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2023-07-06T20:37:04.087 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.087 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.087 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.087 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.088 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.088 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.088 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.088 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2023-07-06T20:37:04.088 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.088 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.088 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.088 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.089 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2023-07-06T20:37:04.089 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2023-07-06T20:37:04.089 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.089 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.089 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.089 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2023-07-06T20:37:04.089 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.090 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.090 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.090 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2023-07-06T20:37:04.090 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.090 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.090 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.091 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.091 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2023-07-06T20:37:04.091 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2023-07-06T20:37:04.091 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.091 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.091 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.091 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.091 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.092 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.092 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.092 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2023-07-06T20:37:04.092 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.092 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.092 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.093 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.093 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2023-07-06T20:37:04.093 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2023-07-06T20:37:04.093 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.093 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.093 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.093 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.093 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.094 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.094 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.094 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2023-07-06T20:37:04.094 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.094 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.094 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.094 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.094 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2023-07-06T20:37:04.095 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2023-07-06T20:37:04.095 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.095 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.095 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.095 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2023-07-06T20:37:04.095 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.095 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.095 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.096 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2023-07-06T20:37:04.096 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.096 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.096 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.096 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.096 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2023-07-06T20:37:04.096 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2023-07-06T20:37:04.097 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.097 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.097 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.097 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.097 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.097 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.098 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.098 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2023-07-06T20:37:04.098 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.098 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.098 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.098 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.099 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2023-07-06T20:37:04.099 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2023-07-06T20:37:04.099 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.099 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.099 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.099 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.099 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.100 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.100 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.100 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-06T20:37:04.100 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.100 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.100 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.101 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.101 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2023-07-06T20:37:04.101 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2023-07-06T20:37:04.101 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.101 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.102 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.102 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.102 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.102 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.102 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2023-07-06T20:37:04.102 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.103 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.103 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.103 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.103 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.103 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.103 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.103 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2023-07-06T20:37:04.103 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.104 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.104 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.104 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2023-07-06T20:37:04.104 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2023-07-06T20:37:04.104 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.104 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.104 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.104 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.105 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.105 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.105 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.105 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2023-07-06T20:37:04.105 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.105 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.105 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.105 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.106 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2023-07-06T20:37:04.106 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2023-07-06T20:37:04.106 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.106 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.106 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.106 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2023-07-06T20:37:04.106 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.107 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.107 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2023-07-06T20:37:04.107 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-06T20:37:04.107 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.107 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.107 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.107 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2023-07-06T20:37:04.108 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2023-07-06T20:37:04.108 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:04.108 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.108 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.108 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:04.108 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2023-07-06T20:37:04.108 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2023-07-06T20:37:04.109 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.109 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-06T20:37:04.109 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-06T20:37:04.109 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.109 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.109 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.109 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2023-07-06T20:37:04.109 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2023-07-06T20:37:04.110 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:04.110 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.110 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.110 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2023-07-06T20:37:04.110 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2023-07-06T20:37:04.110 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2023-07-06T20:37:04.110 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.110 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-06T20:37:04.111 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.111 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.111 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.111 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.111 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2023-07-06T20:37:04.111 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2023-07-06T20:37:04.111 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.111 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.112 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.112 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2023-07-06T20:37:04.112 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.112 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.112 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.112 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-06T20:37:04.112 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-06T20:37:04.113 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.113 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.113 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.113 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2023-07-06T20:37:04.113 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2023-07-06T20:37:04.113 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:04.113 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.113 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.114 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2023-07-06T20:37:04.114 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2023-07-06T20:37:04.114 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2023-07-06T20:37:04.114 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.114 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2023-07-06T20:37:04.114 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-06T20:37:04.114 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.114 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.115 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.115 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2023-07-06T20:37:04.115 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2023-07-06T20:37:04.115 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.115 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.115 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.115 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2023-07-06T20:37:04.115 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.116 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.116 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.116 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2023-07-06T20:37:04.116 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-06T20:37:04.116 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.116 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.116 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.117 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2023-07-06T20:37:04.117 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2023-07-06T20:37:04.117 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:04.117 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.117 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.117 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:04.117 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2023-07-06T20:37:04.117 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2023-07-06T20:37:04.118 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.118 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-06T20:37:04.118 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-06T20:37:04.118 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.118 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.118 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.118 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.118 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.119 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.119 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.119 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.119 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.119 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.119 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.119 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.120 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.120 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.120 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.120 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.120 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.120 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.120 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.121 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.121 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.121 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.121 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.121 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.121 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.122 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.122 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.122 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.122 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.122 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.122 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.122 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.123 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.123 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.123 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.123 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.123 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.123 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.124 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.124 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.124 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.124 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.124 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.124 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.124 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.125 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.125 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.125 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.125 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.125 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.125 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.126 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.126 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.126 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.126 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.126 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.126 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.127 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.127 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.127 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.127 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.127 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.127 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.128 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.128 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.128 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.128 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.128 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.128 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.129 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.129 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.129 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.129 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2023-07-06T20:37:04.129 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2023-07-06T20:37:04.129 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:04.129 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.129 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.130 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:04.130 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.130 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.130 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.130 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-06T20:37:04.130 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.130 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.130 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.131 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.131 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2023-07-06T20:37:04.131 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2023-07-06T20:37:04.131 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.131 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.131 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.131 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2023-07-06T20:37:04.131 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.132 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.132 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.132 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2023-07-06T20:37:04.132 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2023-07-06T20:37:04.132 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap" 2023-07-06T20:37:04.132 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.132 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2023-07-06T20:37:04.133 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.133 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.133 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.133 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.133 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2023-07-06T20:37:04.133 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2023-07-06T20:37:04.133 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.134 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.134 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.134 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.134 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.134 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.134 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.134 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-06T20:37:04.134 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.135 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.135 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.135 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.135 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:04.135 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:04.135 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.135 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.135 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.136 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:04.136 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.136 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.136 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.136 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-06T20:37:04.136 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.136 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.137 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.137 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.137 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2023-07-06T20:37:04.137 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2023-07-06T20:37:04.137 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.137 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.137 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.137 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2023-07-06T20:37:04.138 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2023-07-06T20:37:04.138 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.138 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.138 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-06T20:37:04.138 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-06T20:37:04.138 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.138 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.138 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.139 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2023-07-06T20:37:04.139 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2023-07-06T20:37:04.139 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:04.139 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.139 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.139 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:04.139 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.139 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.140 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.140 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-06T20:37:04.140 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.140 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.140 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.140 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.140 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.140 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.141 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.141 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2023-07-06T20:37:04.141 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.141 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.141 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.141 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2023-07-06T20:37:04.141 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2023-07-06T20:37:04.141 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:04.142 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.142 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.142 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2023-07-06T20:37:04.142 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.142 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.142 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.142 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-06T20:37:04.142 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.143 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.143 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.143 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.143 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2023-07-06T20:37:04.143 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2023-07-06T20:37:04.143 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.143 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.143 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.144 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2023-07-06T20:37:04.144 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.144 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.144 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.144 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-06T20:37:04.144 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.145 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.145 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.145 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.145 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2023-07-06T20:37:04.145 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2023-07-06T20:37:04.145 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.145 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.146 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.146 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2023-07-06T20:37:04.146 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.146 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.146 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.146 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-06T20:37:04.146 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.147 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.147 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.147 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.147 INFO:teuthology.orchestra.run.smithi069.stdout: "alertmanager_web_password": { 2023-07-06T20:37:04.147 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alertmanager_web_password", 2023-07-06T20:37:04.147 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.148 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.148 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.148 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:04.148 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.148 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.148 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.148 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Alertmanager web password", 2023-07-06T20:37:04.149 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.149 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.149 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.149 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.149 INFO:teuthology.orchestra.run.smithi069.stdout: "alertmanager_web_user": { 2023-07-06T20:37:04.149 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alertmanager_web_user", 2023-07-06T20:37:04.149 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.150 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.150 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.150 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:04.150 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.150 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.150 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.150 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Alertmanager web user", 2023-07-06T20:37:04.151 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.151 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.151 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.151 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.151 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2023-07-06T20:37:04.151 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2023-07-06T20:37:04.151 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-06T20:37:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-06T20:37:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2023-07-06T20:37:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2023-07-06T20:37:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-06T20:37:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2023-07-06T20:37:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2023-07-06T20:37:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2023-07-06T20:37:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-06T20:37:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2023-07-06T20:37:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2023-07-06T20:37:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-06T20:37:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2023-07-06T20:37:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2023-07-06T20:37:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-06T20:37:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2023-07-06T20:37:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2023-07-06T20:37:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-06T20:37:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2023-07-06T20:37:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2023-07-06T20:37:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.164 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.164 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-06T20:37:04.164 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.164 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.164 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.164 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2023-07-06T20:37:04.164 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.165 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.165 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.165 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.165 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2023-07-06T20:37:04.165 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2023-07-06T20:37:04.165 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.165 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.165 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.166 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-06T20:37:04.166 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.166 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.166 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.166 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2023-07-06T20:37:04.166 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.166 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.166 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.167 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.167 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2023-07-06T20:37:04.167 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2023-07-06T20:37:04.167 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.167 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.167 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.167 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-06T20:37:04.167 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.168 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.168 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.168 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2023-07-06T20:37:04.168 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.168 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.168 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.168 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.169 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2023-07-06T20:37:04.169 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2023-07-06T20:37:04.169 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.169 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.169 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.169 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-06T20:37:04.169 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.169 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.170 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.170 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2023-07-06T20:37:04.170 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.170 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.170 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.170 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.170 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2023-07-06T20:37:04.171 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2023-07-06T20:37:04.171 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.171 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.171 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.171 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-06T20:37:04.171 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.171 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.171 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.172 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2023-07-06T20:37:04.172 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.172 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.172 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.172 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.172 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2023-07-06T20:37:04.172 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2023-07-06T20:37:04.173 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.173 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.173 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.173 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-06T20:37:04.173 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.173 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.174 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.174 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2023-07-06T20:37:04.174 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.174 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.174 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.174 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.174 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2023-07-06T20:37:04.174 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2023-07-06T20:37:04.175 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.175 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.175 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.175 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-06T20:37:04.175 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.175 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.175 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.176 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2023-07-06T20:37:04.176 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.176 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.176 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.176 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.176 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2023-07-06T20:37:04.176 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2023-07-06T20:37:04.177 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.177 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.177 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.177 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-06T20:37:04.177 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.177 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.177 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.178 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2023-07-06T20:37:04.178 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.178 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.178 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.178 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.178 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2023-07-06T20:37:04.179 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2023-07-06T20:37:04.179 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.179 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.179 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.179 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-06T20:37:04.179 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.180 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.180 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.180 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2023-07-06T20:37:04.180 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.180 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.181 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.181 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.181 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2023-07-06T20:37:04.181 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2023-07-06T20:37:04.181 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.181 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.182 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.182 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-06T20:37:04.182 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.182 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.182 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.182 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2023-07-06T20:37:04.183 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.183 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.183 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.183 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.183 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2023-07-06T20:37:04.183 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2023-07-06T20:37:04.183 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.184 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.184 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.184 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-06T20:37:04.184 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.184 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.184 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.184 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2023-07-06T20:37:04.184 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.185 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.185 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.185 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.185 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2023-07-06T20:37:04.185 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2023-07-06T20:37:04.185 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.185 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.185 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.186 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-06T20:37:04.186 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.186 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.186 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.186 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2023-07-06T20:37:04.186 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.186 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.186 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.187 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.187 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2023-07-06T20:37:04.187 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2023-07-06T20:37:04.187 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.187 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.187 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.187 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-06T20:37:04.187 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.188 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.188 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.188 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2023-07-06T20:37:04.188 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.188 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.188 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.188 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.189 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2023-07-06T20:37:04.189 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2023-07-06T20:37:04.189 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.189 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.189 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.189 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-06T20:37:04.189 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.189 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.190 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.190 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2023-07-06T20:37:04.190 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.190 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.190 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.190 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.190 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2023-07-06T20:37:04.190 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2023-07-06T20:37:04.191 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.191 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.191 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.191 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.191 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.191 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.191 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.191 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2023-07-06T20:37:04.192 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.192 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.192 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.192 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.192 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2023-07-06T20:37:04.192 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2023-07-06T20:37:04.192 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.192 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.193 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.193 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:04.193 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.193 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.193 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.193 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-06T20:37:04.193 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.194 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.194 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.194 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.194 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2023-07-06T20:37:04.194 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2023-07-06T20:37:04.194 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.194 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.194 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.195 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2023-07-06T20:37:04.195 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.195 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.195 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.195 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-06T20:37:04.195 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.195 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.195 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.196 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.196 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2023-07-06T20:37:04.196 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2023-07-06T20:37:04.196 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.196 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.196 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.196 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2023-07-06T20:37:04.196 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.197 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.197 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.197 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-06T20:37:04.197 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.197 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.197 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.197 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.197 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2023-07-06T20:37:04.198 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2023-07-06T20:37:04.198 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.198 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.198 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.198 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2023-07-06T20:37:04.198 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.198 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.198 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.199 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2023-07-06T20:37:04.199 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.199 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.199 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.199 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.199 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2023-07-06T20:37:04.200 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2023-07-06T20:37:04.200 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.200 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.200 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.200 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.200 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.200 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.200 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.201 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-06T20:37:04.201 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.201 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.201 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.201 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.201 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2023-07-06T20:37:04.202 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2023-07-06T20:37:04.202 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.202 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.202 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.202 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:04.202 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.203 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.203 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.203 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2023-07-06T20:37:04.203 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.203 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.203 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.203 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.204 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2023-07-06T20:37:04.204 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2023-07-06T20:37:04.204 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.204 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.204 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.204 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:04.205 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.205 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.205 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.205 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2023-07-06T20:37:04.205 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.205 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.205 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.206 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.206 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.206 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.206 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.206 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.206 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.206 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.207 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.207 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.207 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.207 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.207 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.207 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.207 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.207 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.208 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.208 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.208 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.208 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.208 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.208 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.208 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.208 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2023-07-06T20:37:04.209 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2023-07-06T20:37:04.209 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.209 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.209 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.209 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.209 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.209 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.209 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.210 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-06T20:37:04.210 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.210 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.210 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.210 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.210 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.210 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.211 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.211 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.211 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.211 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.211 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.211 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.212 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.212 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-06T20:37:04.212 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.212 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.212 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.213 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.213 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.213 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.213 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.213 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.213 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.213 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.214 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.214 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.214 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.214 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.214 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.214 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.214 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.215 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.215 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.215 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.215 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.215 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.215 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.215 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.215 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.216 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.216 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.216 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.216 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.216 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.216 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.216 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.216 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.217 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.217 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.217 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.217 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.217 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.217 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.217 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-06T20:37:04.217 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-06T20:37:04.218 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.218 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.218 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.218 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.218 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.218 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.218 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.218 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-06T20:37:04.219 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.219 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.219 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.219 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.219 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-06T20:37:04.219 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-06T20:37:04.219 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.220 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.220 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.220 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2023-07-06T20:37:04.220 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.220 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.220 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.220 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-06T20:37:04.220 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.221 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.221 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.221 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2023-07-06T20:37:04.221 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2023-07-06T20:37:04.221 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.221 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.222 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.222 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:04.222 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.222 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.222 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.222 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2023-07-06T20:37:04.222 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.222 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.223 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.223 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.223 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2023-07-06T20:37:04.223 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2023-07-06T20:37:04.223 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.223 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.223 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.223 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:04.224 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.224 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.224 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.224 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-06T20:37:04.224 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.224 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.224 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.224 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.225 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2023-07-06T20:37:04.225 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2023-07-06T20:37:04.225 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.225 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.225 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.225 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.225 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.225 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.226 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.226 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2023-07-06T20:37:04.226 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.226 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.226 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.226 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2023-07-06T20:37:04.227 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2023-07-06T20:37:04.227 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.227 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.227 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.227 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2023-07-06T20:37:04.227 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.227 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.227 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.228 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2023-07-06T20:37:04.228 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2023-07-06T20:37:04.228 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.228 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2023-07-06T20:37:04.228 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.228 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.228 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.229 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.229 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2023-07-06T20:37:04.229 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2023-07-06T20:37:04.229 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.229 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.229 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.230 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-06T20:37:04.230 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.230 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.230 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.230 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-06T20:37:04.230 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.230 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.230 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.231 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.231 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_web_password": { 2023-07-06T20:37:04.231 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_web_password", 2023-07-06T20:37:04.231 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.231 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.231 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.232 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:04.232 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.232 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.232 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.232 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus web password", 2023-07-06T20:37:04.232 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.232 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.233 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.233 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.233 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_web_user": { 2023-07-06T20:37:04.233 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_web_user", 2023-07-06T20:37:04.233 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.233 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.233 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.234 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:04.234 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.234 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.234 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.234 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus web user", 2023-07-06T20:37:04.234 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.234 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.235 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.235 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.235 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2023-07-06T20:37:04.235 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2023-07-06T20:37:04.235 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.235 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.235 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.236 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.236 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.236 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.236 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.236 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-06T20:37:04.237 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.237 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.237 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.237 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.237 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2023-07-06T20:37:04.237 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2023-07-06T20:37:04.238 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.238 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.238 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.238 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.238 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.238 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.239 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.239 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-06T20:37:04.239 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.239 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.239 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.239 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.239 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2023-07-06T20:37:04.240 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2023-07-06T20:37:04.240 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.240 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.240 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.240 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.240 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.240 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.241 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.241 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-06T20:37:04.241 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.241 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.241 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.241 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.241 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2023-07-06T20:37:04.241 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2023-07-06T20:37:04.242 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.242 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.242 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.242 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.242 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.242 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.242 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.242 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-06T20:37:04.243 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.243 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.243 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.243 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.243 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2023-07-06T20:37:04.243 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2023-07-06T20:37:04.243 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.243 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.244 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.244 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.244 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.244 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.244 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.244 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-06T20:37:04.244 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.244 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.245 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.245 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.245 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2023-07-06T20:37:04.245 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2023-07-06T20:37:04.245 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.245 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.245 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.245 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2023-07-06T20:37:04.246 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.246 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.246 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.246 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2023-07-06T20:37:04.246 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.246 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.246 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.246 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.247 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2023-07-06T20:37:04.247 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2023-07-06T20:37:04.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.247 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.247 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.247 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.247 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.248 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.248 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.248 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-06T20:37:04.248 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.248 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.248 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.248 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.248 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2023-07-06T20:37:04.249 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2023-07-06T20:37:04.249 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.249 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.249 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.249 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.249 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.249 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.249 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.250 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-06T20:37:04.250 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.250 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.250 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.250 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.250 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2023-07-06T20:37:04.250 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2023-07-06T20:37:04.251 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.251 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.251 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.251 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.251 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.251 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.251 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.251 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-06T20:37:04.252 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.252 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.252 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.252 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.252 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2023-07-06T20:37:04.252 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2023-07-06T20:37:04.252 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.253 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.253 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.253 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.253 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.253 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.253 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.253 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2023-07-06T20:37:04.253 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.254 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.254 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.254 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.254 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2023-07-06T20:37:04.254 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2023-07-06T20:37:04.254 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.254 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.254 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.255 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.255 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.255 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.255 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.255 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-06T20:37:04.255 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.256 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.256 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.256 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.256 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2023-07-06T20:37:04.256 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2023-07-06T20:37:04.256 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.256 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.256 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.257 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.257 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.257 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.257 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.257 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-06T20:37:04.257 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.258 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.258 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.258 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.258 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.258 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.258 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.258 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2023-07-06T20:37:04.258 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.259 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.259 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.259 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.259 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.259 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.259 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.260 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.260 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.260 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.260 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.260 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.260 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.260 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.261 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.261 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.261 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.261 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.261 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.261 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.262 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.262 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.262 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.262 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.263 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.263 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.263 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.263 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.263 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.263 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.264 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.264 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.264 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.264 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.264 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.264 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.264 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.265 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.265 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.265 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.265 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.265 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.265 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.265 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.266 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.266 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.266 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.266 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.266 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.266 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.266 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.267 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.267 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.267 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.267 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.267 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.267 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.267 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.268 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.268 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.268 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.268 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.268 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.269 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.269 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.269 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.269 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.269 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.269 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.269 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.270 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.270 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2023-07-06T20:37:04.270 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2023-07-06T20:37:04.270 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.270 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.270 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.270 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2023-07-06T20:37:04.270 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.271 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.271 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.271 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-06T20:37:04.271 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.271 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.271 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.271 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.271 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2023-07-06T20:37:04.272 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2023-07-06T20:37:04.272 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.272 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.272 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.272 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2023-07-06T20:37:04.272 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.272 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.273 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.273 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-06T20:37:04.273 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.273 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.273 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.273 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.273 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.273 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.274 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.274 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2023-07-06T20:37:04.274 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.274 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.274 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.274 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-06T20:37:04.274 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-06T20:37:04.274 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.275 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.275 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.275 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:04.275 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.275 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.275 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.275 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.275 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.276 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.276 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.276 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.276 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2023-07-06T20:37:04.276 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-06T20:37:04.276 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.276 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.276 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.277 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.277 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.277 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.277 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.277 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.277 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.277 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.278 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.278 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.278 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-06T20:37:04.278 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-06T20:37:04.278 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.278 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.278 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.278 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.279 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.279 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.279 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.279 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.279 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.279 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.279 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.280 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2023-07-06T20:37:04.280 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2023-07-06T20:37:04.280 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.280 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.280 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.280 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.280 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.280 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.281 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.281 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.281 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.281 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.281 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.281 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-06T20:37:04.281 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-06T20:37:04.282 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.282 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.282 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.282 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.282 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.282 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.282 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.282 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.283 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.283 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.283 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.283 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.283 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2023-07-06T20:37:04.283 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-06T20:37:04.283 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.284 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.284 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.284 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.284 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.284 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.284 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.284 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.284 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.285 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.285 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.285 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.285 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-06T20:37:04.285 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-06T20:37:04.285 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.286 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.286 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.286 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.286 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.286 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.286 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.286 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.286 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.287 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.287 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.287 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.287 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-06T20:37:04.287 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-06T20:37:04.287 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.287 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.288 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.288 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.288 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.288 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.288 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.288 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.288 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.289 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.289 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.289 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.289 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-06T20:37:04.289 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-06T20:37:04.289 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.289 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.290 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.290 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.290 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.290 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.290 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.290 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.290 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.291 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.291 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.291 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.291 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-06T20:37:04.291 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-06T20:37:04.291 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.292 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.292 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.292 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.292 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.292 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.292 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.293 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.293 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.293 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.293 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.293 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.293 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-06T20:37:04.294 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-06T20:37:04.294 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.294 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.294 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.294 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.294 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.295 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.295 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.295 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.295 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.295 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.295 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.295 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.296 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-06T20:37:04.296 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-06T20:37:04.296 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.296 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.296 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.296 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.297 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.297 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.297 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.297 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.297 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.297 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.297 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.297 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-06T20:37:04.298 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-06T20:37:04.298 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.298 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.298 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.298 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.298 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.298 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.298 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.299 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.299 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.299 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.299 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.299 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.299 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-06T20:37:04.299 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-06T20:37:04.300 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.300 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.300 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.300 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.300 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.300 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.300 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.300 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.301 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.301 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.301 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2023-07-06T20:37:04.301 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-06T20:37:04.301 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.301 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.302 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.302 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:04.302 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.302 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.302 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.302 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.302 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.303 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.303 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.303 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.303 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-06T20:37:04.303 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-06T20:37:04.303 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.303 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.303 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.304 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.304 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.304 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.304 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.304 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.304 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.304 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.304 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.305 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.305 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2023-07-06T20:37:04.305 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2023-07-06T20:37:04.305 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.305 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.305 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.305 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.306 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.306 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.306 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.306 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.306 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.306 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.306 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.306 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.307 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2023-07-06T20:37:04.307 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-06T20:37:04.307 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.307 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.307 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.307 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:04.307 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.307 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.308 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.308 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.308 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.308 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.308 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.308 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.308 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-06T20:37:04.308 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-06T20:37:04.309 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.309 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.309 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.309 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.309 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.309 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.309 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.309 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.310 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.310 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.310 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.310 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.310 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-06T20:37:04.310 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-06T20:37:04.310 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.310 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.311 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.311 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.311 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.311 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.311 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.311 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.312 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.312 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.312 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.312 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-06T20:37:04.312 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-06T20:37:04.312 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.312 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.313 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.313 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.313 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.313 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.313 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.313 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.314 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.314 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.314 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.314 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.314 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-06T20:37:04.314 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-06T20:37:04.314 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.314 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.315 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.315 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.315 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.315 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.315 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.315 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.316 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.316 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.316 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.316 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.316 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2023-07-06T20:37:04.316 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-06T20:37:04.317 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.317 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.317 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.317 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.317 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.317 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.317 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.318 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.318 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.318 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.318 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.318 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.318 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-06T20:37:04.319 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-06T20:37:04.319 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.319 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.319 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.319 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.319 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.320 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.320 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.320 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.320 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.320 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.320 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.320 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.321 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-06T20:37:04.321 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-06T20:37:04.321 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.321 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.321 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.321 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.321 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.321 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.322 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.322 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.322 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.322 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.322 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.322 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-06T20:37:04.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-06T20:37:04.323 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.323 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.323 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.323 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.323 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.323 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.323 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.324 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.324 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.324 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.324 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.324 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-06T20:37:04.324 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-06T20:37:04.325 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.325 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.325 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.325 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.325 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.325 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.325 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.326 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.326 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.326 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.326 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.326 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.326 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-06T20:37:04.326 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-06T20:37:04.326 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.327 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.327 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.327 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.327 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.327 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.327 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.327 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.327 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.328 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.328 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.328 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.328 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-06T20:37:04.328 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-06T20:37:04.328 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.328 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.329 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.329 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.329 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.329 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.329 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.329 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.329 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.329 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.330 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.330 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.330 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-06T20:37:04.330 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-06T20:37:04.330 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.330 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.330 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.330 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.331 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.331 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.331 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.331 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.331 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.331 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.331 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.331 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.332 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-06T20:37:04.332 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-06T20:37:04.332 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.332 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.332 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.332 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.332 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.333 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.333 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.333 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.333 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.333 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.333 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.333 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.333 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2023-07-06T20:37:04.334 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-06T20:37:04.334 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.334 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.334 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.334 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.334 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.334 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.334 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.335 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.335 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.335 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.335 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.335 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.335 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-06T20:37:04.335 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-06T20:37:04.335 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.336 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.336 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.336 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-06T20:37:04.336 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.336 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.336 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.336 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.336 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.337 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.337 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.337 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-06T20:37:04.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-06T20:37:04.337 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.337 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.337 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.338 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:04.338 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.338 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.338 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.338 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.338 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.338 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.339 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.339 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-06T20:37:04.339 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-06T20:37:04.339 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.339 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.339 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.339 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2023-07-06T20:37:04.340 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.340 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.340 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.340 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.340 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.340 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.340 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.340 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.341 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-06T20:37:04.341 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-06T20:37:04.341 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.341 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.341 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.341 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2023-07-06T20:37:04.342 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.342 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.342 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.342 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.342 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.342 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.342 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.342 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.343 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2023-07-06T20:37:04.343 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-06T20:37:04.343 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.343 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.343 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.343 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.344 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.344 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.344 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.344 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.344 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.344 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.344 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.345 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.345 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-06T20:37:04.345 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-06T20:37:04.345 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.345 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.345 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.345 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:04.346 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.346 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.346 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.346 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.346 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.346 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.347 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.347 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.347 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2023-07-06T20:37:04.347 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-06T20:37:04.347 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.347 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.348 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.348 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.348 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.348 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.348 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.348 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.349 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.349 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.349 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.349 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.349 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2023-07-06T20:37:04.350 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-06T20:37:04.350 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.350 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.350 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.350 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.350 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.351 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.351 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.351 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.351 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.351 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.351 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.351 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.351 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-06T20:37:04.352 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-06T20:37:04.352 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.352 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.352 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.352 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:04.352 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.352 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.353 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.353 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.353 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.353 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.353 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.353 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.353 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-06T20:37:04.353 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-06T20:37:04.354 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.354 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.354 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.354 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:04.354 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.354 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.354 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.354 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.355 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.355 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.355 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.355 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.355 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-06T20:37:04.355 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-06T20:37:04.355 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.356 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.356 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.356 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2023-07-06T20:37:04.356 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.356 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.356 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.356 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.356 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.357 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.357 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.357 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.357 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2023-07-06T20:37:04.357 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2023-07-06T20:37:04.357 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.357 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.358 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.358 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.358 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.358 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.358 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.358 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.358 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.358 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.359 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.359 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.359 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2023-07-06T20:37:04.359 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2023-07-06T20:37:04.359 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.359 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.359 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.360 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.360 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.360 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.360 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.360 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.360 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.360 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.360 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.361 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.361 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2023-07-06T20:37:04.361 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2023-07-06T20:37:04.361 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.361 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.361 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.361 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.361 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2023-07-06T20:37:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.363 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2023-07-06T20:37:04.363 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2023-07-06T20:37:04.363 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.363 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.363 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.363 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2023-07-06T20:37:04.363 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.363 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.364 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.364 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.364 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.364 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.364 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.364 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.364 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2023-07-06T20:37:04.365 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2023-07-06T20:37:04.365 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.365 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.365 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.365 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.365 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.365 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.365 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.366 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.366 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.366 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.366 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.366 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.366 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.366 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.367 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.367 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.367 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.367 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.367 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.367 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.367 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.368 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.368 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.368 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.368 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.368 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.368 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.368 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.369 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.369 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.369 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.369 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.369 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.369 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.369 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.370 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.370 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.370 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.370 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.370 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.370 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.370 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.371 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.371 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.371 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.371 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.371 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.371 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.371 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.372 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.372 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.372 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.372 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.372 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.372 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.373 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.373 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.373 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.373 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.373 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.373 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.374 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.374 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.374 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.374 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.374 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.374 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.375 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.375 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.375 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.375 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.375 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.375 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.376 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.376 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.376 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.376 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.376 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.376 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.376 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.377 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.377 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.377 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2023-07-06T20:37:04.377 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2023-07-06T20:37:04.377 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.377 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.377 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.377 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.378 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.378 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.378 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.378 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2023-07-06T20:37:04.378 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.378 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.378 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.378 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.379 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2023-07-06T20:37:04.379 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2023-07-06T20:37:04.379 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.379 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.379 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.379 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.380 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.380 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.380 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.380 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.380 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.380 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.380 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.381 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2023-07-06T20:37:04.381 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2023-07-06T20:37:04.381 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.381 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.381 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.381 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2023-07-06T20:37:04.381 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.381 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.382 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.382 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.382 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.382 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.382 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.382 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.382 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2023-07-06T20:37:04.383 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2023-07-06T20:37:04.383 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.383 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.383 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.383 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2023-07-06T20:37:04.383 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.383 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.384 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.384 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.384 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.384 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.384 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.384 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.384 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2023-07-06T20:37:04.385 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2023-07-06T20:37:04.385 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.385 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.385 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.385 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.385 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.386 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.386 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.386 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.386 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.386 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.387 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.387 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.387 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2023-07-06T20:37:04.387 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2023-07-06T20:37:04.387 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.387 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.388 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.388 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2023-07-06T20:37:04.388 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.388 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.388 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.388 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.388 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.389 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.389 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.389 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.389 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2023-07-06T20:37:04.389 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2023-07-06T20:37:04.389 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.389 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.389 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.390 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2023-07-06T20:37:04.390 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.390 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.390 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.390 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.390 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2023-07-06T20:37:04.390 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.390 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.391 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.391 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.391 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.391 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.391 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2023-07-06T20:37:04.391 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2023-07-06T20:37:04.391 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.391 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.392 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.392 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2023-07-06T20:37:04.392 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2023-07-06T20:37:04.392 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2023-07-06T20:37:04.392 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.392 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.393 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.393 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.393 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.393 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.393 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2023-07-06T20:37:04.393 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2023-07-06T20:37:04.393 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.393 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.394 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.394 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.394 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.394 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.394 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.394 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.394 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.394 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.395 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.395 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.395 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.395 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.395 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.395 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2023-07-06T20:37:04.395 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.396 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.396 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.396 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2023-07-06T20:37:04.396 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2023-07-06T20:37:04.396 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.396 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.396 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.396 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.397 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.397 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.397 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.397 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2023-07-06T20:37:04.397 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.397 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.397 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.397 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.398 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.398 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.398 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.398 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.398 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.398 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.398 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.399 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.399 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.399 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.399 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.399 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.399 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.399 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.399 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.400 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.400 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.400 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.400 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.400 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.400 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.400 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.401 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.401 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.401 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.401 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.401 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.401 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.402 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.402 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.402 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.402 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.402 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.402 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.402 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.403 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.403 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.403 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.403 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.403 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.403 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.404 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.404 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.404 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.404 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.404 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.404 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.405 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.405 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.405 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.405 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.405 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.406 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.406 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.406 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.406 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.406 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.406 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.407 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.407 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.407 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.407 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.407 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.407 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.407 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.407 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.408 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.408 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.408 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.408 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.408 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2023-07-06T20:37:04.408 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2023-07-06T20:37:04.408 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.408 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.409 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.409 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2023-07-06T20:37:04.409 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.409 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.409 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.409 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-06T20:37:04.409 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.409 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.410 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.410 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.410 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2023-07-06T20:37:04.410 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2023-07-06T20:37:04.410 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.410 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.410 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.411 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2023-07-06T20:37:04.411 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.411 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.411 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.411 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-06T20:37:04.411 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.411 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.411 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.412 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.412 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2023-07-06T20:37:04.412 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2023-07-06T20:37:04.412 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.412 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.412 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2023-07-06T20:37:04.412 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.413 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.413 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.413 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2023-07-06T20:37:04.413 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.413 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.413 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.413 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.413 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2023-07-06T20:37:04.414 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2023-07-06T20:37:04.414 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.414 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.414 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.414 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2023-07-06T20:37:04.414 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.414 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.414 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.415 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-06T20:37:04.415 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.415 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.415 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.415 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.415 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2023-07-06T20:37:04.415 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2023-07-06T20:37:04.415 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.416 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.416 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.416 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.416 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.416 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.416 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.416 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-06T20:37:04.416 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.417 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.417 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.417 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.417 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:04.417 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:04.417 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.417 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.417 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.418 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:04.418 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.418 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.418 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.418 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2023-07-06T20:37:04.418 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.418 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.419 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.419 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2023-07-06T20:37:04.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2023-07-06T20:37:04.419 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.419 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.419 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.419 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2023-07-06T20:37:04.420 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.420 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.420 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.420 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-06T20:37:04.420 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.420 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.420 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.420 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.421 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.421 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.421 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2023-07-06T20:37:04.421 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.421 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.421 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.421 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.422 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.422 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.422 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.422 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.422 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.423 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.423 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.423 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.423 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.423 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.423 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.423 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.423 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.424 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.424 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.424 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.424 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.424 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.425 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.425 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.425 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.425 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.425 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.425 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.425 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.425 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.426 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.426 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.426 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.426 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.426 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.426 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.427 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.427 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.427 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.427 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.428 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.428 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.428 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.428 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.428 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.429 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.429 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.429 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.429 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.429 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.429 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.430 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.430 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.430 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.430 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.430 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.431 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.431 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.431 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.431 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.432 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.432 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.432 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.432 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.432 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.432 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.432 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.433 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.433 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.433 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.433 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2023-07-06T20:37:04.433 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2023-07-06T20:37:04.433 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.434 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.434 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.434 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2023-07-06T20:37:04.434 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.434 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.434 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.434 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.434 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.435 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.435 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.435 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.435 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2023-07-06T20:37:04.435 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2023-07-06T20:37:04.435 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.435 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.435 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.436 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2023-07-06T20:37:04.436 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.436 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.436 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.436 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.436 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.437 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.437 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.437 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:04.437 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:04.437 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.437 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.437 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:04.438 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.438 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.438 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.438 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.438 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.438 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.438 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.438 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.439 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.439 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.439 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.439 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2023-07-06T20:37:04.439 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2023-07-06T20:37:04.439 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2023-07-06T20:37:04.439 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.439 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2023-07-06T20:37:04.440 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2023-07-06T20:37:04.440 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.440 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.440 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.440 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2023-07-06T20:37:04.440 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.440 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.441 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.441 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-06T20:37:04.441 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.441 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.441 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.441 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.441 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2023-07-06T20:37:04.441 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2023-07-06T20:37:04.442 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.442 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.442 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.442 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2023-07-06T20:37:04.442 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.442 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.442 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.442 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-06T20:37:04.443 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.443 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.443 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.443 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.443 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2023-07-06T20:37:04.443 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2023-07-06T20:37:04.443 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.443 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.444 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.444 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.444 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.444 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2023-07-06T20:37:04.444 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.444 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.445 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.445 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.445 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:04.445 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:04.445 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.445 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.445 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.446 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2023-07-06T20:37:04.446 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2023-07-06T20:37:04.446 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.446 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.446 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-06T20:37:04.446 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.446 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.446 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.447 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.447 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.447 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.447 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.447 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.447 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.447 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.447 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.448 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.448 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.448 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.448 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.448 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.449 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.449 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.449 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.449 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.449 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.449 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.449 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.449 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.450 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.450 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.450 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.450 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.450 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.450 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.451 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.451 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.451 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.451 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.451 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.451 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.451 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.451 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.452 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.452 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.452 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.452 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.452 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.452 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.452 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.452 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.453 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.453 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.453 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.453 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.453 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.453 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.453 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.454 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.454 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.454 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.454 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.454 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.454 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.454 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.454 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.455 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.455 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.455 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.455 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.455 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.455 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.455 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.455 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.456 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.456 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.456 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.456 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.456 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.456 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2023-07-06T20:37:04.456 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2023-07-06T20:37:04.457 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.457 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.457 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.457 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.457 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.457 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.457 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.458 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2023-07-06T20:37:04.458 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.458 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.458 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.459 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2023-07-06T20:37:04.459 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2023-07-06T20:37:04.459 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.459 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.459 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.460 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2023-07-06T20:37:04.460 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.460 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.460 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.460 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2023-07-06T20:37:04.460 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.461 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.461 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.461 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2023-07-06T20:37:04.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2023-07-06T20:37:04.462 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.462 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.462 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.462 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2023-07-06T20:37:04.462 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.462 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.463 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-06T20:37:04.463 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.463 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.463 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.463 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.463 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2023-07-06T20:37:04.463 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2023-07-06T20:37:04.464 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.464 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.464 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.464 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2023-07-06T20:37:04.464 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2023-07-06T20:37:04.464 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2023-07-06T20:37:04.464 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.465 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-06T20:37:04.465 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.465 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.465 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.465 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.465 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2023-07-06T20:37:04.465 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2023-07-06T20:37:04.465 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.466 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.466 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.466 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.466 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.466 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.466 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.466 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2023-07-06T20:37:04.466 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.467 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.467 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.467 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2023-07-06T20:37:04.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2023-07-06T20:37:04.467 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.467 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.467 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2023-07-06T20:37:04.468 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.468 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.468 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.468 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-06T20:37:04.468 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.468 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2023-07-06T20:37:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.469 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.470 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.470 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.470 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.470 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.470 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.470 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.470 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.470 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.471 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.471 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.471 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.471 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.471 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.471 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.471 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.471 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.472 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.472 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.472 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.472 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.472 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.472 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.472 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.472 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.473 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.473 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.473 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.473 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.473 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.473 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.473 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.473 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.474 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.474 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.474 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.474 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.474 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.474 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.474 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.475 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.475 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.475 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.475 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.475 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.475 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.475 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.475 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.476 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.476 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.476 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.476 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.476 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.476 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.477 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.477 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.477 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.477 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.477 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.477 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.477 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.477 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.478 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.478 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.478 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.478 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.478 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.478 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.479 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.479 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.479 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.479 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.479 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.479 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2023-07-06T20:37:04.479 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.480 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.480 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.480 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.480 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.480 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.480 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.481 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.481 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.481 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.481 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.481 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.481 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.481 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.481 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.482 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.482 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.482 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.482 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.482 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.482 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.483 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.483 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.483 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.483 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.483 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.484 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.484 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.484 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.484 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.484 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.484 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.485 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.485 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.485 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.485 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.486 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.486 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.486 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.486 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.486 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.487 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.487 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.487 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.487 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.487 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.487 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.488 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.488 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.488 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.488 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.488 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.488 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.489 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.489 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.489 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.489 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.489 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.489 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.489 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.489 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.490 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.490 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.490 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.490 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.490 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.490 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.490 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.490 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.491 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.491 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.491 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.491 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.491 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2023-07-06T20:37:04.491 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.491 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.491 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.492 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2023-07-06T20:37:04.492 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2023-07-06T20:37:04.492 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.492 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.492 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.492 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2023-07-06T20:37:04.492 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.492 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.493 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.493 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-06T20:37:04.493 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.493 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.493 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.493 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.493 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2023-07-06T20:37:04.494 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2023-07-06T20:37:04.494 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.494 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.494 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.494 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:04.494 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2023-07-06T20:37:04.494 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.494 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.495 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-06T20:37:04.495 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.495 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.495 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.495 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.495 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.495 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.495 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.496 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.496 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.496 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.496 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.496 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.496 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.496 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.497 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.497 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.497 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.497 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.497 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.497 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.497 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.497 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.498 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.498 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.498 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.498 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.498 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.498 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.498 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.498 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.499 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.499 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.499 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.499 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.499 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.499 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.499 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.499 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.500 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.500 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.500 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.500 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.500 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.500 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.500 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.500 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.501 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.501 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.501 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.501 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.501 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.501 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.501 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.501 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.502 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.502 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.502 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.502 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.502 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.502 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.502 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.502 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.503 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.503 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.503 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.503 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.503 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.503 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.503 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.504 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.504 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.504 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.504 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.504 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.505 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.505 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.505 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2023-07-06T20:37:04.505 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.505 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.505 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.505 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2023-07-06T20:37:04.505 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2023-07-06T20:37:04.506 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.506 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.506 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.506 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2023-07-06T20:37:04.506 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.506 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.507 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.507 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2023-07-06T20:37:04.507 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-06T20:37:04.507 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.507 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.507 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.507 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.508 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.508 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.508 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.508 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.508 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.508 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.509 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.509 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.509 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.509 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.509 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.509 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.509 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.510 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.510 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.510 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.510 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.510 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.510 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.510 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.510 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.511 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.511 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.511 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.511 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.511 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.512 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.512 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.512 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.512 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.512 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.512 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.512 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.512 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.513 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.513 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.513 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.513 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.513 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.513 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.514 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.514 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.514 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.514 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.514 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.514 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.514 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.515 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.515 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.515 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.515 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.515 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.516 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.516 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.516 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.516 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.516 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.517 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.517 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.517 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.517 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.517 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.517 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.518 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.518 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.518 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.518 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.518 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2023-07-06T20:37:04.518 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2023-07-06T20:37:04.518 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.519 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.519 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.519 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.519 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.519 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.519 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2023-07-06T20:37:04.519 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-06T20:37:04.520 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.520 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.520 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.520 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2023-07-06T20:37:04.520 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2023-07-06T20:37:04.520 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.520 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.520 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.521 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2023-07-06T20:37:04.521 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.521 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.521 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2023-07-06T20:37:04.521 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.521 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.522 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.522 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.522 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2023-07-06T20:37:04.522 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2023-07-06T20:37:04.522 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.522 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.522 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.522 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2023-07-06T20:37:04.523 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.523 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.523 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.523 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2023-07-06T20:37:04.523 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.523 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.523 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.524 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.524 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2023-07-06T20:37:04.524 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2023-07-06T20:37:04.524 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.524 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.524 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.524 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.524 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.525 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.525 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.525 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2023-07-06T20:37:04.525 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.525 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.525 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.525 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.526 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2023-07-06T20:37:04.526 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2023-07-06T20:37:04.526 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.526 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.526 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.526 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2023-07-06T20:37:04.526 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.526 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.527 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.527 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-06T20:37:04.527 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-06T20:37:04.527 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.527 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.527 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.527 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.527 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.528 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.528 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2023-07-06T20:37:04.528 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.528 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.528 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.528 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.528 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.529 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.529 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.529 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.529 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.529 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.529 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.529 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.529 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.530 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.530 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.530 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.530 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.530 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.530 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.530 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.530 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.531 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.531 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.531 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.531 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.531 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.531 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.531 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.531 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.532 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.532 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.532 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.532 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.532 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.532 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.532 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.533 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.533 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.533 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.533 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.533 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.533 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.533 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.534 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.534 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.534 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.534 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.534 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.534 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.534 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.535 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.535 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.535 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.535 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.535 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.535 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.535 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.536 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.536 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.536 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.536 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.536 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.536 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.537 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.537 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.537 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.537 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.537 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.537 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.537 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.538 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.538 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.538 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.539 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.539 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.539 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2023-07-06T20:37:04.539 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.539 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.540 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.540 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.540 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.540 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.540 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.540 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.541 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.541 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.541 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.541 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.541 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.542 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.542 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.542 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.542 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.542 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.543 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.543 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.543 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.543 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.543 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.543 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.544 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.544 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.544 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.544 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.544 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.544 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.544 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.545 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.545 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.545 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.545 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.545 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.545 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.545 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.546 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.546 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.546 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.546 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.546 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.546 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.546 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.546 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.547 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.547 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.547 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.547 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.547 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.547 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.547 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.547 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.548 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.548 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.548 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.548 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.548 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.548 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.548 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.549 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.549 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.549 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.549 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.549 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.549 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.549 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.549 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.550 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.550 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.550 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.550 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.550 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.550 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2023-07-06T20:37:04.550 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.551 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.551 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.551 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.551 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.551 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.551 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.551 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.551 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.552 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.552 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.552 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.552 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.552 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.552 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.552 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.552 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.553 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.553 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.553 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.553 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.553 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.553 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.553 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.553 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.554 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.554 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.554 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.554 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.554 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.554 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.554 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.555 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.555 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.555 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.555 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.555 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.555 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.555 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.556 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.556 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.556 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.556 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.556 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.556 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.556 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.557 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.557 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.557 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.557 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.557 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.557 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.557 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.557 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.558 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.558 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.558 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.558 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.558 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.558 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.558 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.559 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.559 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.559 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.559 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.559 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.559 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.560 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.560 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.560 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.560 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.560 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.560 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.560 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2023-07-06T20:37:04.561 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.561 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.561 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.561 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2023-07-06T20:37:04.561 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2023-07-06T20:37:04.561 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.562 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.562 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.562 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.562 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.562 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.562 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.562 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-06T20:37:04.562 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.563 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.563 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.563 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.563 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.563 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.564 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.564 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.564 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.564 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.564 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.564 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.564 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.565 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.565 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.565 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.565 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.565 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.565 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.566 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.566 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.566 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.566 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.566 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.566 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.566 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.567 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.567 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.567 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.567 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.567 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.567 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.567 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.567 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.568 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.568 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.568 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.568 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.568 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.569 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.569 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.569 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.569 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.569 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.569 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.569 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.569 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.570 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.570 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.570 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.570 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.570 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.570 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.571 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.571 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.571 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.571 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.571 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.572 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.572 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.572 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.572 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.572 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.573 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.573 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.573 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.573 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.573 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.573 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.573 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2023-07-06T20:37:04.574 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2023-07-06T20:37:04.574 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.574 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.574 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.574 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.574 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.574 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.574 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.575 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2023-07-06T20:37:04.575 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2023-07-06T20:37:04.575 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2023-07-06T20:37:04.575 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.575 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2023-07-06T20:37:04.575 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.575 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.576 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.576 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.576 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.576 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.576 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.576 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2023-07-06T20:37:04.576 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.576 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.577 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.577 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.577 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.577 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.577 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.577 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.577 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.577 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.578 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.578 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.578 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.578 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.578 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.578 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.578 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.578 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.579 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.579 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.579 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.579 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.579 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.579 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.579 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.579 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.580 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.580 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.580 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.580 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.580 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.580 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.580 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.580 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.581 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.581 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.581 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.581 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.581 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.581 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.581 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.582 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.582 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.582 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.582 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.582 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.582 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.582 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.582 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.583 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.583 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.583 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.583 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.583 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.583 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.583 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.583 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.584 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.584 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.584 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.584 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.584 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.584 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.584 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.585 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.585 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.585 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.585 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.585 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.585 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.585 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.585 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.586 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.586 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.586 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.586 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.586 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2023-07-06T20:37:04.586 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.586 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.586 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.587 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.587 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.587 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.587 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.587 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.587 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.587 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.587 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.588 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.588 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.588 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.588 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.588 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.588 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.588 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.589 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.589 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.589 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.589 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.589 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.589 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.589 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.590 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.590 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.590 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.590 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.590 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.590 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.590 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.591 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.591 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.591 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.591 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.591 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.591 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.591 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.592 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.592 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.592 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.592 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.592 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.592 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.593 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.593 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.593 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.593 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.593 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.593 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.593 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.594 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.594 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.594 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.594 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.594 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.594 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.595 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.595 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.595 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.595 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.595 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.595 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.595 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.596 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.596 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.596 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.596 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.596 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.597 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.597 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.597 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.597 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.597 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.597 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.598 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2023-07-06T20:37:04.598 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.598 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.598 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.598 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.599 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.599 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.599 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.599 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.599 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.599 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.599 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.599 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.600 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.600 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.600 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.600 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.600 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.600 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.600 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.600 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.601 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.601 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.601 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.601 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.601 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.601 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.601 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.602 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.602 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.602 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.602 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.602 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.602 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.602 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.602 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.603 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.603 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.603 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.603 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.603 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.603 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.603 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.603 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.604 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.604 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.604 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.604 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.604 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.604 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.604 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.604 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.605 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.605 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.605 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.605 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.605 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.605 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.605 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.605 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.606 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.606 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.606 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.606 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.606 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.606 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.606 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.606 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.607 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.607 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.607 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.607 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.607 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.607 INFO:teuthology.orchestra.run.smithi069.stdout: "noautoscale": { 2023-07-06T20:37:04.608 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "noautoscale", 2023-07-06T20:37:04.608 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.608 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.608 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.608 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.608 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.608 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.608 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.609 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "global autoscale flag", 2023-07-06T20:37:04.609 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-06T20:37:04.609 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.609 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.609 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.609 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:04.609 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:04.609 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.610 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.610 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.610 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:04.610 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.610 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.610 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.610 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.611 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.611 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.611 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.611 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.611 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2023-07-06T20:37:04.611 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2023-07-06T20:37:04.611 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:04.611 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.612 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.612 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2023-07-06T20:37:04.612 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2023-07-06T20:37:04.612 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.612 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.612 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2023-07-06T20:37:04.612 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-06T20:37:04.612 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.613 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.613 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.613 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.613 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.613 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.613 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2023-07-06T20:37:04.613 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.613 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.614 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.614 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2023-07-06T20:37:04.614 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2023-07-06T20:37:04.614 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.614 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.614 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.614 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.615 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.615 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.615 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.615 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-06T20:37:04.615 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.615 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.616 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.616 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.616 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2023-07-06T20:37:04.616 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2023-07-06T20:37:04.616 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.616 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.616 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.617 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.617 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.617 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.617 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.617 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.617 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.618 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.618 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.618 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.618 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.618 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.618 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.618 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.618 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.619 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.619 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.619 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.619 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.619 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.619 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.620 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.620 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.620 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.620 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.620 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.620 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.621 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.621 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.621 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.621 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.621 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.622 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.622 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.622 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.622 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.622 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.622 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.623 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.623 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.623 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.623 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.623 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.623 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.623 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.624 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.624 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.624 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.624 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.624 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.624 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.625 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.625 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.625 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.625 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.625 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.625 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.625 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.626 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.626 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.626 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.626 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.626 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.627 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.627 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.627 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.627 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.627 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.627 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.627 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.628 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.628 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.628 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.628 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.628 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.628 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.628 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.628 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.629 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.629 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2023-07-06T20:37:04.629 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2023-07-06T20:37:04.629 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.629 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.629 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.629 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2023-07-06T20:37:04.629 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.630 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.630 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.630 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2023-07-06T20:37:04.630 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.630 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.630 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.630 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.630 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:04.631 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:04.631 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.631 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.631 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.631 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2023-07-06T20:37:04.631 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.631 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.632 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.632 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2023-07-06T20:37:04.632 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.632 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.632 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.632 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.632 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.632 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.633 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.633 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2023-07-06T20:37:04.633 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.633 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.633 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.633 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2023-07-06T20:37:04.633 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2023-07-06T20:37:04.634 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.634 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.634 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.634 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.634 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.634 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.634 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.635 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.635 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.635 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.635 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.635 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2023-07-06T20:37:04.635 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2023-07-06T20:37:04.635 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.635 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.636 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.636 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.636 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.636 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.636 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.636 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-06T20:37:04.636 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-06T20:37:04.637 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.637 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.637 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.637 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.637 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.637 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.637 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.637 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.638 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.638 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.638 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.638 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.638 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.638 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.638 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.638 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.639 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.639 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.639 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.639 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.639 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.639 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.639 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.639 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.640 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.640 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.640 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.640 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.640 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.640 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.640 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.640 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.641 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.641 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.641 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.641 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.641 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.641 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.641 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.642 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.642 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.642 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.642 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.642 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.642 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.642 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.642 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.643 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.643 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.643 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.643 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.643 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.643 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.643 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.644 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.644 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.644 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.644 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.644 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.644 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.644 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.645 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.645 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.645 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.645 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.645 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.645 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.646 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.646 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.646 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.646 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.646 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.646 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.647 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2023-07-06T20:37:04.647 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2023-07-06T20:37:04.647 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.647 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.647 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.647 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.647 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.648 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.648 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.648 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.648 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.648 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.648 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.649 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.649 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-06T20:37:04.649 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-06T20:37:04.649 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.649 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.649 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.650 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2023-07-06T20:37:04.650 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.650 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.650 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.650 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.650 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.651 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.651 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.651 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.651 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2023-07-06T20:37:04.651 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2023-07-06T20:37:04.651 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:04.652 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.652 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.652 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2023-07-06T20:37:04.652 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.652 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.653 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.653 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.653 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.653 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.653 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.653 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.653 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2023-07-06T20:37:04.654 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2023-07-06T20:37:04.654 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.654 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.654 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.654 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2023-07-06T20:37:04.654 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.654 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.654 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.655 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-06T20:37:04.655 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.655 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.655 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.655 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.655 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2023-07-06T20:37:04.655 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2023-07-06T20:37:04.655 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.656 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.656 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.656 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2023-07-06T20:37:04.656 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.656 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.656 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.656 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-06T20:37:04.657 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.657 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.657 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.657 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.657 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2023-07-06T20:37:04.657 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2023-07-06T20:37:04.657 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.657 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.658 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.658 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2023-07-06T20:37:04.658 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.658 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.658 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.658 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.658 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.658 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.659 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.659 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.659 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2023-07-06T20:37:04.659 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2023-07-06T20:37:04.659 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.659 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.659 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.659 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2023-07-06T20:37:04.660 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.660 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.660 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.660 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2023-07-06T20:37:04.660 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2023-07-06T20:37:04.660 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.660 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.660 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.661 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.661 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.661 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.661 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2023-07-06T20:37:04.661 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2023-07-06T20:37:04.661 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.661 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.661 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.662 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2023-07-06T20:37:04.662 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2023-07-06T20:37:04.662 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2023-07-06T20:37:04.662 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.662 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.662 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.662 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.662 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.663 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.663 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.663 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.663 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.663 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2023-07-06T20:37:04.663 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.663 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.664 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.664 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.664 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.664 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.664 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.664 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.664 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.664 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.665 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.665 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.665 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.665 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.665 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.665 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.665 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.665 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.666 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.666 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.666 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.666 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.666 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.666 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.666 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.666 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.667 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.667 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.667 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.667 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.667 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.667 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.667 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.667 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.668 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.668 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.668 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.668 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.668 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.668 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.668 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.669 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.669 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.669 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.669 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.669 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.669 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.670 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.670 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.670 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.670 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.670 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.670 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.670 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.670 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.671 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.671 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.671 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.671 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.671 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.671 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.672 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.672 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.672 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.672 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.672 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.672 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.672 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.673 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.673 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.673 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.673 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.673 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.673 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2023-07-06T20:37:04.673 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2023-07-06T20:37:04.674 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.674 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.674 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.674 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:04.674 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.674 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.675 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.675 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.675 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.675 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.675 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.675 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.676 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2023-07-06T20:37:04.676 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2023-07-06T20:37:04.676 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.676 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.676 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.676 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.677 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.677 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.677 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.677 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.677 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.677 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.678 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.678 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.678 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2023-07-06T20:37:04.678 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2023-07-06T20:37:04.678 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.678 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.679 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.679 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.679 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.679 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.679 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.679 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.679 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.679 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.680 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.680 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.680 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.680 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.680 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.680 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2023-07-06T20:37:04.681 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.681 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.681 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.681 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2023-07-06T20:37:04.681 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2023-07-06T20:37:04.681 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.681 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.681 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.682 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.682 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.682 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.682 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.682 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.682 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.682 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.683 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.683 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2023-07-06T20:37:04.683 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2023-07-06T20:37:04.683 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.683 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.683 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.683 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.684 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.684 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.684 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.684 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.684 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.684 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.684 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.684 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.685 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.685 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.685 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.685 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.685 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.685 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.685 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.685 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.686 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.686 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.686 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.686 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.686 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.686 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.686 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.686 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.687 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.687 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.687 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.687 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.687 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.687 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.687 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.687 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.688 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.688 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.688 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.688 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.688 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.688 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.688 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.688 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.689 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.689 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.689 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.689 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.689 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.689 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.689 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.690 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.690 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.690 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.690 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.690 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.690 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.690 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.690 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.691 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.691 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.691 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.691 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.691 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.691 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.691 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.691 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.692 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.692 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.692 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.692 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.692 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.692 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.692 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.692 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.693 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.693 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.693 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.693 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.693 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.693 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.693 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.693 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2023-07-06T20:37:04.694 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2023-07-06T20:37:04.694 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.694 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.694 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.694 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.694 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.694 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.694 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.695 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.695 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.695 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.695 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.695 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.695 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2023-07-06T20:37:04.695 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2023-07-06T20:37:04.695 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.696 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.696 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.696 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.696 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.696 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.696 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.696 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.697 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.697 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.697 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.697 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.697 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.697 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.697 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.698 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2023-07-06T20:37:04.698 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.698 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.698 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.698 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.698 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.699 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.699 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.699 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.699 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.699 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.699 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.699 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.700 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.700 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.700 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.700 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.700 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.700 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.701 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.701 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.701 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.701 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.701 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.701 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.701 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.702 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.702 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.702 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.702 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.702 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.702 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.702 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.703 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.703 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.703 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.703 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.703 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.703 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.703 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.704 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.704 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.704 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.704 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.704 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.704 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.705 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.705 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.705 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.705 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.705 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.705 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.706 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.706 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.706 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.706 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.706 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.706 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.707 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.707 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.707 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.707 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.707 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.708 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.708 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.708 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.708 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.708 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.708 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.709 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.709 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.709 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.709 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.709 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.709 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.709 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.710 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.710 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2023-07-06T20:37:04.710 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.710 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.710 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.710 INFO:teuthology.orchestra.run.smithi069.stdout: "drive_group_interval": { 2023-07-06T20:37:04.710 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "drive_group_interval", 2023-07-06T20:37:04.710 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:04.711 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.711 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.711 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300.0", 2023-07-06T20:37:04.711 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.711 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.711 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.711 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-06T20:37:04.711 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.712 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.712 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.712 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.712 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.712 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.712 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.712 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.712 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.713 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.713 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.713 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.713 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.713 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.713 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.713 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.713 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.714 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.714 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.714 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.714 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.714 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.714 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.714 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.714 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.715 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.715 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.715 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.715 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.715 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.715 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.715 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.715 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.716 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.716 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.716 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.716 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.716 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.716 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.716 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.717 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.717 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.717 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.717 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.717 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.717 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.717 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.717 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.718 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.718 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.718 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.718 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.718 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.718 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.718 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.718 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.719 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.719 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.719 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.719 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.719 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.719 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.719 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.719 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.720 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.720 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.720 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.720 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.720 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.720 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.720 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.720 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.721 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.721 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.721 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2023-07-06T20:37:04.721 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2023-07-06T20:37:04.721 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.721 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.721 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.722 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2023-07-06T20:37:04.722 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.722 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.722 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.722 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-06T20:37:04.722 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.722 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.722 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.723 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.723 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.723 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.723 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.723 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2023-07-06T20:37:04.723 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.723 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.723 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.724 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.724 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.724 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.724 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.724 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.724 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.724 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.725 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.725 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.725 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.725 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.725 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.725 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.725 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.726 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.726 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.726 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.726 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.726 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.726 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.727 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.727 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.727 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.727 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.727 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.727 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.727 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.728 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.728 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.728 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.728 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.728 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.728 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.729 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.729 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.729 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.729 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.729 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.729 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.729 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.730 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.730 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.730 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.730 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.730 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.730 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.730 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.731 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.731 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.731 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.731 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.731 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.731 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.732 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.732 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.732 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.732 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.732 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.732 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.733 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.733 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.733 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.733 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.733 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.733 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.734 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.734 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.734 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.734 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.734 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.734 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2023-07-06T20:37:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2023-07-06T20:37:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.737 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.737 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2023-07-06T20:37:04.737 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2023-07-06T20:37:04.737 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.737 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.737 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.737 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2023-07-06T20:37:04.737 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.738 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.738 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.738 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.738 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.738 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.738 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.738 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.738 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2023-07-06T20:37:04.739 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2023-07-06T20:37:04.739 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.739 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.739 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.739 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.739 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.739 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.739 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.740 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.740 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.740 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.740 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.740 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.740 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2023-07-06T20:37:04.740 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2023-07-06T20:37:04.740 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.741 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.741 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.741 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.741 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.741 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.741 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.741 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.741 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.742 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.742 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.742 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.742 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2023-07-06T20:37:04.742 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2023-07-06T20:37:04.742 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:04.742 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.742 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.743 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.743 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.743 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.743 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.743 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.743 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.744 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.744 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.744 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.744 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2023-07-06T20:37:04.744 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2023-07-06T20:37:04.745 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.745 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.745 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.745 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.745 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.746 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.746 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.746 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.746 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.746 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.746 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.747 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.747 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2023-07-06T20:37:04.747 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2023-07-06T20:37:04.747 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.747 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.747 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.748 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.748 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.748 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.748 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.748 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.749 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.749 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.749 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.749 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.749 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2023-07-06T20:37:04.749 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2023-07-06T20:37:04.750 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.750 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.750 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.750 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.751 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.751 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.751 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.751 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.751 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.751 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.752 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.752 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2023-07-06T20:37:04.752 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2023-07-06T20:37:04.752 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.753 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.753 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.753 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.753 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2023-07-06T20:37:04.753 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2023-07-06T20:37:04.753 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.754 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.754 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.754 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.754 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.754 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.755 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2023-07-06T20:37:04.755 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2023-07-06T20:37:04.755 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.755 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.755 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.755 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.755 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.755 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.756 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.756 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.756 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.756 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.756 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.756 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.756 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2023-07-06T20:37:04.757 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2023-07-06T20:37:04.757 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.757 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.757 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.757 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.757 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.757 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.758 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.758 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.758 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.758 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.758 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.758 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.758 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2023-07-06T20:37:04.759 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2023-07-06T20:37:04.759 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.759 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.759 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.759 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.759 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.759 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.759 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.760 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.760 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.760 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.760 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.760 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.760 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.761 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.761 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2023-07-06T20:37:04.761 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.761 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.761 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.762 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2023-07-06T20:37:04.762 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2023-07-06T20:37:04.762 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.762 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.762 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.762 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.763 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.763 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.763 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.763 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2023-07-06T20:37:04.763 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.763 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.764 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.764 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.764 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2023-07-06T20:37:04.764 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2023-07-06T20:37:04.764 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.764 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.765 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.765 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.765 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.765 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.765 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.766 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2023-07-06T20:37:04.766 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.766 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.766 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.766 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.766 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.766 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.767 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.767 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.767 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.767 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.767 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.767 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.767 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.767 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.768 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.768 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.768 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.768 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.768 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.768 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.768 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.768 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.769 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.769 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.769 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.769 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.769 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.769 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.769 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.769 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.770 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.770 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.770 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.770 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.770 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.770 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.770 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.770 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.771 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.771 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.771 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.771 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.771 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.771 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.771 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.772 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.772 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.772 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.772 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.772 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.772 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.772 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.772 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.773 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.773 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.773 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.773 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.773 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.773 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.773 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.773 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.774 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.774 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.774 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.774 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.774 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.774 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.774 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.775 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.775 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.775 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.775 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.775 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.775 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.775 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.775 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.776 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.776 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2023-07-06T20:37:04.776 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.776 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.776 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.776 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.776 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.777 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.777 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.777 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.777 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.777 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.777 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.777 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.777 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.778 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.778 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.778 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.778 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.778 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.778 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.778 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.778 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.779 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.779 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.779 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.779 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.779 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.779 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.779 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.779 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.780 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.780 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.780 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.780 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.780 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.780 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.780 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.780 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.781 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.781 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.781 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.781 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.781 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.781 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.781 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.781 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.782 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.782 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.782 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.782 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.782 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.782 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.782 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.782 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.783 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.783 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.783 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.783 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.783 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.783 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.783 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.784 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.784 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.784 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.784 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.784 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.784 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.785 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.785 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.785 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.785 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.785 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.785 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.786 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.786 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.786 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.786 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.786 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2023-07-06T20:37:04.786 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.786 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.787 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.787 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.787 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.787 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.787 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.787 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.788 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.788 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.788 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.788 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.788 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.788 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.789 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.789 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.789 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.789 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.789 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.789 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.789 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.790 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.790 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.790 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.790 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.790 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.790 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.791 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.791 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.791 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.791 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.791 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.791 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.792 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.792 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.792 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.792 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.792 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.793 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.793 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.793 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.793 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.793 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.793 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.794 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.794 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.794 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.794 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.794 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.794 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.795 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.795 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.795 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.795 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.795 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.795 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.795 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.795 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.796 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.796 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.796 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.796 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.796 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.796 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.796 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.796 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.797 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.797 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.797 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.797 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.797 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.797 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.797 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.798 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.798 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.798 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.798 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2023-07-06T20:37:04.798 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.798 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.798 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.798 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2023-07-06T20:37:04.799 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2023-07-06T20:37:04.799 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.799 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.799 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.799 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-06T20:37:04.799 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.799 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.799 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.800 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.800 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.800 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.800 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.800 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.800 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:04.800 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:04.801 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.801 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.801 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.801 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2023-07-06T20:37:04.801 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.801 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.801 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.801 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.802 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.802 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.802 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.802 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.802 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.802 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.802 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.802 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.803 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.803 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.803 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.803 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.803 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.803 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.803 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.803 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.804 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.804 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.804 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.804 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.804 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.804 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.804 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.805 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.805 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.805 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.805 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.805 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.805 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.805 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.806 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.806 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.806 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.806 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.806 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.806 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.806 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.806 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.807 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.807 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.807 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.807 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.807 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.807 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.807 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.808 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.808 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.808 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.808 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.808 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.808 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.808 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.808 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.809 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.809 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.809 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.809 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.809 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.809 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.809 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.809 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.810 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.810 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.810 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.810 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.810 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.810 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.810 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.810 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.811 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.811 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.811 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.811 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.811 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.811 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.811 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.812 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2023-07-06T20:37:04.812 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.812 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.812 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.812 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2023-07-06T20:37:04.813 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2023-07-06T20:37:04.813 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.813 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.813 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.813 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.813 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.813 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.814 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.814 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-06T20:37:04.814 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.814 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.814 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.814 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.814 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2023-07-06T20:37:04.815 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2023-07-06T20:37:04.815 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.815 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.815 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.815 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.815 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.815 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.816 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.816 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-06T20:37:04.816 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.816 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.816 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.816 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.816 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2023-07-06T20:37:04.816 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2023-07-06T20:37:04.817 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.817 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.817 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.817 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.817 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.817 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.817 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.818 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-06T20:37:04.818 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.818 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.818 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.818 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.818 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2023-07-06T20:37:04.818 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2023-07-06T20:37:04.819 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.819 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.819 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.819 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.819 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.819 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.819 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.819 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-06T20:37:04.820 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.820 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.820 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.820 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.820 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2023-07-06T20:37:04.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2023-07-06T20:37:04.820 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.821 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.821 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.821 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.821 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.821 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.821 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.822 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-06T20:37:04.822 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.822 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.822 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.822 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.822 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2023-07-06T20:37:04.823 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2023-07-06T20:37:04.823 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.823 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.823 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.823 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.823 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.824 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.824 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.824 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.824 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.824 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.825 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.825 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.825 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2023-07-06T20:37:04.825 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2023-07-06T20:37:04.825 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.825 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.826 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.826 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.826 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.826 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.826 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.826 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.826 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.827 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.827 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.827 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.827 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2023-07-06T20:37:04.827 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2023-07-06T20:37:04.827 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.827 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.827 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.828 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-06T20:37:04.828 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.828 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.828 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.828 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.828 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.828 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.829 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.829 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.829 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2023-07-06T20:37:04.829 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2023-07-06T20:37:04.829 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.829 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.829 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.829 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.830 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.830 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.830 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.830 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.830 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.830 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.830 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.830 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.831 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:04.831 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:04.831 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.831 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.831 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.831 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2023-07-06T20:37:04.831 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2023-07-06T20:37:04.831 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.832 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.832 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.832 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.832 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.832 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.832 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.832 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2023-07-06T20:37:04.833 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2023-07-06T20:37:04.833 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.833 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.833 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.833 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2023-07-06T20:37:04.833 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.833 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.833 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.834 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.834 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.834 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.834 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.834 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.834 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2023-07-06T20:37:04.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2023-07-06T20:37:04.834 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.835 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.835 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.835 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.835 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.835 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.835 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.835 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.835 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.836 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.836 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.836 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.836 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2023-07-06T20:37:04.836 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2023-07-06T20:37:04.836 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.836 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.837 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.837 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.837 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.837 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.837 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.837 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.837 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.837 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.838 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.838 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.838 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.838 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.838 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.838 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.838 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.839 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.839 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.839 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.839 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.839 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.839 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.839 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.839 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.840 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.840 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.840 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.840 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.840 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.840 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.840 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.840 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.841 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.841 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.841 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.841 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.841 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.841 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.841 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.842 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.842 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.842 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.842 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.842 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.842 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.842 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.843 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.843 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.843 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.843 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.843 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.844 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.844 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.844 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.844 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.844 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.844 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.845 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.845 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.845 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.845 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.845 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.845 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.845 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.846 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.846 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.846 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.846 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.846 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.846 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.846 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.847 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.847 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.847 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.847 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.847 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.847 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.847 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.848 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.848 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2023-07-06T20:37:04.848 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2023-07-06T20:37:04.848 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.848 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.849 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.849 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.849 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.849 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.849 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.849 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.849 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.850 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.850 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.850 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.850 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2023-07-06T20:37:04.850 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2023-07-06T20:37:04.850 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.851 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.851 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.851 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.851 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.851 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.851 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.852 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.852 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.852 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.852 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.852 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.853 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2023-07-06T20:37:04.853 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2023-07-06T20:37:04.853 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.853 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.853 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.853 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-06T20:37:04.854 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.854 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.854 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.854 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.854 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.854 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.854 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.854 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.855 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.855 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.855 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.855 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2023-07-06T20:37:04.855 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.855 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.855 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.856 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.856 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.856 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.856 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.856 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.856 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.856 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.857 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.857 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.857 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.857 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.857 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.857 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.857 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.857 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.858 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.858 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.858 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.858 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.858 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.858 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.858 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.858 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.859 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.859 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.859 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.859 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.859 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.859 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.859 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.859 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.860 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.860 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.860 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.860 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.860 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.860 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.860 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.860 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.861 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.861 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.861 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.861 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.861 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.861 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.861 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.861 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.862 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.862 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.862 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.862 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.862 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.862 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.862 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.863 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.863 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.863 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.863 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.863 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.863 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.863 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.863 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.864 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.864 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.864 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.864 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.864 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.864 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.864 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.865 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.865 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.865 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.865 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2023-07-06T20:37:04.865 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.865 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.865 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.866 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.866 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.866 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.866 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.866 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.866 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.866 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.866 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.867 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.867 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.867 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.867 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.867 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.867 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.867 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.867 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.868 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.868 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.868 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.868 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.868 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.868 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.868 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.869 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.869 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.869 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.869 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.869 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.869 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.869 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.870 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.870 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.870 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.870 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.870 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.870 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.870 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.871 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.871 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.871 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.871 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.871 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.871 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.872 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.872 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.872 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.872 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.872 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.872 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.872 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.873 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.873 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.873 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.873 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.873 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.873 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.873 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.874 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.874 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.874 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.874 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.874 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.874 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.874 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.875 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.875 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.875 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.875 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.875 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.875 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.876 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2023-07-06T20:37:04.876 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2023-07-06T20:37:04.876 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.876 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.876 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.876 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2023-07-06T20:37:04.876 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.876 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.877 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.877 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-06T20:37:04.877 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.877 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.877 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.877 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.877 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2023-07-06T20:37:04.878 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2023-07-06T20:37:04.878 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.878 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.878 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.878 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:04.878 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.878 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.878 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.879 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-06T20:37:04.879 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.879 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.879 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.879 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.879 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.879 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.880 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.880 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2023-07-06T20:37:04.880 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.880 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.880 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.881 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2023-07-06T20:37:04.881 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2023-07-06T20:37:04.881 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:04.881 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.881 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.881 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2023-07-06T20:37:04.882 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.882 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.882 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.882 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.882 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.883 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.883 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.883 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2023-07-06T20:37:04.883 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2023-07-06T20:37:04.883 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.883 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.883 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.884 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.884 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.884 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.884 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.884 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.884 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.884 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.885 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.885 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.885 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:04.885 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:04.885 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.885 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.885 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.885 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:04.886 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.886 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.886 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.886 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.886 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.886 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.886 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.886 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.887 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.887 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.887 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.887 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.887 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.887 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.887 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.888 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.888 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.888 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.888 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.888 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.888 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.888 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.888 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.889 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.889 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.889 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.889 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.889 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.889 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.889 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.890 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.890 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.890 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.890 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.890 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.890 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.890 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.890 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.891 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.891 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.891 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.891 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.891 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.891 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.891 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.892 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.892 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.892 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.892 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.892 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.892 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.892 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.892 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.893 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.893 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.893 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.893 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.893 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.893 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.893 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.894 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.894 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.894 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.894 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.894 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.894 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.894 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.894 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.895 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.895 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.895 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.895 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.895 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.895 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.895 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.895 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.896 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.896 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2023-07-06T20:37:04.896 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2023-07-06T20:37:04.896 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.896 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.896 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.896 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.897 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.897 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.897 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.897 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.897 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.897 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.897 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.897 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.898 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2023-07-06T20:37:04.898 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2023-07-06T20:37:04.898 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.898 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.898 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.898 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2023-07-06T20:37:04.898 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.898 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.899 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.899 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.899 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.899 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.899 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.899 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2023-07-06T20:37:04.899 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2023-07-06T20:37:04.900 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.900 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.900 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.900 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-06T20:37:04.900 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.900 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.901 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.901 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.901 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.901 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.901 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.901 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.901 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.902 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.902 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2023-07-06T20:37:04.902 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.902 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.902 INFO:teuthology.orchestra.run.smithi069.stdout: "modules": [ 2023-07-06T20:37:04.902 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2023-07-06T20:37:04.903 INFO:teuthology.orchestra.run.smithi069.stdout: "dashboard", 2023-07-06T20:37:04.903 INFO:teuthology.orchestra.run.smithi069.stdout: "iostat", 2023-07-06T20:37:04.903 INFO:teuthology.orchestra.run.smithi069.stdout: "nfs", 2023-07-06T20:37:04.903 INFO:teuthology.orchestra.run.smithi069.stdout: "restful" 2023-07-06T20:37:04.903 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.903 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2023-07-06T20:37:04.903 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.904 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2023-07-06T20:37:04.904 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.904 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.904 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.904 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:04.904 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:04.905 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.905 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.905 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.905 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:04.905 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.905 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.905 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.906 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2023-07-06T20:37:04.906 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.906 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.906 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.906 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.906 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.907 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.907 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.907 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.907 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.907 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.907 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.908 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.908 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.908 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.908 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.908 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.908 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.909 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.909 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.909 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.909 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.909 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.910 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.910 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.910 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.910 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.910 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.910 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.911 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.911 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.911 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.911 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.911 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.911 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.912 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.912 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.912 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.912 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.912 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.912 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.912 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.913 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.913 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.913 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.913 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.913 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.913 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.913 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.913 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.914 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.914 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.914 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.914 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.914 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.914 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.914 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.914 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.915 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.915 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.915 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.915 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.915 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.915 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.915 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.916 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.916 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.916 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.916 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.916 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.916 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.916 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.917 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.917 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2023-07-06T20:37:04.917 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2023-07-06T20:37:04.917 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.917 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.917 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.917 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.917 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.918 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.918 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.918 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2023-07-06T20:37:04.918 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.918 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.918 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.918 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.918 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2023-07-06T20:37:04.919 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2023-07-06T20:37:04.919 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.919 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.919 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.919 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2023-07-06T20:37:04.919 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.919 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.919 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.920 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2023-07-06T20:37:04.920 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.920 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.920 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.920 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2023-07-06T20:37:04.920 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2023-07-06T20:37:04.921 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.921 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.921 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.921 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.921 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.921 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.921 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.921 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2023-07-06T20:37:04.922 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.922 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.922 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.922 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.922 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2023-07-06T20:37:04.922 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2023-07-06T20:37:04.922 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.922 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.923 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.923 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.923 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.923 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.923 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.923 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2023-07-06T20:37:04.923 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.923 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.924 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.924 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.924 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2023-07-06T20:37:04.924 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2023-07-06T20:37:04.924 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.924 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.924 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.925 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2023-07-06T20:37:04.925 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.925 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.925 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.925 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2023-07-06T20:37:04.925 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.925 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.925 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.926 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.926 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2023-07-06T20:37:04.926 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2023-07-06T20:37:04.926 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.926 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.926 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.926 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.926 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.927 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.927 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.927 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2023-07-06T20:37:04.927 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.927 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.928 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.928 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.928 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2023-07-06T20:37:04.928 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2023-07-06T20:37:04.928 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.928 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.928 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.928 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.929 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.929 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.929 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.929 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-06T20:37:04.929 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.929 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.930 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.930 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.930 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2023-07-06T20:37:04.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2023-07-06T20:37:04.930 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.930 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.930 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.930 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.931 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.931 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.931 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.931 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2023-07-06T20:37:04.931 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.931 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.932 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.932 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.932 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.932 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.932 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2023-07-06T20:37:04.933 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.933 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.933 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.933 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2023-07-06T20:37:04.933 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2023-07-06T20:37:04.933 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.933 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.934 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.934 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.934 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.934 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.934 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.934 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2023-07-06T20:37:04.935 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.935 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.935 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.935 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2023-07-06T20:37:04.935 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2023-07-06T20:37:04.935 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.935 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.936 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.936 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2023-07-06T20:37:04.936 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.936 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.936 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.936 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2023-07-06T20:37:04.936 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-06T20:37:04.937 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.937 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.937 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.937 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2023-07-06T20:37:04.937 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2023-07-06T20:37:04.937 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:04.937 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.937 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.938 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:04.938 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2023-07-06T20:37:04.938 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2023-07-06T20:37:04.938 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.938 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-06T20:37:04.938 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-06T20:37:04.939 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.939 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.939 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.939 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2023-07-06T20:37:04.939 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2023-07-06T20:37:04.939 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:04.940 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.940 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.940 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2023-07-06T20:37:04.940 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2023-07-06T20:37:04.940 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2023-07-06T20:37:04.941 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.941 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-06T20:37:04.941 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.941 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.941 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.941 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.941 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2023-07-06T20:37:04.942 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2023-07-06T20:37:04.942 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.942 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.942 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.942 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2023-07-06T20:37:04.942 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.942 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.942 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.943 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-06T20:37:04.943 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-06T20:37:04.943 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.943 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.943 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.943 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2023-07-06T20:37:04.943 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2023-07-06T20:37:04.944 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:04.944 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.944 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.944 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2023-07-06T20:37:04.944 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2023-07-06T20:37:04.944 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2023-07-06T20:37:04.944 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.944 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2023-07-06T20:37:04.945 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-06T20:37:04.945 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.945 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.945 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.945 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2023-07-06T20:37:04.945 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2023-07-06T20:37:04.945 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.945 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.946 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.946 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2023-07-06T20:37:04.946 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.946 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.946 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.946 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2023-07-06T20:37:04.946 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-06T20:37:04.946 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.947 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.947 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.947 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2023-07-06T20:37:04.947 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2023-07-06T20:37:04.947 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:04.947 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.947 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.948 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:04.948 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2023-07-06T20:37:04.948 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2023-07-06T20:37:04.948 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.948 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-06T20:37:04.948 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-06T20:37:04.948 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.948 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.949 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.949 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:04.949 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:04.949 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.949 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.949 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.949 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.950 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.950 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.950 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.950 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.950 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.950 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.950 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.950 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.951 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.951 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.951 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.951 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.951 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.951 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.951 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.951 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:04.952 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:04.952 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.952 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.952 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.952 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.952 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.952 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.953 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.953 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.953 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.953 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.953 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:04.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:04.953 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.954 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.954 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.954 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:04.954 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.954 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.954 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.954 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:04.955 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:04.955 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:04.955 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:04.955 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:04.955 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:04.955 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.955 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.955 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.956 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.956 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.956 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.956 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:04.956 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:04.956 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.956 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.956 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.957 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.957 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.957 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.957 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.957 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:04.957 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.957 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.958 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.958 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.958 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2023-07-06T20:37:04.958 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2023-07-06T20:37:04.958 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:04.958 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.958 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.959 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:04.959 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.959 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.959 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.959 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-06T20:37:04.959 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.959 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.960 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.960 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.960 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2023-07-06T20:37:04.960 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2023-07-06T20:37:04.960 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.960 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.961 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.961 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2023-07-06T20:37:04.961 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.961 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.961 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:04.961 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2023-07-06T20:37:04.961 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2023-07-06T20:37:04.962 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap" 2023-07-06T20:37:04.962 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:04.962 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2023-07-06T20:37:04.962 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.962 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.962 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.962 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.963 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2023-07-06T20:37:04.963 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2023-07-06T20:37:04.963 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.963 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.963 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.963 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:04.964 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.964 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.964 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.964 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-06T20:37:04.964 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.964 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.965 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.965 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.965 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:04.965 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:04.965 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.966 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.966 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.966 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:04.966 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.966 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.966 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.967 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-06T20:37:04.967 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.967 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.967 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.967 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.968 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2023-07-06T20:37:04.968 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2023-07-06T20:37:04.968 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.968 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.968 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.968 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2023-07-06T20:37:04.968 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2023-07-06T20:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-06T20:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-06T20:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.969 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2023-07-06T20:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2023-07-06T20:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.970 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-06T20:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2023-07-06T20:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2023-07-06T20:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2023-07-06T20:37:04.972 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2023-07-06T20:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-06T20:37:04.973 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2023-07-06T20:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2023-07-06T20:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.974 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2023-07-06T20:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-06T20:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.975 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2023-07-06T20:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2023-07-06T20:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2023-07-06T20:37:04.976 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-06T20:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.977 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: "alertmanager_web_password": { 2023-07-06T20:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alertmanager_web_password", 2023-07-06T20:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.978 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Alertmanager web password", 2023-07-06T20:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: "alertmanager_web_user": { 2023-07-06T20:37:04.979 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alertmanager_web_user", 2023-07-06T20:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.980 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Alertmanager web user", 2023-07-06T20:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2023-07-06T20:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2023-07-06T20:37:04.981 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-06T20:37:04.982 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-06T20:37:04.983 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.983 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.983 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.983 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2023-07-06T20:37:04.983 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2023-07-06T20:37:04.983 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:04.983 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.984 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.984 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:04.984 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.984 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.984 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.984 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-06T20:37:04.985 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.985 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.985 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.985 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.985 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2023-07-06T20:37:04.985 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2023-07-06T20:37:04.985 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:04.986 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.986 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.986 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2023-07-06T20:37:04.986 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.986 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.986 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.986 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-06T20:37:04.987 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.987 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.987 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.987 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.987 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2023-07-06T20:37:04.987 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2023-07-06T20:37:04.987 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.988 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.988 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.988 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.988 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.988 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.989 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.989 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-06T20:37:04.989 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.989 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.989 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.989 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.989 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2023-07-06T20:37:04.989 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2023-07-06T20:37:04.990 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.990 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.990 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.990 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:04.990 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.990 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.991 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.991 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-06T20:37:04.991 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.991 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.991 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.991 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.992 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2023-07-06T20:37:04.992 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2023-07-06T20:37:04.992 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:04.992 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.992 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.992 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:04.993 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.993 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.993 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.993 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-06T20:37:04.993 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.993 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.994 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.994 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.994 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2023-07-06T20:37:04.994 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2023-07-06T20:37:04.994 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.994 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.994 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.994 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-06T20:37:04.995 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.995 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.995 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.995 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2023-07-06T20:37:04.995 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.995 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.995 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.996 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.996 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2023-07-06T20:37:04.996 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2023-07-06T20:37:04.996 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.996 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.996 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:04.996 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-06T20:37:04.997 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.997 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.997 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.997 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2023-07-06T20:37:04.997 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.997 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:04.998 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:04.998 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:04.998 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2023-07-06T20:37:04.998 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2023-07-06T20:37:04.998 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:04.998 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:04.999 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:04.999 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-06T20:37:04.999 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:04.999 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:04.999 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:04.999 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2023-07-06T20:37:04.999 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:04.999 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.000 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.000 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.000 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2023-07-06T20:37:05.000 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2023-07-06T20:37:05.000 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.000 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.000 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.001 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-06T20:37:05.001 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.001 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.001 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.001 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2023-07-06T20:37:05.001 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.001 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.001 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.002 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2023-07-06T20:37:05.002 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2023-07-06T20:37:05.002 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.002 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.002 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.002 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-06T20:37:05.002 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.003 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.003 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.003 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2023-07-06T20:37:05.003 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.003 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.003 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.003 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.004 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2023-07-06T20:37:05.004 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2023-07-06T20:37:05.004 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.004 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.004 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.004 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-06T20:37:05.004 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.004 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.005 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.005 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2023-07-06T20:37:05.005 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.005 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.005 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.005 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.005 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2023-07-06T20:37:05.006 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2023-07-06T20:37:05.006 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.006 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.006 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.006 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-06T20:37:05.006 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.006 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.006 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.007 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2023-07-06T20:37:05.007 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.007 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.007 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.007 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.007 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2023-07-06T20:37:05.007 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2023-07-06T20:37:05.007 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.008 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.008 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.008 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-06T20:37:05.008 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.008 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.008 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.008 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2023-07-06T20:37:05.008 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.009 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.009 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.009 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.009 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2023-07-06T20:37:05.009 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2023-07-06T20:37:05.009 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.009 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.010 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.010 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-06T20:37:05.010 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.010 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.010 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.010 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2023-07-06T20:37:05.010 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.010 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.011 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.011 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.011 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2023-07-06T20:37:05.011 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2023-07-06T20:37:05.011 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.011 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.011 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.011 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-06T20:37:05.012 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.012 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.012 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.012 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2023-07-06T20:37:05.012 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.012 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.012 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.013 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.013 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2023-07-06T20:37:05.013 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2023-07-06T20:37:05.013 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.013 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.013 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.013 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-06T20:37:05.014 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.014 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.014 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.014 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2023-07-06T20:37:05.014 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.014 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.014 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.014 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.015 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2023-07-06T20:37:05.015 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2023-07-06T20:37:05.015 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.015 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.015 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.015 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-06T20:37:05.016 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.016 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.016 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.016 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2023-07-06T20:37:05.016 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.016 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.016 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.017 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.017 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2023-07-06T20:37:05.017 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2023-07-06T20:37:05.017 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.017 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.017 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.017 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-06T20:37:05.018 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.018 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.018 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.018 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2023-07-06T20:37:05.018 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.018 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.018 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.019 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.019 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2023-07-06T20:37:05.019 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2023-07-06T20:37:05.019 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.019 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.019 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.020 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-06T20:37:05.020 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.020 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.020 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.020 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2023-07-06T20:37:05.020 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.021 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.021 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.021 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.021 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2023-07-06T20:37:05.021 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2023-07-06T20:37:05.021 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.021 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.022 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.022 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.022 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.022 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.022 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.022 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2023-07-06T20:37:05.023 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.023 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.023 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.023 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.023 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2023-07-06T20:37:05.024 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2023-07-06T20:37:05.024 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.024 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.024 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.024 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:05.025 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.025 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.025 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.025 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-06T20:37:05.025 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.025 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.025 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.025 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.026 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2023-07-06T20:37:05.026 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2023-07-06T20:37:05.026 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.026 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.026 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.026 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2023-07-06T20:37:05.026 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.026 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.027 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.027 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-06T20:37:05.027 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.027 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.027 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.027 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.027 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2023-07-06T20:37:05.027 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2023-07-06T20:37:05.028 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.028 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.028 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.028 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2023-07-06T20:37:05.028 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.028 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.028 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.029 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-06T20:37:05.029 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.029 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.029 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.029 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.029 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2023-07-06T20:37:05.029 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2023-07-06T20:37:05.029 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.030 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.030 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.030 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2023-07-06T20:37:05.030 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.030 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.030 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.030 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2023-07-06T20:37:05.030 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.031 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.031 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.031 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.031 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2023-07-06T20:37:05.031 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2023-07-06T20:37:05.031 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.031 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.032 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.032 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.032 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.032 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.032 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.032 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-06T20:37:05.032 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.032 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.033 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.033 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.033 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2023-07-06T20:37:05.033 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2023-07-06T20:37:05.033 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.033 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.033 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.033 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:05.034 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.034 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.034 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.034 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2023-07-06T20:37:05.034 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.034 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.034 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.035 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.035 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2023-07-06T20:37:05.035 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2023-07-06T20:37:05.035 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.035 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.035 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.035 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:05.035 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.036 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.036 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.036 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2023-07-06T20:37:05.036 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.036 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.036 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.036 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.036 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.037 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.037 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.037 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.037 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.037 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.037 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.037 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.037 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.038 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.038 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.038 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.038 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.038 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.038 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.038 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.039 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.039 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.039 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.039 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.039 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.039 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2023-07-06T20:37:05.039 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2023-07-06T20:37:05.039 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.040 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.040 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.040 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.040 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.040 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.040 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.040 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-06T20:37:05.041 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.041 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.041 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.041 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.041 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.041 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.041 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.042 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.042 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.042 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.042 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.042 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.042 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.042 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-06T20:37:05.043 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.043 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.043 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.043 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.043 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.043 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.044 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.044 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.044 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.044 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.044 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.044 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.044 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.045 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.045 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.045 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.045 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.045 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.045 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.045 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.046 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.046 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.046 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.046 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.046 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.046 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.047 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.047 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.047 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.047 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.047 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.047 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.048 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.048 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.048 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.048 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.048 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.048 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.049 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-06T20:37:05.049 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-06T20:37:05.049 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.049 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.049 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.049 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.050 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.050 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.050 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.050 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-06T20:37:05.050 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.050 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.051 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.051 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-06T20:37:05.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-06T20:37:05.051 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.051 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.052 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.052 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2023-07-06T20:37:05.052 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.052 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.052 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.052 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-06T20:37:05.052 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.052 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.053 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.053 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2023-07-06T20:37:05.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2023-07-06T20:37:05.053 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.053 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.053 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.054 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:05.054 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.054 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.054 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.054 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2023-07-06T20:37:05.054 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.054 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.055 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.055 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.055 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2023-07-06T20:37:05.055 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2023-07-06T20:37:05.055 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.055 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.055 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.055 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:05.056 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.056 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.056 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.056 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-06T20:37:05.056 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.056 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.057 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.057 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2023-07-06T20:37:05.057 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2023-07-06T20:37:05.057 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.057 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.057 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.057 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.058 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.058 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.058 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.058 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2023-07-06T20:37:05.058 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.058 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.058 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.059 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.059 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2023-07-06T20:37:05.059 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2023-07-06T20:37:05.059 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.059 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.059 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.059 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2023-07-06T20:37:05.059 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.060 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.060 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.060 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2023-07-06T20:37:05.060 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2023-07-06T20:37:05.060 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.060 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2023-07-06T20:37:05.060 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.061 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.061 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.061 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.061 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2023-07-06T20:37:05.061 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2023-07-06T20:37:05.061 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.061 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.061 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.062 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-06T20:37:05.062 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.062 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.062 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.062 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-06T20:37:05.062 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.062 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.062 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.063 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.063 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_web_password": { 2023-07-06T20:37:05.063 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_web_password", 2023-07-06T20:37:05.063 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.063 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.063 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.063 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:05.063 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.064 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.064 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.064 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus web password", 2023-07-06T20:37:05.064 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.064 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.064 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.064 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.065 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_web_user": { 2023-07-06T20:37:05.065 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_web_user", 2023-07-06T20:37:05.065 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.065 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.065 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.065 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:05.065 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.065 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.066 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.066 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus web user", 2023-07-06T20:37:05.066 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.066 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.066 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.066 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.066 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2023-07-06T20:37:05.066 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2023-07-06T20:37:05.067 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.067 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.067 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.067 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.067 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.067 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.067 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.068 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-06T20:37:05.068 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.068 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.068 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.068 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2023-07-06T20:37:05.068 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2023-07-06T20:37:05.068 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.069 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.069 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.069 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.069 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.069 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.069 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.069 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-06T20:37:05.069 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.070 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.070 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.070 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2023-07-06T20:37:05.070 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2023-07-06T20:37:05.070 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.070 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.071 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.071 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.071 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.071 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.071 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.071 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-06T20:37:05.071 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.071 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.072 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.072 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.072 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2023-07-06T20:37:05.072 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2023-07-06T20:37:05.072 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.073 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.073 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.073 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.073 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.073 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.073 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.073 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-06T20:37:05.073 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.074 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.074 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.074 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.074 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2023-07-06T20:37:05.074 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2023-07-06T20:37:05.074 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.075 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.075 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.075 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.075 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.075 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.075 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.075 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-06T20:37:05.076 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.076 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.076 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.076 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.076 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2023-07-06T20:37:05.076 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2023-07-06T20:37:05.077 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.077 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.077 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.077 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2023-07-06T20:37:05.077 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.077 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.078 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.078 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2023-07-06T20:37:05.078 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.078 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.078 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.078 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.078 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2023-07-06T20:37:05.079 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2023-07-06T20:37:05.079 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.079 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.079 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.079 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.079 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.080 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.080 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.080 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-06T20:37:05.080 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.080 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.081 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.081 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.081 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2023-07-06T20:37:05.081 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2023-07-06T20:37:05.081 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.082 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.082 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.082 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.082 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.082 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.082 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.082 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-06T20:37:05.083 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.083 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.083 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.083 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2023-07-06T20:37:05.083 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2023-07-06T20:37:05.083 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.083 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.084 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.084 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.084 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.084 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.084 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.084 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-06T20:37:05.084 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.084 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.085 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.085 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.085 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2023-07-06T20:37:05.085 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2023-07-06T20:37:05.085 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.085 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.085 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.085 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.086 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.086 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.086 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.086 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2023-07-06T20:37:05.086 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.086 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.086 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.087 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.087 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2023-07-06T20:37:05.087 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2023-07-06T20:37:05.087 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.087 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.087 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.087 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.087 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.088 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.088 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.088 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-06T20:37:05.088 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.088 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.088 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.088 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.088 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2023-07-06T20:37:05.089 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2023-07-06T20:37:05.089 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.089 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.089 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.089 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.089 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.089 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.089 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.090 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-06T20:37:05.090 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.090 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.090 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.090 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.090 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.090 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.091 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.091 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2023-07-06T20:37:05.091 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.091 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.091 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.091 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.091 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.091 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.092 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.092 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.092 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.092 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.092 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.092 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.092 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.092 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.093 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.093 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.093 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.093 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.093 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.093 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.093 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.094 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.094 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.094 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.094 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.094 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.094 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.094 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.094 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.095 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.095 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.095 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.095 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.095 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.095 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.095 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.096 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.096 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.096 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.096 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.096 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.096 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.096 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.096 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.097 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.097 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.097 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.097 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.097 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.097 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.097 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.097 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.098 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.098 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.098 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.098 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.098 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.098 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.098 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.099 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.099 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.099 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.099 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.099 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.099 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.100 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.100 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.100 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.100 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.100 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.100 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.100 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.100 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.101 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2023-07-06T20:37:05.101 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2023-07-06T20:37:05.101 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.101 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.101 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.102 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2023-07-06T20:37:05.102 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.102 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.102 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-06T20:37:05.102 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.102 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.103 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.103 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.103 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2023-07-06T20:37:05.103 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2023-07-06T20:37:05.103 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.103 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.103 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.104 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2023-07-06T20:37:05.104 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.104 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.104 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.104 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-06T20:37:05.104 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.104 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.105 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.105 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.105 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.105 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.105 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.105 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2023-07-06T20:37:05.106 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.106 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.106 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.106 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-06T20:37:05.106 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-06T20:37:05.106 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.107 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.107 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.107 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:05.107 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.107 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.108 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.108 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.108 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.108 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.108 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.109 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2023-07-06T20:37:05.109 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-06T20:37:05.109 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.109 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.109 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.109 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.110 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.110 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.110 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.110 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.110 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.110 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.110 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.110 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.111 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-06T20:37:05.111 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-06T20:37:05.111 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.111 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.111 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.111 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.111 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.112 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.112 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.112 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.112 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.112 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.112 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.112 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.113 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2023-07-06T20:37:05.113 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2023-07-06T20:37:05.113 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.113 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.113 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.113 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.113 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.113 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.114 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.114 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.114 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.114 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.114 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.114 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.114 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-06T20:37:05.115 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-06T20:37:05.115 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.115 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.115 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.115 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.115 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.115 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.115 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.116 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.116 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.116 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.116 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.116 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.116 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2023-07-06T20:37:05.116 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-06T20:37:05.116 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.117 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.117 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.117 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.117 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.117 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.117 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.117 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.117 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.118 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.118 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.118 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.118 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-06T20:37:05.118 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-06T20:37:05.118 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.118 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.119 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.119 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.119 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.119 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.119 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.119 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.119 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.119 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.120 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.120 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.120 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-06T20:37:05.120 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-06T20:37:05.120 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.120 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.120 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.120 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.121 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.121 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.121 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.121 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.121 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.121 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.121 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.122 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.122 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-06T20:37:05.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-06T20:37:05.122 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.122 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.122 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.123 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.123 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.123 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.123 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.123 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.123 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.123 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.124 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.124 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.124 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-06T20:37:05.124 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-06T20:37:05.124 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.124 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.125 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.125 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.125 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.125 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.125 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.125 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.126 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.126 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.126 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.126 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.127 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-06T20:37:05.127 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-06T20:37:05.127 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.127 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.127 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.127 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.127 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.127 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.128 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.128 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.128 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.128 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.128 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.128 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.128 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-06T20:37:05.128 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-06T20:37:05.129 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.129 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.129 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.129 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.129 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.129 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.129 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.130 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.130 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.130 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.130 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.130 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.130 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-06T20:37:05.130 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-06T20:37:05.130 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.131 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.131 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.131 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.131 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.131 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.131 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.131 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.132 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.132 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.132 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.132 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.132 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-06T20:37:05.132 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-06T20:37:05.132 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.132 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.133 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.133 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.133 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.133 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.133 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.133 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.133 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.134 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.134 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.134 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.134 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2023-07-06T20:37:05.134 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-06T20:37:05.134 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.135 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.135 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.135 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:05.135 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.135 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.135 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.136 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.136 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.136 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.136 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.136 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.136 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-06T20:37:05.136 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-06T20:37:05.137 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.137 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.137 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.137 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.137 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.138 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.138 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.138 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.138 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.138 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.138 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.139 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.139 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2023-07-06T20:37:05.139 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2023-07-06T20:37:05.139 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.139 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.140 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.140 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.140 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.140 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.140 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.140 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.140 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.141 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.141 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.141 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.141 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2023-07-06T20:37:05.141 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-06T20:37:05.141 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.141 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.141 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.142 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:05.142 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.142 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.142 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.142 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.142 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.142 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.142 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.143 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.143 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-06T20:37:05.143 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-06T20:37:05.143 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.143 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.143 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.143 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.144 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.144 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.144 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.144 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.144 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.144 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.144 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.145 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.145 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-06T20:37:05.145 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-06T20:37:05.145 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.145 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.145 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.145 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.145 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.146 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.146 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.146 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.146 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.146 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.146 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.146 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.147 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-06T20:37:05.147 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-06T20:37:05.147 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.147 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.147 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.147 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.147 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.147 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.148 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.148 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.148 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.148 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.148 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.148 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.148 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-06T20:37:05.148 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-06T20:37:05.149 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.149 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.149 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.149 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.149 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.149 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.149 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.150 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.150 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.150 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.150 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.150 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.150 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2023-07-06T20:37:05.150 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-06T20:37:05.150 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.151 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.151 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.151 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.151 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.151 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.151 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.151 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.152 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.152 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.152 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.152 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.152 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-06T20:37:05.152 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-06T20:37:05.152 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.152 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.153 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.153 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.153 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.153 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.153 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.153 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.153 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.153 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.154 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.154 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.154 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-06T20:37:05.154 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-06T20:37:05.154 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.154 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.154 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.154 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.155 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.155 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.155 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.155 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.155 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.155 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.155 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.156 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.156 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-06T20:37:05.156 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-06T20:37:05.156 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.156 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.156 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.156 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.157 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.157 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.157 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.157 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.157 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.157 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.157 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.158 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.158 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-06T20:37:05.158 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-06T20:37:05.158 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.158 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.158 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.158 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.158 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.159 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.159 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.159 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.159 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.159 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.159 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.160 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.160 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-06T20:37:05.160 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-06T20:37:05.160 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.160 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.160 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.161 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.161 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.161 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.161 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.161 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.161 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.161 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.162 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.162 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.162 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-06T20:37:05.162 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-06T20:37:05.162 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.162 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.163 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.163 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.163 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.163 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.163 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.164 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.164 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.164 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.164 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.164 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.164 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-06T20:37:05.165 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-06T20:37:05.165 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.165 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.165 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.165 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.166 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.166 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.166 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.166 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.166 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.166 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.167 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.167 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.167 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-06T20:37:05.167 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-06T20:37:05.167 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.167 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.167 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.168 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.168 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.168 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.168 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.168 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.168 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.168 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.168 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.169 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.169 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2023-07-06T20:37:05.169 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-06T20:37:05.169 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.169 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.169 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.169 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.170 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.170 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.170 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.170 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.170 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.170 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.170 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.171 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.171 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-06T20:37:05.171 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-06T20:37:05.171 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.171 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.171 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.171 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-06T20:37:05.171 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.172 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.172 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.172 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.172 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.172 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.172 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.172 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.173 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-06T20:37:05.173 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-06T20:37:05.173 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.173 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.173 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.173 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:05.173 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.174 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.174 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.174 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.174 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.174 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.174 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.174 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.175 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-06T20:37:05.175 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-06T20:37:05.175 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.175 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.175 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.175 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2023-07-06T20:37:05.175 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.175 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.176 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.176 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.176 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.176 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.176 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.176 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.176 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-06T20:37:05.177 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-06T20:37:05.177 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.177 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.177 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.177 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2023-07-06T20:37:05.177 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.177 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.177 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.178 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.178 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.178 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.178 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.178 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.178 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2023-07-06T20:37:05.178 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-06T20:37:05.178 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.179 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.179 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.179 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.179 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.179 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.179 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.179 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.179 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.180 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.180 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.180 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.180 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-06T20:37:05.180 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-06T20:37:05.180 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.180 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.181 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.181 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2023-07-06T20:37:05.181 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.181 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.181 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.181 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.181 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.181 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.182 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.182 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.182 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2023-07-06T20:37:05.182 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-06T20:37:05.182 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.182 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.182 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.182 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.183 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.183 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.183 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.183 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.183 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.183 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.183 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.183 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.184 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2023-07-06T20:37:05.184 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-06T20:37:05.184 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.184 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.184 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.185 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.185 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.185 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.185 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.185 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.185 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.185 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.185 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.186 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.186 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-06T20:37:05.186 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-06T20:37:05.186 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.186 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.186 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.187 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:05.187 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.187 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.187 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.187 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.187 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.187 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.187 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.188 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.188 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-06T20:37:05.188 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-06T20:37:05.188 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.188 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.188 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.189 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:05.189 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.189 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.189 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.189 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.189 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.189 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.189 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-06T20:37:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-06T20:37:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2023-07-06T20:37:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.191 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.191 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.191 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.191 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.191 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.191 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.191 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.192 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2023-07-06T20:37:05.192 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2023-07-06T20:37:05.192 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.192 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.192 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.192 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.192 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.193 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.193 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.193 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.193 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.193 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.193 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.194 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.194 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2023-07-06T20:37:05.194 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2023-07-06T20:37:05.194 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.194 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.194 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.194 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.195 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.195 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.195 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.195 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.195 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.196 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.196 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.196 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.196 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2023-07-06T20:37:05.196 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2023-07-06T20:37:05.196 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.196 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.197 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.197 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.197 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.197 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.197 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.198 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2023-07-06T20:37:05.198 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.198 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.198 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.198 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.198 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2023-07-06T20:37:05.199 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2023-07-06T20:37:05.199 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.199 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.199 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.199 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2023-07-06T20:37:05.199 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.199 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.199 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.200 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.200 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.200 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.200 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.200 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.200 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2023-07-06T20:37:05.200 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2023-07-06T20:37:05.200 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.201 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.201 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.201 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.201 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.201 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.201 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.201 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.201 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.202 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.202 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.202 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.202 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.202 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.202 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.202 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.202 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.203 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.203 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.203 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.203 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.203 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.203 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.203 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.204 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.204 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.204 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.204 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.204 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.204 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.204 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.204 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.205 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.205 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.205 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.205 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.205 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.205 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.205 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.206 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.206 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.206 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.206 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.206 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.206 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.206 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.206 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.207 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.207 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.207 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.207 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.207 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.207 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.207 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.207 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.208 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.208 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.208 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.208 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.208 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.208 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.208 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.208 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.209 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.209 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.209 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.209 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.209 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.209 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.209 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.209 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.210 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.210 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.210 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.210 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.210 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.210 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.210 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.210 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.211 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.211 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.211 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.211 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2023-07-06T20:37:05.211 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2023-07-06T20:37:05.211 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.211 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.212 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.212 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.212 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.212 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.212 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.212 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2023-07-06T20:37:05.212 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.212 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.213 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.213 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.213 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2023-07-06T20:37:05.213 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2023-07-06T20:37:05.213 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.213 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.213 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.213 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.214 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.214 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.214 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.214 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.214 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.214 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.215 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.215 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.215 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2023-07-06T20:37:05.215 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2023-07-06T20:37:05.215 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.215 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.215 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.215 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2023-07-06T20:37:05.216 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.216 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.216 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.216 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.216 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.217 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.217 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.217 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.217 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2023-07-06T20:37:05.217 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2023-07-06T20:37:05.217 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.217 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.217 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.218 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2023-07-06T20:37:05.218 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.218 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.218 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.218 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.218 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.219 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.219 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.219 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.219 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2023-07-06T20:37:05.219 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2023-07-06T20:37:05.219 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.219 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.220 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.220 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.220 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.220 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.220 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.220 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.220 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.220 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.221 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.221 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2023-07-06T20:37:05.221 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2023-07-06T20:37:05.221 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.221 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.221 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.221 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2023-07-06T20:37:05.222 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.222 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.222 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.222 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.222 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.222 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.222 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.223 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.223 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2023-07-06T20:37:05.223 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2023-07-06T20:37:05.223 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.223 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.223 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.223 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2023-07-06T20:37:05.223 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.224 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.224 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.224 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.224 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2023-07-06T20:37:05.224 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.224 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.224 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.225 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.225 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.225 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2023-07-06T20:37:05.225 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2023-07-06T20:37:05.225 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.225 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.226 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.226 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2023-07-06T20:37:05.226 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2023-07-06T20:37:05.226 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2023-07-06T20:37:05.226 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.226 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.226 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.227 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.227 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.227 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.227 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2023-07-06T20:37:05.227 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2023-07-06T20:37:05.227 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.227 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.227 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.228 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.228 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.228 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.228 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.228 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.228 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.229 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.229 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.229 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.229 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.229 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.229 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.229 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2023-07-06T20:37:05.230 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.230 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.230 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.230 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2023-07-06T20:37:05.230 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2023-07-06T20:37:05.230 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.230 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.230 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.231 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.231 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.231 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.231 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.231 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2023-07-06T20:37:05.231 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.231 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.232 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.232 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.232 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.232 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.232 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.232 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.232 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.232 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.233 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.233 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.233 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.233 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.233 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.233 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.233 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.234 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.234 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.234 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.234 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.234 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.234 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.234 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.234 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.235 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.235 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.235 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.235 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.235 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.235 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.235 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.235 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.236 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.236 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.236 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.236 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.236 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.236 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.237 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.237 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.237 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.237 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.237 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.237 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.237 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.237 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.238 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.238 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.238 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.238 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.238 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.238 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.238 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.238 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.239 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.239 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.239 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.239 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.239 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.239 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.239 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.239 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.240 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.240 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.240 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.240 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.240 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.240 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.240 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.240 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.241 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.241 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.241 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2023-07-06T20:37:05.241 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2023-07-06T20:37:05.241 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.241 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.241 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.242 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2023-07-06T20:37:05.242 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.242 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.242 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.242 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-06T20:37:05.242 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.242 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.242 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.243 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.243 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2023-07-06T20:37:05.243 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2023-07-06T20:37:05.243 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.243 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.243 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.243 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2023-07-06T20:37:05.243 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.244 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.244 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.244 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-06T20:37:05.244 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.244 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.244 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.244 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.244 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2023-07-06T20:37:05.245 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2023-07-06T20:37:05.245 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.245 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.245 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.245 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2023-07-06T20:37:05.245 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.245 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.245 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.246 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2023-07-06T20:37:05.246 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.246 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.246 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.246 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.246 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2023-07-06T20:37:05.246 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2023-07-06T20:37:05.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.247 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.247 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.247 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2023-07-06T20:37:05.247 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.247 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.247 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.247 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-06T20:37:05.248 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.248 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.248 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.248 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.248 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2023-07-06T20:37:05.248 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2023-07-06T20:37:05.248 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.248 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.249 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.249 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.249 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.249 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.249 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.249 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-06T20:37:05.249 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.249 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.250 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.250 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.250 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:05.250 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:05.250 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.250 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.250 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.250 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:05.251 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.251 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.251 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.251 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2023-07-06T20:37:05.251 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.251 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.251 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.252 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.252 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2023-07-06T20:37:05.252 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2023-07-06T20:37:05.252 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.252 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.252 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.252 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2023-07-06T20:37:05.252 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.253 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.253 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.253 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-06T20:37:05.253 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.253 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.253 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.253 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.254 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.254 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.254 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.254 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2023-07-06T20:37:05.254 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.254 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.254 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.254 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.255 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.255 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.255 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.255 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.255 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.255 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.255 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.255 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.256 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.256 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.256 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.256 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.256 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.256 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.256 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.257 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.257 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.257 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.257 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.257 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.257 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.257 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.257 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2023-07-06T20:37:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2023-07-06T20:37:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2023-07-06T20:37:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2023-07-06T20:37:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2023-07-06T20:37:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2023-07-06T20:37:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2023-07-06T20:37:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2023-07-06T20:37:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2023-07-06T20:37:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2023-07-06T20:37:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2023-07-06T20:37:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2023-07-06T20:37:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2023-07-06T20:37:05.271 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.271 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.271 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.271 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-06T20:37:05.271 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.271 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.271 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.271 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.272 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2023-07-06T20:37:05.272 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2023-07-06T20:37:05.272 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.272 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.272 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.272 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2023-07-06T20:37:05.272 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.272 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.273 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.273 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-06T20:37:05.273 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.273 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.273 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.273 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.273 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2023-07-06T20:37:05.273 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2023-07-06T20:37:05.274 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.274 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.274 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.274 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.274 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.274 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.274 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.275 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2023-07-06T20:37:05.275 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.275 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.275 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.275 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.275 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:05.275 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:05.275 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.276 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.276 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.276 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2023-07-06T20:37:05.276 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2023-07-06T20:37:05.276 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.276 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.276 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-06T20:37:05.276 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.277 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.277 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.277 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.277 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.277 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.277 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.277 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.278 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.278 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.278 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.278 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.278 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.278 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.278 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.279 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.279 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.279 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.279 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.279 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.279 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.279 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.280 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.280 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.280 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.280 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.280 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.280 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.280 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.280 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.281 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.281 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.281 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.281 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.281 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.281 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.281 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.282 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.282 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.282 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.282 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.282 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.282 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.282 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.282 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.283 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.283 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.283 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.283 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.283 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.283 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.283 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.283 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.284 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.284 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.284 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.284 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.284 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.284 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.284 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.284 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.285 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.285 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.285 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.285 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.285 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.285 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.285 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.286 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.286 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.286 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.286 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.286 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2023-07-06T20:37:05.286 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2023-07-06T20:37:05.286 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.286 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.287 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.287 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.287 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.287 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.287 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.287 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2023-07-06T20:37:05.287 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.287 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.288 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.288 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2023-07-06T20:37:05.288 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2023-07-06T20:37:05.288 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.288 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.288 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.288 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2023-07-06T20:37:05.289 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.289 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.289 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.289 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2023-07-06T20:37:05.289 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.290 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.290 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.290 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.290 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2023-07-06T20:37:05.290 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2023-07-06T20:37:05.290 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.290 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.290 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.291 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2023-07-06T20:37:05.291 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.291 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.291 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.291 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-06T20:37:05.291 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.291 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.291 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.292 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2023-07-06T20:37:05.292 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2023-07-06T20:37:05.292 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.292 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.292 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.292 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2023-07-06T20:37:05.293 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2023-07-06T20:37:05.293 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2023-07-06T20:37:05.293 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.293 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-06T20:37:05.293 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.293 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.293 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.293 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.294 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2023-07-06T20:37:05.294 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2023-07-06T20:37:05.294 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.294 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.294 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.294 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.294 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.294 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.295 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.295 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2023-07-06T20:37:05.295 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.295 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.295 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.295 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.295 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2023-07-06T20:37:05.295 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2023-07-06T20:37:05.296 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.296 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.296 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2023-07-06T20:37:05.296 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.296 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.296 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.297 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-06T20:37:05.297 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.297 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.297 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.297 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.297 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.297 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.297 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.298 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2023-07-06T20:37:05.298 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.298 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.298 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.298 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.298 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.298 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.300 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.300 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.300 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.300 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.300 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.300 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.300 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.300 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.301 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.301 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.301 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.301 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.301 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.301 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.301 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.302 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.302 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.302 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.302 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.302 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.302 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.302 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.303 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.303 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.303 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.303 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.303 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.303 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.303 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.303 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.304 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.304 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.304 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.304 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.304 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.304 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.304 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.305 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.305 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.305 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.305 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.305 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.305 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.305 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.305 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.306 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.306 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.306 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.306 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.306 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.306 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.306 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.306 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.307 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.307 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.307 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.307 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.307 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.307 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.307 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.308 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.308 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2023-07-06T20:37:05.308 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.308 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.308 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.308 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.308 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.308 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.309 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.309 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.309 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.309 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.309 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.309 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.309 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.310 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.310 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.310 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.310 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.310 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.310 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.310 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.310 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.311 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.311 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.311 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.311 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.311 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.311 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.311 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.312 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.312 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.312 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.312 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.312 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.312 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.312 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.313 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.313 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.313 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.313 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.313 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.313 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.313 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.313 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.314 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.314 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.314 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.314 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.314 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.314 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.314 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.314 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.315 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.315 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.315 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.315 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.315 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.315 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.315 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.315 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.316 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.316 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.316 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.316 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.316 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.316 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.316 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.317 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.317 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.317 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.317 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.317 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.317 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.317 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.317 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.318 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.318 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2023-07-06T20:37:05.318 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.318 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.318 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.318 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2023-07-06T20:37:05.318 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2023-07-06T20:37:05.318 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.319 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.319 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.319 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2023-07-06T20:37:05.319 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.319 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.319 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.319 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-06T20:37:05.319 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.320 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.320 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.320 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.320 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2023-07-06T20:37:05.320 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2023-07-06T20:37:05.320 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.320 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.321 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.321 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:05.321 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2023-07-06T20:37:05.321 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.321 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.321 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-06T20:37:05.321 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.322 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.322 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.322 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.322 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.322 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.322 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.323 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.323 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.323 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.323 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.323 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.323 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.323 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.324 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.324 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.324 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.324 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.324 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.324 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.324 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.324 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.325 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.325 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.325 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.325 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.325 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.325 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.325 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.325 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.326 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.326 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.326 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.326 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.326 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.326 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.326 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.327 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.327 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.327 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.327 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.327 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.327 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.327 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.327 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.328 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.328 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.328 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.328 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.328 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.328 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.328 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.328 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.329 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.329 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.329 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.329 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.329 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.329 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.329 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.329 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.330 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.330 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.330 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.330 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.330 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.330 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.330 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.330 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.331 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.331 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.331 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.331 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.331 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.331 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2023-07-06T20:37:05.331 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.332 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.332 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.332 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2023-07-06T20:37:05.332 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2023-07-06T20:37:05.332 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.332 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.332 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.332 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2023-07-06T20:37:05.333 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.333 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.333 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.333 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2023-07-06T20:37:05.333 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-06T20:37:05.333 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.333 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.333 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.334 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.334 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.334 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.334 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.334 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.334 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.334 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.334 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.335 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.335 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.335 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.335 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.335 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.335 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.335 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.335 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.336 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.336 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.336 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.336 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.336 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.336 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.336 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.337 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.337 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.337 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.337 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.337 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.337 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.337 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.337 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.338 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.338 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.338 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.338 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.338 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.338 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.338 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.338 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.339 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.339 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.339 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.339 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.339 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.339 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.339 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.339 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.340 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.340 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.340 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.340 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.340 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.340 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.340 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.341 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.341 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.341 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.341 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.341 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.341 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.341 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.341 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.342 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.342 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.342 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.342 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.342 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.342 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.342 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.342 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.343 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2023-07-06T20:37:05.343 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2023-07-06T20:37:05.343 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.343 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.343 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.343 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.343 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.343 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.344 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.344 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2023-07-06T20:37:05.344 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-06T20:37:05.344 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.344 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.344 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.344 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2023-07-06T20:37:05.345 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2023-07-06T20:37:05.345 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.345 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.345 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.345 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2023-07-06T20:37:05.345 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.345 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.345 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.346 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2023-07-06T20:37:05.346 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.346 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.346 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.346 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.347 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2023-07-06T20:37:05.347 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2023-07-06T20:37:05.347 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.347 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.347 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.348 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2023-07-06T20:37:05.348 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.348 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.348 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.348 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2023-07-06T20:37:05.348 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.348 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.349 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.349 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.349 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2023-07-06T20:37:05.349 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2023-07-06T20:37:05.349 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.349 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.349 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.349 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.350 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.350 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.350 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.350 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2023-07-06T20:37:05.350 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.350 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.350 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.351 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.351 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2023-07-06T20:37:05.351 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2023-07-06T20:37:05.351 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.351 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.351 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.351 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2023-07-06T20:37:05.352 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.352 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.352 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.352 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-06T20:37:05.352 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-06T20:37:05.352 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.352 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.353 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.353 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.353 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.353 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.353 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2023-07-06T20:37:05.353 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.353 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.354 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.354 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.354 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.354 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.354 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.355 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.355 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.355 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.355 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.355 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.355 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.355 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.356 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.356 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.356 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.356 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.356 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.356 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.356 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.357 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.357 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.357 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.357 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.357 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.357 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.358 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.358 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.358 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.358 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.358 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.358 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.358 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.359 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.359 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.359 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.359 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.359 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.359 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.359 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.359 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.360 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.360 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.360 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.360 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.360 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.360 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.360 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.361 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.361 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.361 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.361 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.361 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.361 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.361 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.361 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.362 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.362 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.362 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.362 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.362 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.362 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.362 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.362 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.363 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.363 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.363 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.363 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.363 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.363 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.363 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.364 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.364 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.364 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.364 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.364 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2023-07-06T20:37:05.365 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.365 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.365 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.365 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.365 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.365 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.365 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.366 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.366 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.366 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.366 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.366 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.366 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.366 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.366 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.367 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.367 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.367 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.367 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.367 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.367 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.367 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.368 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.368 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.368 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.368 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.368 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.368 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.369 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.369 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.369 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.369 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.369 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.369 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.369 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.370 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.370 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.370 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.370 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.370 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.370 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.370 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.371 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.371 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.371 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.371 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.371 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.371 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.371 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.371 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.372 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.372 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.372 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.372 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.372 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.372 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.373 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.373 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.373 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.373 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.373 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.373 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.373 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.373 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.374 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.374 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.374 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.374 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.374 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.374 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.374 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.374 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.375 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.375 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.375 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.375 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.375 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2023-07-06T20:37:05.376 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.376 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.376 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.376 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.376 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.376 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.376 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.377 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.377 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.377 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.377 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.377 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.377 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.377 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.378 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.378 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.378 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.378 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.378 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.378 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.379 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.379 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.379 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.379 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.379 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.379 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.379 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.379 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.380 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.380 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.380 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.380 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.380 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.380 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.380 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.381 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.381 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.381 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.381 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.381 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.381 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.381 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.381 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.382 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.382 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.382 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.382 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.382 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.382 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.382 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.383 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.383 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.383 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.383 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.383 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.383 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.383 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.384 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.384 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.384 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.384 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.384 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.384 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.384 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.384 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.385 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.385 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.385 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.385 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.386 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.386 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.386 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.386 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.386 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.386 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.386 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.387 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2023-07-06T20:37:05.387 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.387 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.387 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.387 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2023-07-06T20:37:05.387 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2023-07-06T20:37:05.387 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.387 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.388 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.388 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.388 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.388 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.388 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.388 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-06T20:37:05.388 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.389 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.389 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.389 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.389 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.389 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.390 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.390 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.390 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.390 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.390 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.391 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.391 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.391 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.391 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.391 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.391 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.391 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.392 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.392 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.392 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.392 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.392 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.392 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.392 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.393 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.393 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.393 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.393 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.393 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.393 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.393 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.393 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.394 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.394 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.394 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.394 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.394 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.394 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.394 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.394 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.395 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.395 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.395 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.395 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.395 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.395 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.395 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.396 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.396 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.396 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.396 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.396 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.396 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.396 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.396 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.397 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.397 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.397 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.397 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.397 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.397 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.397 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.398 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.398 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.398 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.398 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.398 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.398 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.398 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.398 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.399 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.399 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.399 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.399 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2023-07-06T20:37:05.399 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2023-07-06T20:37:05.399 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.399 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.400 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.400 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.400 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.400 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.400 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.400 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2023-07-06T20:37:05.400 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2023-07-06T20:37:05.401 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2023-07-06T20:37:05.401 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.401 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2023-07-06T20:37:05.401 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.401 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.401 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.401 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.401 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.402 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.402 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.402 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2023-07-06T20:37:05.402 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.402 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.402 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.402 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.403 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.403 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.403 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.403 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.403 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.403 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.403 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.403 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.404 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.404 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.404 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.404 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.404 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.404 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.404 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.405 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.405 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.405 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.405 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.405 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.405 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.405 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.405 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.406 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.406 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.406 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.406 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.406 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.406 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.406 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.406 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.407 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.407 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.407 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.407 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.407 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.407 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.407 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.407 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.408 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.408 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.408 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.408 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.408 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.408 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.409 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.409 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.409 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.409 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.410 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.410 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.410 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.410 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.411 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.411 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.411 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.412 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.412 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.412 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.412 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.413 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.413 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.413 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.413 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.414 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.414 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.414 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.415 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.415 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.415 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.415 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.415 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.415 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2023-07-06T20:37:05.416 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.416 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.416 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.416 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.416 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.416 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.417 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.417 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.417 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.417 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.417 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.417 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.417 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.418 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.418 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.418 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.418 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.418 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.418 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.418 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.419 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.419 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.419 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.419 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.419 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.419 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.420 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.420 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.420 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.420 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.420 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.421 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.421 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.421 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.421 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.421 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.422 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.422 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.422 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.422 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.422 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.422 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.423 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.423 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.423 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.423 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.423 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.423 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.424 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.424 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.424 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.424 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.424 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.425 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.425 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.425 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.425 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.425 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.425 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.425 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.426 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.426 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.426 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.426 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.426 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.426 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.426 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.426 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.427 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.427 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.427 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2023-07-06T20:37:05.427 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.427 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.427 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.427 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.428 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.428 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.428 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.428 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.428 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.428 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.428 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.429 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.429 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.429 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.429 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.429 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.429 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.429 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.430 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.430 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.430 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.430 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.430 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.430 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.430 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.430 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.431 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.431 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.431 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.431 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.431 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.431 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.432 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.432 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.432 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.432 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.432 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.432 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.432 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.432 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.433 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.433 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.433 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.433 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.433 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.433 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.433 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.434 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.434 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.434 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.434 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.434 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.434 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.434 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.435 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.435 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.435 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.435 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.435 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.435 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.435 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.435 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: "noautoscale": { 2023-07-06T20:37:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "noautoscale", 2023-07-06T20:37:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.437 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.438 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.438 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.438 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.438 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.438 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.438 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "global autoscale flag", 2023-07-06T20:37:05.438 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-06T20:37:05.439 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.439 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.439 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.439 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:05.439 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:05.439 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.439 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.440 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.440 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:05.440 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.440 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.440 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.440 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.440 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.441 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.441 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.441 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.441 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2023-07-06T20:37:05.441 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2023-07-06T20:37:05.441 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:05.441 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.442 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.442 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2023-07-06T20:37:05.442 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2023-07-06T20:37:05.442 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.442 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.442 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2023-07-06T20:37:05.442 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-06T20:37:05.443 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.443 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.443 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.443 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.443 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.443 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.443 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2023-07-06T20:37:05.444 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.444 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.444 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.444 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2023-07-06T20:37:05.444 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2023-07-06T20:37:05.444 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.444 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.445 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.445 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.445 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.445 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.445 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.445 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-06T20:37:05.445 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.446 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.446 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.446 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.446 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2023-07-06T20:37:05.446 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2023-07-06T20:37:05.446 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.446 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.447 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.447 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.447 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.447 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.447 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.447 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.447 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.447 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.448 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.448 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.448 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.448 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.448 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.448 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.448 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.449 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.449 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.449 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.449 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.449 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.449 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.449 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.449 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.450 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.450 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.450 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.450 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.450 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.450 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.450 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.450 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.451 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.451 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.451 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.451 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.451 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.451 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.452 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.452 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.452 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.452 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.452 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.452 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.452 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.453 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.453 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.453 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.453 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.453 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.453 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.454 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.454 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.454 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.454 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.454 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.454 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.454 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.454 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.455 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.455 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.455 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.455 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.455 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.455 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.455 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.456 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.456 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.456 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.456 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.456 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.456 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.456 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.456 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.457 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.457 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.457 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.457 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.457 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.457 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2023-07-06T20:37:05.458 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2023-07-06T20:37:05.458 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.458 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.458 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.459 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2023-07-06T20:37:05.459 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.459 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.460 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.460 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2023-07-06T20:37:05.460 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.460 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.461 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.461 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2023-07-06T20:37:05.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2023-07-06T20:37:05.462 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.462 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.462 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.462 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2023-07-06T20:37:05.463 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.463 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.463 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.463 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2023-07-06T20:37:05.464 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.464 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.464 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.464 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.465 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.465 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.465 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.465 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2023-07-06T20:37:05.465 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.465 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.465 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.465 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2023-07-06T20:37:05.466 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2023-07-06T20:37:05.466 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.466 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.466 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.466 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.466 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.466 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.466 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.467 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.467 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.467 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.467 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.467 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2023-07-06T20:37:05.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2023-07-06T20:37:05.467 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.468 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.468 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.468 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.468 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.468 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.468 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-06T20:37:05.469 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-06T20:37:05.469 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.469 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.469 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.469 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.469 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.470 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.470 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.470 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.470 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.470 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.470 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.470 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.470 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.471 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.471 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.471 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.471 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.471 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.471 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.471 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.471 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.472 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.472 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.472 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.472 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.472 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.472 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.472 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.473 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.473 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.473 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.473 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.473 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.473 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.473 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.474 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.474 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.474 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.474 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.474 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.474 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.474 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.474 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.475 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.475 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.475 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.475 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.475 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.475 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.475 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.476 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.476 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.476 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.476 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.476 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.477 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.477 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.477 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.477 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.477 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.477 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.477 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.477 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.478 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.478 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.478 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.478 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.478 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.478 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.479 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2023-07-06T20:37:05.479 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2023-07-06T20:37:05.479 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.479 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.479 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.479 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.479 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.480 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.480 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.480 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.480 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.480 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.480 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.480 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-06T20:37:05.481 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-06T20:37:05.481 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.481 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.481 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.481 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2023-07-06T20:37:05.481 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.481 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.482 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.482 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.482 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.482 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.482 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.482 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.482 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2023-07-06T20:37:05.483 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2023-07-06T20:37:05.483 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:05.483 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.483 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.483 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2023-07-06T20:37:05.483 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.483 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.484 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.484 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.484 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.484 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.484 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.484 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2023-07-06T20:37:05.485 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2023-07-06T20:37:05.485 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.485 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.485 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.485 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2023-07-06T20:37:05.485 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.485 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.485 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.486 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-06T20:37:05.486 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.486 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.486 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.486 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.486 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2023-07-06T20:37:05.486 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2023-07-06T20:37:05.486 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.487 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.487 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.487 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2023-07-06T20:37:05.487 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.487 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.487 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.487 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-06T20:37:05.488 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.488 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.488 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.488 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.488 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2023-07-06T20:37:05.488 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2023-07-06T20:37:05.488 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.489 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.489 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.489 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2023-07-06T20:37:05.489 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.489 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.489 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.490 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.490 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.490 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.490 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.490 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.490 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2023-07-06T20:37:05.490 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2023-07-06T20:37:05.490 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.491 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.491 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.491 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2023-07-06T20:37:05.491 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.491 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.491 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.492 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2023-07-06T20:37:05.492 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2023-07-06T20:37:05.492 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.492 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.492 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.492 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.493 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.493 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.493 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2023-07-06T20:37:05.493 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2023-07-06T20:37:05.493 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.493 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.493 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.494 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2023-07-06T20:37:05.494 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2023-07-06T20:37:05.494 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2023-07-06T20:37:05.494 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.494 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.494 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.494 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.494 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.495 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.495 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.495 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.495 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.495 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2023-07-06T20:37:05.495 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.495 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.496 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.496 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.496 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.496 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.496 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.496 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.496 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.496 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.497 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.497 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.497 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.497 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.497 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.497 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.497 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.498 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.498 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.498 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.498 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.498 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.498 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.498 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.498 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.499 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.499 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.499 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.499 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.499 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.499 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.499 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.499 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.500 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.500 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.500 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.500 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.500 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.500 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.500 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.501 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.501 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.501 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.501 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.501 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.501 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.501 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.501 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.502 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.502 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.502 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.502 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.502 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.502 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.502 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.503 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.503 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.503 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.503 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.503 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.503 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.503 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.503 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.504 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.504 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.504 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.504 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.504 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.504 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.504 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.505 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.505 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.505 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.505 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2023-07-06T20:37:05.505 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2023-07-06T20:37:05.505 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.505 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.505 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.506 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2023-07-06T20:37:05.506 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.506 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.506 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.506 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.506 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.506 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.506 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.507 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.507 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2023-07-06T20:37:05.507 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2023-07-06T20:37:05.507 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.507 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.507 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.507 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.508 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.508 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.508 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.508 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.508 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.508 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.508 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.509 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2023-07-06T20:37:05.509 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2023-07-06T20:37:05.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.509 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.509 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.509 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.509 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.510 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.510 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.510 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.510 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.510 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.510 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.510 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.511 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.511 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2023-07-06T20:37:05.511 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.511 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.511 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.511 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2023-07-06T20:37:05.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2023-07-06T20:37:05.512 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.512 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.512 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.512 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.512 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.512 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.512 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.513 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.513 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.513 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.513 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.513 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.513 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2023-07-06T20:37:05.513 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2023-07-06T20:37:05.513 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.514 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.514 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.514 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.514 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.514 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.514 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.514 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.514 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.515 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.515 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.515 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.515 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.515 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.515 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.516 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.516 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.516 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.516 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.516 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.516 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.516 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.516 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.517 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.517 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.517 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.517 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.517 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.517 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.517 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.518 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.518 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.518 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.518 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.518 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.518 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.519 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.519 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.519 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.519 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.519 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.519 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.519 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.519 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.520 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.520 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.520 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.520 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.520 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.520 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.521 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.521 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.521 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.521 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.521 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.521 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.522 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.522 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.522 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.522 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.522 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.522 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.522 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.523 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.523 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.523 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.523 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.523 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.523 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.523 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.523 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.524 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.524 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.524 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.524 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.524 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.524 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.525 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.525 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.525 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2023-07-06T20:37:05.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2023-07-06T20:37:05.525 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.525 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.525 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.525 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.526 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.526 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.526 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.526 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.526 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.526 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.526 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.527 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.527 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2023-07-06T20:37:05.527 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2023-07-06T20:37:05.527 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.527 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.527 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.527 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.527 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.528 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.528 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.528 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.528 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.528 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.528 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.528 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.529 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.529 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.529 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.529 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2023-07-06T20:37:05.529 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.529 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.529 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.529 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.530 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.530 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.530 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.530 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.530 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.530 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.531 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.531 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.531 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.531 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.531 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.531 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.531 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.531 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.532 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.532 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.532 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.532 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.532 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.532 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.532 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.533 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.533 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.533 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.533 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.533 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.533 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.533 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.534 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.534 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.534 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.534 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.535 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.535 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.535 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.536 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.536 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.536 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.536 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.537 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.537 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.537 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.538 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.538 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.538 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.539 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.539 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.539 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.539 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.540 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.540 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.540 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.541 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.541 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.541 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.541 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.542 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.542 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.542 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.542 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.542 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.543 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.543 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.543 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.543 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.544 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.544 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.544 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.544 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.545 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.545 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2023-07-06T20:37:05.545 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.545 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.545 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.546 INFO:teuthology.orchestra.run.smithi069.stdout: "drive_group_interval": { 2023-07-06T20:37:05.546 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "drive_group_interval", 2023-07-06T20:37:05.546 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:05.546 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.546 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.546 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300.0", 2023-07-06T20:37:05.546 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.547 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.547 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.547 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-06T20:37:05.547 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.547 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.547 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.547 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.548 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.548 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.548 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.548 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.548 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.549 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.549 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.549 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.549 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.549 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.549 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.549 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.549 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.550 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.550 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.550 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.550 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.550 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.550 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.550 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.550 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.551 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.551 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.551 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.551 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.551 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.551 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.551 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.551 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.552 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.552 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.552 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.552 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.552 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.552 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.553 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.553 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.553 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.553 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.553 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.553 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.553 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.553 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.554 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.554 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.554 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.554 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.554 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.554 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.554 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.554 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.555 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.555 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.555 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.555 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.555 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.555 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.555 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.556 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.556 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.556 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.556 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.556 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.556 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.556 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.557 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.557 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2023-07-06T20:37:05.557 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2023-07-06T20:37:05.557 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.557 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.557 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.557 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2023-07-06T20:37:05.558 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.558 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.558 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.558 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-06T20:37:05.558 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.558 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.558 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.559 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.559 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.559 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.559 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.559 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2023-07-06T20:37:05.559 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.559 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.559 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.560 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.560 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.560 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.560 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.560 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.560 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.560 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.561 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.561 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.561 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.561 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.561 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.561 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.561 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.562 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.562 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.562 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.562 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.562 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.562 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.562 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.562 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.563 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.563 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.563 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.563 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.563 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.563 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.563 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.564 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.564 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.564 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.564 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.564 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.564 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.564 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.565 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.565 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.565 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.565 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.565 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.565 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.565 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.566 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.566 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.566 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.566 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.566 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.566 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.566 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.566 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.567 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.567 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.567 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.567 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.567 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.567 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.567 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.567 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.568 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.568 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.568 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.568 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.568 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.568 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.568 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.569 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.569 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.569 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.569 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2023-07-06T20:37:05.569 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2023-07-06T20:37:05.569 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.569 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.569 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.570 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.570 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.570 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.570 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.570 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.570 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.570 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.571 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.571 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2023-07-06T20:37:05.571 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2023-07-06T20:37:05.571 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.571 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.571 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.571 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2023-07-06T20:37:05.572 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.572 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.572 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.572 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.572 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.572 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.572 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.573 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2023-07-06T20:37:05.573 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2023-07-06T20:37:05.573 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.573 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.573 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.573 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.573 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.574 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.574 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.574 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.574 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.574 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.574 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.574 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.574 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2023-07-06T20:37:05.575 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2023-07-06T20:37:05.575 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.575 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.575 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.575 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.575 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.575 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.576 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.576 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.576 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.576 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.576 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.576 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.576 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2023-07-06T20:37:05.577 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2023-07-06T20:37:05.577 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2023-07-06T20:37:05.577 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.577 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.577 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.577 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.577 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.577 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.578 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.578 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.578 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.578 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.578 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.578 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2023-07-06T20:37:05.578 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2023-07-06T20:37:05.578 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.579 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.579 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.579 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.579 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.579 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.579 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.579 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.580 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.580 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.580 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.580 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.580 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2023-07-06T20:37:05.581 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2023-07-06T20:37:05.581 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.581 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.581 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.582 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.582 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.582 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.582 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.583 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.583 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.583 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.583 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.583 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.583 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2023-07-06T20:37:05.583 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2023-07-06T20:37:05.583 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.584 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.584 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.584 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.584 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.584 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.584 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.584 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.585 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.585 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.585 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.585 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.585 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2023-07-06T20:37:05.585 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2023-07-06T20:37:05.585 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.585 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.586 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.586 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.586 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2023-07-06T20:37:05.586 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2023-07-06T20:37:05.586 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.586 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.586 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.586 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.587 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.587 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.587 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2023-07-06T20:37:05.587 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2023-07-06T20:37:05.587 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.587 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.587 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.588 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.588 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.588 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.588 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.588 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.588 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.589 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.589 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.589 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2023-07-06T20:37:05.589 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2023-07-06T20:37:05.589 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.589 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.589 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.589 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.590 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.590 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.590 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.590 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.590 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.590 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.590 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.590 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.591 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2023-07-06T20:37:05.591 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2023-07-06T20:37:05.591 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.591 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.591 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.591 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.591 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.592 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.592 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.592 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.592 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.592 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.592 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.592 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.592 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.593 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.593 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.593 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2023-07-06T20:37:05.593 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.593 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.593 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.593 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2023-07-06T20:37:05.593 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2023-07-06T20:37:05.594 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.594 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.594 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.594 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.594 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.594 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.594 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.595 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2023-07-06T20:37:05.595 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.595 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.595 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.595 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.595 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2023-07-06T20:37:05.595 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2023-07-06T20:37:05.595 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.596 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.596 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.596 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.596 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.596 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.596 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.596 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2023-07-06T20:37:05.596 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.597 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.597 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.597 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.597 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.597 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.597 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.597 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.598 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.598 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.598 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.598 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.598 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.599 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.599 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.599 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.599 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.599 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.600 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.600 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.600 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.600 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.601 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.601 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.601 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.601 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.602 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.602 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.602 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.602 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.603 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.603 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.603 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.603 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.604 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.604 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.604 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.604 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.605 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.605 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.605 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.605 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.605 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.605 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.606 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.606 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.606 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.606 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.606 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.607 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.607 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.607 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.607 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.607 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.607 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.608 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.608 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.608 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.609 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.609 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.609 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.609 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.610 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.610 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.610 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.610 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.611 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.611 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.611 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.612 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.612 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.612 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.612 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.613 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.613 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.613 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.613 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.614 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2023-07-06T20:37:05.614 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.614 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.615 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.615 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.615 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.616 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.616 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.616 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.616 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.616 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.616 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.617 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.617 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.617 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.617 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.617 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.617 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.617 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.618 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.618 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.618 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.618 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.618 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.618 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.618 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.619 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.619 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.619 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.619 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.619 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.619 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.619 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.619 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.620 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.620 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.620 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.620 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.620 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.620 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.620 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.621 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.621 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.621 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.621 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.621 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.622 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.622 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.622 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.623 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.623 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.623 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.623 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.624 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.624 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.624 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.625 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.625 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.625 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.625 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.625 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.626 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.626 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.626 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.626 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.626 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.626 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.627 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.627 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.627 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.627 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.627 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.627 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.627 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.627 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.628 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.628 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.628 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2023-07-06T20:37:05.628 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.628 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.628 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.628 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.629 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.629 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.629 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.629 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.629 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.629 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.629 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.630 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.630 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.630 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.630 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.630 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.630 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.631 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.631 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.631 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.631 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.631 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.631 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.631 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.632 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.632 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.632 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.632 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.632 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.632 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.633 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.633 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.633 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.633 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.633 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.633 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.633 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.633 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.634 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.634 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.634 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.634 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.634 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.634 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.635 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.635 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.635 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.635 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.635 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.635 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.635 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.635 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.636 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.636 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.636 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.636 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.636 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.636 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.636 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.637 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.637 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.637 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.637 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.637 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.637 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.638 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.638 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.638 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.638 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.638 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.638 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.638 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.638 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.639 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.639 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.639 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2023-07-06T20:37:05.639 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.639 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.639 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.639 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2023-07-06T20:37:05.639 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2023-07-06T20:37:05.640 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.640 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.640 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.640 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-06T20:37:05.640 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.640 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.641 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.641 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.641 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.641 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.641 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.641 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.641 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:05.641 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:05.642 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.642 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.642 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.642 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2023-07-06T20:37:05.642 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.642 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.642 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.642 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.643 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.643 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.643 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.643 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.643 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.643 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.644 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.644 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.644 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.644 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.644 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.644 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.645 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.645 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.645 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.645 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.645 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.645 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.645 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.645 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.646 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.646 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.646 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.646 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.646 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.646 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.646 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.646 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.647 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.647 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.647 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.647 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.647 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.647 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.647 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.648 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.648 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.648 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.648 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.648 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.648 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.648 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.649 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.649 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.649 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.649 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.649 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.649 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.649 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.650 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.650 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.650 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.650 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.650 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.650 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.650 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.650 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.651 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.651 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.651 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.651 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.651 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.651 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.651 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.651 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.652 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.652 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.652 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.652 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.652 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.652 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.653 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.653 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.653 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.654 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.654 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.654 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.654 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2023-07-06T20:37:05.655 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.655 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.655 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.655 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2023-07-06T20:37:05.656 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2023-07-06T20:37:05.656 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.656 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.657 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.657 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.657 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.657 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.658 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.658 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-06T20:37:05.658 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.658 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.658 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.659 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.659 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2023-07-06T20:37:05.659 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2023-07-06T20:37:05.659 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.659 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.659 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.659 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.660 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.660 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.660 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.660 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-06T20:37:05.660 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.660 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.660 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.661 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.661 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2023-07-06T20:37:05.661 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2023-07-06T20:37:05.661 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.661 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.661 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.661 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2023-07-06T20:37:05.661 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.662 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.662 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.662 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-06T20:37:05.662 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.662 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.662 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.662 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.663 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2023-07-06T20:37:05.663 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2023-07-06T20:37:05.663 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.663 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.663 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.663 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.663 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.664 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.664 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.664 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-06T20:37:05.664 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.664 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.664 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.664 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.665 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2023-07-06T20:37:05.665 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2023-07-06T20:37:05.665 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.665 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.665 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.665 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.665 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.665 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.666 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.666 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-06T20:37:05.666 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.666 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.666 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.666 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.666 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2023-07-06T20:37:05.666 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2023-07-06T20:37:05.667 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.667 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.667 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.667 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.667 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.667 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.667 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.667 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.668 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.668 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.668 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.668 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.668 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2023-07-06T20:37:05.668 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2023-07-06T20:37:05.668 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.669 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.669 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.669 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.669 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.669 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.669 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.669 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.669 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.670 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.670 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.670 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.670 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2023-07-06T20:37:05.670 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2023-07-06T20:37:05.670 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.670 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-06T20:37:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.671 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2023-07-06T20:37:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2023-07-06T20:37:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.672 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.673 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.674 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2023-07-06T20:37:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2023-07-06T20:37:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.675 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2023-07-06T20:37:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2023-07-06T20:37:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.676 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2023-07-06T20:37:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.677 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2023-07-06T20:37:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2023-07-06T20:37:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.678 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.679 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2023-07-06T20:37:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2023-07-06T20:37:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.680 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.681 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.682 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.683 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.683 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.683 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.683 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.684 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.684 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.684 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.684 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.684 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.684 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.685 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.685 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.685 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.685 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.685 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.686 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.686 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.686 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.686 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.687 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.687 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.687 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.687 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.688 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.688 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.688 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.689 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.689 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.689 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.690 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.690 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.690 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.691 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.691 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.691 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.691 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.692 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.692 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.692 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.692 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.693 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.693 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.693 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.693 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.693 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.693 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.693 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.694 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.694 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.694 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.694 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.695 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.695 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.695 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.695 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.695 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.695 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.695 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.696 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.696 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.696 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.696 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.696 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.696 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.696 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.696 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.697 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.697 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.697 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.697 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.697 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2023-07-06T20:37:05.697 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2023-07-06T20:37:05.697 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.698 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.698 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.698 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.698 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.698 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.698 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.698 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.699 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.699 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.699 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.699 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.699 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2023-07-06T20:37:05.699 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2023-07-06T20:37:05.699 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.700 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.700 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.700 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.700 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.700 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.700 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.700 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.701 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.701 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.701 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.701 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.701 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2023-07-06T20:37:05.701 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2023-07-06T20:37:05.701 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.701 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.702 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.702 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-06T20:37:05.702 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.702 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.702 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.702 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.702 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.703 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.703 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.703 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.703 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.703 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.703 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.703 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2023-07-06T20:37:05.703 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.704 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.704 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.704 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.704 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.704 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.704 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.705 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.705 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.705 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.705 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.705 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.705 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.705 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.706 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.706 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.706 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.706 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.706 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.706 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.706 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.707 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.707 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.707 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.707 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.707 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.707 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.707 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.707 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.708 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.708 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.708 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.708 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.708 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.709 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.709 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.709 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.709 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.709 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.709 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.710 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.710 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.710 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.710 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.710 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.710 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.711 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.711 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.711 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.711 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.711 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.711 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.712 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.712 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.712 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.712 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.712 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.712 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.713 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.713 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.713 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.713 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.713 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.713 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.713 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.714 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.714 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.714 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.714 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.714 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.714 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2023-07-06T20:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.721 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.721 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.721 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.721 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.721 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.722 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.722 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.722 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.722 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.722 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.722 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.723 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.723 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.723 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.723 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.723 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.723 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.723 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.723 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.724 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.724 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.724 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.724 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.724 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.724 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.724 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.725 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.725 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.725 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.725 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.725 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.725 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.725 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.725 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.726 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.726 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.726 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.726 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.726 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.726 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2023-07-06T20:37:05.726 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2023-07-06T20:37:05.726 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.727 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.727 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.727 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2023-07-06T20:37:05.727 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.727 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.727 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.727 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-06T20:37:05.727 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.728 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.728 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.728 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.728 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2023-07-06T20:37:05.728 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2023-07-06T20:37:05.728 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.728 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.729 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.729 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2023-07-06T20:37:05.729 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.729 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.729 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.729 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-06T20:37:05.729 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.729 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.730 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.730 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.730 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.730 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.730 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.730 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2023-07-06T20:37:05.730 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2023-07-06T20:37:05.730 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2023-07-06T20:37:05.731 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2023-07-06T20:37:05.731 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2023-07-06T20:37:05.731 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2023-07-06T20:37:05.731 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2023-07-06T20:37:05.731 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.731 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.731 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2023-07-06T20:37:05.732 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.732 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.732 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.732 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.732 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.732 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.732 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.732 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.733 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2023-07-06T20:37:05.733 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2023-07-06T20:37:05.733 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.733 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.733 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.733 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.733 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.733 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.734 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.734 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.734 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.734 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.734 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.734 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.734 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2023-07-06T20:37:05.735 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2023-07-06T20:37:05.735 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2023-07-06T20:37:05.735 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.735 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.735 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2023-07-06T20:37:05.735 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.735 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.735 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.736 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.736 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.736 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.736 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.736 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2023-07-06T20:37:05.736 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2023-07-06T20:37:05.736 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.737 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.737 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.737 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.737 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.737 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.737 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.737 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.738 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.738 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.738 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.738 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.738 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.738 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.738 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.739 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.739 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.739 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.739 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.739 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2023-07-06T20:37:05.739 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2023-07-06T20:37:05.740 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.740 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.740 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.740 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.740 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.740 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.740 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.741 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.741 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.741 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.741 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.742 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.742 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2023-07-06T20:37:05.742 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2023-07-06T20:37:05.743 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.743 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.743 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.743 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2023-07-06T20:37:05.744 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.744 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.744 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2023-07-06T20:37:05.744 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2023-07-06T20:37:05.744 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2023-07-06T20:37:05.745 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2023-07-06T20:37:05.745 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2023-07-06T20:37:05.745 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2023-07-06T20:37:05.745 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2023-07-06T20:37:05.745 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.745 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.746 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.746 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.746 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.746 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.746 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2023-07-06T20:37:05.747 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2023-07-06T20:37:05.747 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2023-07-06T20:37:05.747 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.747 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2023-07-06T20:37:05.747 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2023-07-06T20:37:05.748 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.748 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.748 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.748 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.749 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.749 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.749 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.749 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.749 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2023-07-06T20:37:05.750 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2023-07-06T20:37:05.750 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.750 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.750 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2023-07-06T20:37:05.750 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.751 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.751 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.751 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.751 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.752 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.752 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.752 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.752 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2023-07-06T20:37:05.752 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2023-07-06T20:37:05.752 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2023-07-06T20:37:05.753 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.753 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.753 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2023-07-06T20:37:05.753 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.753 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.753 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.753 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.753 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.754 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.754 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.754 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.754 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2023-07-06T20:37:05.754 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2023-07-06T20:37:05.754 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2023-07-06T20:37:05.754 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2023-07-06T20:37:05.755 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2023-07-06T20:37:05.755 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-06T20:37:05.755 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2023-07-06T20:37:05.755 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2023-07-06T20:37:05.755 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2023-07-06T20:37:05.755 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2023-07-06T20:37:05.756 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2023-07-06T20:37:05.756 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2023-07-06T20:37:05.756 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2023-07-06T20:37:05.757 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.757 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.757 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.758 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.758 INFO:teuthology.orchestra.run.smithi069.stdout: "services": { 2023-07-06T20:37:05.758 INFO:teuthology.orchestra.run.smithi069.stdout: "dashboard": "https://172.21.15.69:8443/" 2023-07-06T20:37:05.758 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.759 INFO:teuthology.orchestra.run.smithi069.stdout: "always_on_modules": { 2023-07-06T20:37:05.759 INFO:teuthology.orchestra.run.smithi069.stdout: "octopus": [ 2023-07-06T20:37:05.759 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2023-07-06T20:37:05.759 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2023-07-06T20:37:05.760 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2023-07-06T20:37:05.760 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2023-07-06T20:37:05.760 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2023-07-06T20:37:05.760 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2023-07-06T20:37:05.761 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2023-07-06T20:37:05.761 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2023-07-06T20:37:05.761 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2023-07-06T20:37:05.761 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2023-07-06T20:37:05.762 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.762 INFO:teuthology.orchestra.run.smithi069.stdout: "pacific": [ 2023-07-06T20:37:05.762 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2023-07-06T20:37:05.762 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2023-07-06T20:37:05.763 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2023-07-06T20:37:05.763 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2023-07-06T20:37:05.763 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2023-07-06T20:37:05.763 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2023-07-06T20:37:05.763 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2023-07-06T20:37:05.763 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2023-07-06T20:37:05.764 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2023-07-06T20:37:05.764 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2023-07-06T20:37:05.764 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.764 INFO:teuthology.orchestra.run.smithi069.stdout: "quincy": [ 2023-07-06T20:37:05.764 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2023-07-06T20:37:05.764 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2023-07-06T20:37:05.764 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2023-07-06T20:37:05.765 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2023-07-06T20:37:05.765 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2023-07-06T20:37:05.765 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2023-07-06T20:37:05.765 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2023-07-06T20:37:05.765 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2023-07-06T20:37:05.765 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2023-07-06T20:37:05.765 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2023-07-06T20:37:05.766 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2023-07-06T20:37:05.766 INFO:teuthology.orchestra.run.smithi069.stdout: "reef": [ 2023-07-06T20:37:05.766 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2023-07-06T20:37:05.766 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2023-07-06T20:37:05.766 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2023-07-06T20:37:05.766 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2023-07-06T20:37:05.766 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2023-07-06T20:37:05.767 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2023-07-06T20:37:05.767 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2023-07-06T20:37:05.767 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2023-07-06T20:37:05.767 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2023-07-06T20:37:05.767 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2023-07-06T20:37:05.767 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2023-07-06T20:37:05.768 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.768 INFO:teuthology.orchestra.run.smithi069.stdout: "last_failure_osd_epoch": 3, 2023-07-06T20:37:05.768 INFO:teuthology.orchestra.run.smithi069.stdout: "active_clients": [ 2023-07-06T20:37:05.768 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.768 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2023-07-06T20:37:05.769 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2023-07-06T20:37:05.769 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.769 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2023-07-06T20:37:05.769 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2023-07-06T20:37:05.769 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2392941374 2023-07-06T20:37:05.770 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.770 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2023-07-06T20:37:05.770 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.770 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.770 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "libcephsqlite", 2023-07-06T20:37:05.770 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2023-07-06T20:37:05.770 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.771 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2023-07-06T20:37:05.771 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2023-07-06T20:37:05.771 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 4003303292 2023-07-06T20:37:05.771 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.771 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2023-07-06T20:37:05.771 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.772 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.772 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2023-07-06T20:37:05.772 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2023-07-06T20:37:05.772 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.772 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2023-07-06T20:37:05.772 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2023-07-06T20:37:05.773 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1213432936 2023-07-06T20:37:05.773 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.773 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2023-07-06T20:37:05.773 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:05.773 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.773 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2023-07-06T20:37:05.774 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2023-07-06T20:37:05.774 INFO:teuthology.orchestra.run.smithi069.stdout: { 2023-07-06T20:37:05.774 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2023-07-06T20:37:05.774 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2023-07-06T20:37:05.774 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2449578480 2023-07-06T20:37:05.775 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.775 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2023-07-06T20:37:05.775 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:05.775 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2023-07-06T20:37:05.775 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:37:05.777 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:03 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1227748546' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-06T20:37:05.777 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:04 smithi098 ceph-mon[128968]: pgmap v860: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 20 MiB/s wr, 2.34k op/s 2023-07-06T20:37:05.778 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:03 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1227748546' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-06T20:37:05.778 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:04 smithi156 ceph-mon[129280]: pgmap v860: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 20 MiB/s wr, 2.34k op/s 2023-07-06T20:37:05.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.274+0000 7f7b62bff700 1 -- 172.21.15.69:0/1227748546 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7b4c0a1220 msgr2=0x7f7b4c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:05.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.274+0000 7f7b62bff700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7b4c0a1220 0x7f7b4c0a36d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c100c80 tx=0x7f7b50009260 comp rx=0 tx=0).stop 2023-07-06T20:37:05.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.274+0000 7f7b62bff700 1 -- 172.21.15.69:0/1227748546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5c100e20 msgr2=0x7f7b5c198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:05.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.274+0000 7f7b62bff700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5c100e20 0x7f7b5c198080 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7b48004760 tx=0x7f7b48004790 comp rx=0 tx=0).stop 2023-07-06T20:37:05.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.274+0000 7f7b62bff700 1 -- 172.21.15.69:0/1227748546 shutdown_connections 2023-07-06T20:37:05.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.274+0000 7f7b62bff700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f7b4c0a1220 0x7f7b4c0a36d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:05.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.274+0000 7f7b62bff700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b5c1017b0 0x7f7b5c19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:05.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.274+0000 7f7b62bff700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b5c0ffc30 0x7f7b5c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:05.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.274+0000 7f7b62bff700 1 --2- 172.21.15.69:0/1227748546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5c100e20 0x7f7b5c198080 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:05.779 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.274+0000 7f7b62bff700 1 -- 172.21.15.69:0/1227748546 >> 172.21.15.69:0/1227748546 conn(0x7f7b5c0fb210 msgr2=0x7f7b5c1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:05.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.274+0000 7f7b62bff700 1 -- 172.21.15.69:0/1227748546 shutdown_connections 2023-07-06T20:37:05.780 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:03.274+0000 7f7b62bff700 1 -- 172.21.15.69:0/1227748546 wait complete. 2023-07-06T20:37:05.786 DEBUG:tasks.check_counter:Getting stats from a 2023-07-06T20:37:05.787 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-06T20:37:05.787 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:03 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1227748546' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-06T20:37:05.788 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:04 smithi069 ceph-mon[126399]: pgmap v860: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 20 MiB/s wr, 2.34k op/s 2023-07-06T20:37:06.517 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:37:07.067 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:06 smithi069 ceph-mon[126399]: pgmap v861: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 2.08k op/s 2023-07-06T20:37:07.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:06 smithi156 ceph-mon[129280]: pgmap v861: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 2.08k op/s 2023-07-06T20:37:07.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:06 smithi098 ceph-mon[128968]: pgmap v861: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 2.08k op/s 2023-07-06T20:37:08.067 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:37:08.067 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T20:37:08.067 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2023-07-06T20:37:08.067 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2023-07-06T20:37:08.067 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2023-07-06T20:37:08.068 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2023-07-06T20:37:08.068 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:08.068 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2023-07-06T20:37:08.068 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.003773974, 2023-07-06T20:37:08.068 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2023-07-06T20:37:08.068 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000682379, 2023-07-06T20:37:08.068 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-06T20:37:08.069 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:08.069 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:08.069 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:08.069 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:08.069 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:08.069 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:08.069 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:08.070 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:08.070 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:08.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:08.070 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-06T20:37:08.070 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2023-07-06T20:37:08.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:08.071 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T20:37:08.071 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 6, 2023-07-06T20:37:08.071 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 244, 2023-07-06T20:37:08.071 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 3312, 2023-07-06T20:37:08.072 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 194784, 2023-07-06T20:37:08.072 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:08.072 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 2, 2023-07-06T20:37:08.072 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.024507110, 2023-07-06T20:37:08.072 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.018982888, 2023-07-06T20:37:08.073 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.002828093, 2023-07-06T20:37:08.073 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-06T20:37:08.073 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:08.073 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 243, 2023-07-06T20:37:08.073 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.082449013, 2023-07-06T20:37:08.073 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000339296 2023-07-06T20:37:08.074 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:08.074 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:08.074 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:08.074 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:08.074 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:08.074 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:08.074 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-07-06T20:37:08.075 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 194784 2023-07-06T20:37:08.075 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:08.075 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T20:37:08.075 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 319, 2023-07-06T20:37:08.075 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 316, 2023-07-06T20:37:08.075 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 500432, 2023-07-06T20:37:08.076 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 180848, 2023-07-06T20:37:08.076 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2023-07-06T20:37:08.076 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2023-07-06T20:37:08.076 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.057575930, 2023-07-06T20:37:08.076 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.025653352, 2023-07-06T20:37:08.076 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.020818814, 2023-07-06T20:37:08.076 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.009027606, 2023-07-06T20:37:08.077 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:08.077 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 315, 2023-07-06T20:37:08.077 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.112820610, 2023-07-06T20:37:08.077 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000358160 2023-07-06T20:37:08.077 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:08.077 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:08.077 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:08.078 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:08.078 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:08.078 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:08.078 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 500432, 2023-07-06T20:37:08.078 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 180848 2023-07-06T20:37:08.078 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:08.079 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2023-07-06T20:37:08.079 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2023-07-06T20:37:08.079 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2023-07-06T20:37:08.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:08.079 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2023-07-06T20:37:08.079 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2023-07-06T20:37:08.080 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2023-07-06T20:37:08.080 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T20:37:08.080 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2023-07-06T20:37:08.080 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T20:37:08.080 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2023-07-06T20:37:08.080 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T20:37:08.081 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T20:37:08.081 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T20:37:08.081 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T20:37:08.081 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T20:37:08.081 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2023-07-06T20:37:08.082 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T20:37:08.082 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T20:37:08.082 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2023-07-06T20:37:08.082 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2023-07-06T20:37:08.082 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2023-07-06T20:37:08.083 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2023-07-06T20:37:08.083 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T20:37:08.083 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T20:37:08.083 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T20:37:08.083 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T20:37:08.083 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T20:37:08.083 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2023-07-06T20:37:08.083 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2023-07-06T20:37:08.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2023-07-06T20:37:08.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T20:37:08.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T20:37:08.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2023-07-06T20:37:08.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2023-07-06T20:37:08.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2023-07-06T20:37:08.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2023-07-06T20:37:08.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T20:37:08.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2023-07-06T20:37:08.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T20:37:08.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2023-07-06T20:37:08.085 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148704, 2023-07-06T20:37:08.085 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 29, 2023-07-06T20:37:08.085 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2023-07-06T20:37:08.086 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2023-07-06T20:37:08.086 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2023-07-06T20:37:08.086 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2023-07-06T20:37:08.086 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2023-07-06T20:37:08.086 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2023-07-06T20:37:08.086 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2023-07-06T20:37:08.086 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2023-07-06T20:37:08.086 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2023-07-06T20:37:08.087 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2023-07-06T20:37:08.087 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T20:37:08.087 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2023-07-06T20:37:08.087 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2023-07-06T20:37:08.087 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2023-07-06T20:37:08.087 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2023-07-06T20:37:08.087 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2023-07-06T20:37:08.088 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2023-07-06T20:37:08.088 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2023-07-06T20:37:08.088 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2023-07-06T20:37:08.088 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2023-07-06T20:37:08.088 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:08.088 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T20:37:08.088 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-07-06T20:37:08.088 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2023-07-06T20:37:08.089 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-07-06T20:37:08.089 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 321, 2023-07-06T20:37:08.089 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 455326, 2023-07-06T20:37:08.089 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:08.089 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 321, 2023-07-06T20:37:08.089 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-07-06T20:37:08.089 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-07-06T20:37:08.090 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 321, 2023-07-06T20:37:08.090 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 455326, 2023-07-06T20:37:08.090 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-07-06T20:37:08.090 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:08.090 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:08.090 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:08.091 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:08.091 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:08.091 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:37:08.501 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-07-06T20:37:08.502 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-07-06T20:37:08.502 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2023-07-06T20:37:08.502 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2023-07-06T20:37:08.502 DEBUG:tasks.check_counter:Getting stats from d 2023-07-06T20:37:08.503 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-06T20:37:09.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:08 smithi156 ceph-mon[129280]: pgmap v862: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 1.87k op/s 2023-07-06T20:37:09.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:08 smithi098 ceph-mon[128968]: pgmap v862: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 1.87k op/s 2023-07-06T20:37:09.226 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:37:09.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:08 smithi069 ceph-mon[126399]: pgmap v862: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 1.87k op/s 2023-07-06T20:37:10.778 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:37:10.778 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T20:37:10.779 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2023-07-06T20:37:10.779 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2023-07-06T20:37:10.779 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2023-07-06T20:37:10.779 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2023-07-06T20:37:10.779 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:10.779 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2023-07-06T20:37:10.779 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.003411064, 2023-07-06T20:37:10.779 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2023-07-06T20:37:10.780 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000792948, 2023-07-06T20:37:10.780 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-06T20:37:10.780 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:10.780 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:10.780 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:10.780 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:10.780 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:10.781 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:10.781 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:10.781 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:10.781 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:10.781 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:10.781 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-06T20:37:10.781 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2023-07-06T20:37:10.782 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:10.782 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T20:37:10.782 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2, 2023-07-06T20:37:10.782 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 246, 2023-07-06T20:37:10.782 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 320, 2023-07-06T20:37:10.782 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 195040, 2023-07-06T20:37:10.782 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:10.782 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 2, 2023-07-06T20:37:10.783 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.024751689, 2023-07-06T20:37:10.783 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.019605912, 2023-07-06T20:37:10.783 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.002313322, 2023-07-06T20:37:10.783 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-06T20:37:10.783 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:10.783 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 244, 2023-07-06T20:37:10.783 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.081746207, 2023-07-06T20:37:10.783 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000335025 2023-07-06T20:37:10.784 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:10.784 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:10.784 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:10.784 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:10.784 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:10.784 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:10.784 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-06T20:37:10.785 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 195040 2023-07-06T20:37:10.785 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:10.785 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T20:37:10.785 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 328, 2023-07-06T20:37:10.785 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 323, 2023-07-06T20:37:10.785 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 504784, 2023-07-06T20:37:10.785 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 184256, 2023-07-06T20:37:10.785 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2023-07-06T20:37:10.786 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2023-07-06T20:37:10.786 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.059620270, 2023-07-06T20:37:10.786 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.026106771, 2023-07-06T20:37:10.786 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.021989173, 2023-07-06T20:37:10.786 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.009543464, 2023-07-06T20:37:10.786 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:10.786 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 323, 2023-07-06T20:37:10.786 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.117582677, 2023-07-06T20:37:10.787 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000364033 2023-07-06T20:37:10.787 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:10.787 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:10.787 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:10.787 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:10.787 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:10.787 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:10.787 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 504784, 2023-07-06T20:37:10.788 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 184256 2023-07-06T20:37:10.788 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:10.788 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2023-07-06T20:37:10.788 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2023-07-06T20:37:10.788 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2023-07-06T20:37:10.788 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:10.788 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2023-07-06T20:37:10.789 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2023-07-06T20:37:10.789 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2023-07-06T20:37:10.789 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T20:37:10.789 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2023-07-06T20:37:10.789 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T20:37:10.789 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2023-07-06T20:37:10.789 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T20:37:10.789 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T20:37:10.790 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T20:37:10.790 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T20:37:10.790 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T20:37:10.790 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2023-07-06T20:37:10.790 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T20:37:10.790 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T20:37:10.790 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2023-07-06T20:37:10.790 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2023-07-06T20:37:10.791 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2023-07-06T20:37:10.791 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2023-07-06T20:37:10.791 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T20:37:10.791 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T20:37:10.791 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T20:37:10.791 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T20:37:10.791 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T20:37:10.792 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2023-07-06T20:37:10.792 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2023-07-06T20:37:10.792 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2023-07-06T20:37:10.792 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T20:37:10.792 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T20:37:10.792 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2023-07-06T20:37:10.792 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2023-07-06T20:37:10.792 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2023-07-06T20:37:10.793 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2023-07-06T20:37:10.793 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T20:37:10.793 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2023-07-06T20:37:10.793 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T20:37:10.793 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2023-07-06T20:37:10.793 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148744, 2023-07-06T20:37:10.793 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 30, 2023-07-06T20:37:10.793 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2023-07-06T20:37:10.794 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2023-07-06T20:37:10.794 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2023-07-06T20:37:10.794 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2023-07-06T20:37:10.794 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2023-07-06T20:37:10.794 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2023-07-06T20:37:10.794 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2023-07-06T20:37:10.794 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2023-07-06T20:37:10.795 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2023-07-06T20:37:10.795 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2023-07-06T20:37:10.795 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T20:37:10.795 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2023-07-06T20:37:10.795 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2023-07-06T20:37:10.795 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2023-07-06T20:37:10.795 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2023-07-06T20:37:10.795 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2023-07-06T20:37:10.796 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2023-07-06T20:37:10.796 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2023-07-06T20:37:10.796 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2023-07-06T20:37:10.796 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2023-07-06T20:37:10.796 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:10.796 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T20:37:10.796 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-07-06T20:37:10.796 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2023-07-06T20:37:10.797 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-07-06T20:37:10.797 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 326, 2023-07-06T20:37:10.797 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 455976, 2023-07-06T20:37:10.797 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:10.797 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 326, 2023-07-06T20:37:10.797 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-07-06T20:37:10.797 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-07-06T20:37:10.797 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 326, 2023-07-06T20:37:10.798 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 455976, 2023-07-06T20:37:10.798 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-07-06T20:37:10.798 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:10.798 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:10.798 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:10.798 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:10.798 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:10.799 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:37:11.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:10 smithi156 ceph-mon[129280]: pgmap v863: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 10 MiB/s wr, 1.27k op/s 2023-07-06T20:37:11.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:10 smithi098 ceph-mon[128968]: pgmap v863: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 10 MiB/s wr, 1.27k op/s 2023-07-06T20:37:11.145 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:10 smithi069 ceph-mon[126399]: pgmap v863: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 10 MiB/s wr, 1.27k op/s 2023-07-06T20:37:11.213 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-07-06T20:37:11.214 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-07-06T20:37:11.214 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-07-06T20:37:11.214 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-07-06T20:37:11.214 DEBUG:tasks.check_counter:Getting stats from g 2023-07-06T20:37:11.215 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-06T20:37:11.937 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:37:13.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:12 smithi156 ceph-mon[129280]: pgmap v864: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.8 MiB/s wr, 224 op/s 2023-07-06T20:37:13.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:12 smithi098 ceph-mon[128968]: pgmap v864: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.8 MiB/s wr, 224 op/s 2023-07-06T20:37:13.188 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:12 smithi069 ceph-mon[126399]: pgmap v864: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.8 MiB/s wr, 224 op/s 2023-07-06T20:37:13.502 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:37:13.503 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T20:37:13.503 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 326, 2023-07-06T20:37:13.503 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 321, 2023-07-06T20:37:13.503 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 504240, 2023-07-06T20:37:13.503 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 183072, 2023-07-06T20:37:13.503 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:13.503 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2023-07-06T20:37:13.503 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.061310393, 2023-07-06T20:37:13.504 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.026092991, 2023-07-06T20:37:13.504 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.021711614, 2023-07-06T20:37:13.504 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.009012948, 2023-07-06T20:37:13.504 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:13.504 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 321, 2023-07-06T20:37:13.504 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.051303294, 2023-07-06T20:37:13.504 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000159823 2023-07-06T20:37:13.505 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:13.505 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:13.505 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:13.505 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:13.505 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:13.505 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:13.505 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 504240, 2023-07-06T20:37:13.506 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 183072 2023-07-06T20:37:13.506 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:13.506 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T20:37:13.506 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2, 2023-07-06T20:37:13.506 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 244, 2023-07-06T20:37:13.506 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 320, 2023-07-06T20:37:13.506 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 194688, 2023-07-06T20:37:13.507 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:13.507 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 2, 2023-07-06T20:37:13.507 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.025001774, 2023-07-06T20:37:13.507 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.019721066, 2023-07-06T20:37:13.508 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.002632202, 2023-07-06T20:37:13.508 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-06T20:37:13.508 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:13.508 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 242, 2023-07-06T20:37:13.509 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.047697309, 2023-07-06T20:37:13.509 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000197096 2023-07-06T20:37:13.509 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:13.509 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:13.509 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:13.510 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:13.510 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:13.510 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:13.510 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-06T20:37:13.511 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 194688 2023-07-06T20:37:13.511 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:13.511 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T20:37:13.511 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2023-07-06T20:37:13.511 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2023-07-06T20:37:13.512 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2023-07-06T20:37:13.512 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2023-07-06T20:37:13.512 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2023-07-06T20:37:13.512 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2023-07-06T20:37:13.512 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.000982814, 2023-07-06T20:37:13.513 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2023-07-06T20:37:13.513 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000855270, 2023-07-06T20:37:13.513 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-06T20:37:13.513 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:13.513 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:13.514 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:13.514 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:13.514 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:13.514 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:13.515 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:13.515 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:13.515 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:13.515 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:13.515 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-06T20:37:13.516 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2023-07-06T20:37:13.516 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:13.516 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2023-07-06T20:37:13.516 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2023-07-06T20:37:13.517 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2023-07-06T20:37:13.517 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:13.517 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2023-07-06T20:37:13.517 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2023-07-06T20:37:13.517 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2023-07-06T20:37:13.518 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T20:37:13.518 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2023-07-06T20:37:13.518 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T20:37:13.518 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2023-07-06T20:37:13.519 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T20:37:13.519 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T20:37:13.519 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T20:37:13.519 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T20:37:13.519 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T20:37:13.519 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2023-07-06T20:37:13.520 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T20:37:13.520 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T20:37:13.520 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2023-07-06T20:37:13.520 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2023-07-06T20:37:13.520 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2023-07-06T20:37:13.521 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2023-07-06T20:37:13.521 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T20:37:13.521 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T20:37:13.521 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T20:37:13.521 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T20:37:13.521 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T20:37:13.521 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2023-07-06T20:37:13.521 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2023-07-06T20:37:13.522 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2023-07-06T20:37:13.522 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T20:37:13.522 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T20:37:13.522 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2023-07-06T20:37:13.522 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2023-07-06T20:37:13.522 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2023-07-06T20:37:13.522 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2023-07-06T20:37:13.523 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T20:37:13.523 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2023-07-06T20:37:13.523 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T20:37:13.523 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2023-07-06T20:37:13.523 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148704, 2023-07-06T20:37:13.523 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 29, 2023-07-06T20:37:13.523 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2023-07-06T20:37:13.523 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2023-07-06T20:37:13.524 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2023-07-06T20:37:13.524 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2023-07-06T20:37:13.524 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2023-07-06T20:37:13.524 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2023-07-06T20:37:13.524 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2023-07-06T20:37:13.524 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2023-07-06T20:37:13.524 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2023-07-06T20:37:13.525 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2023-07-06T20:37:13.525 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T20:37:13.525 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2023-07-06T20:37:13.525 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2023-07-06T20:37:13.525 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2023-07-06T20:37:13.525 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2023-07-06T20:37:13.525 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2023-07-06T20:37:13.525 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2023-07-06T20:37:13.526 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2023-07-06T20:37:13.526 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2023-07-06T20:37:13.526 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2023-07-06T20:37:13.526 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:13.526 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T20:37:13.526 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-07-06T20:37:13.526 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2023-07-06T20:37:13.526 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-07-06T20:37:13.527 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 324, 2023-07-06T20:37:13.527 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 455716, 2023-07-06T20:37:13.527 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:13.527 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 324, 2023-07-06T20:37:13.527 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-07-06T20:37:13.527 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-07-06T20:37:13.527 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 324, 2023-07-06T20:37:13.527 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 455716, 2023-07-06T20:37:13.528 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-07-06T20:37:13.528 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:13.528 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:13.528 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:13.528 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:13.528 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:13.528 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:37:13.929 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-07-06T20:37:13.930 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-07-06T20:37:13.930 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-07-06T20:37:13.931 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-07-06T20:37:13.931 DEBUG:tasks.check_counter:Getting stats from j 2023-07-06T20:37:13.932 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-06T20:37:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.a/config 2023-07-06T20:37:15.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:14 smithi069 ceph-mon[126399]: pgmap v865: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 559 KiB/s wr, 81 op/s 2023-07-06T20:37:15.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:14 smithi156 ceph-mon[129280]: pgmap v865: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 559 KiB/s wr, 81 op/s 2023-07-06T20:37:15.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:14 smithi098 ceph-mon[128968]: pgmap v865: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 559 KiB/s wr, 81 op/s 2023-07-06T20:37:16.208 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-07-06T20:37:16.209 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T20:37:16.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 327, 2023-07-06T20:37:16.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 324, 2023-07-06T20:37:16.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 502608, 2023-07-06T20:37:16.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 185584, 2023-07-06T20:37:16.209 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:16.210 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2023-07-06T20:37:16.210 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.062059953, 2023-07-06T20:37:16.210 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.025882582, 2023-07-06T20:37:16.210 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.022434317, 2023-07-06T20:37:16.210 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.009757492, 2023-07-06T20:37:16.210 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:16.210 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 323, 2023-07-06T20:37:16.210 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.078892295, 2023-07-06T20:37:16.211 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000244248 2023-07-06T20:37:16.211 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:16.211 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:16.211 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:16.211 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:16.211 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:16.211 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:16.211 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 502608, 2023-07-06T20:37:16.212 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 185584 2023-07-06T20:37:16.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:16.212 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T20:37:16.212 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 6, 2023-07-06T20:37:16.212 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 250, 2023-07-06T20:37:16.212 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 3312, 2023-07-06T20:37:16.212 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 195840, 2023-07-06T20:37:16.213 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:16.213 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 2, 2023-07-06T20:37:16.213 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.025651480, 2023-07-06T20:37:16.213 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.019786025, 2023-07-06T20:37:16.213 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.002960404, 2023-07-06T20:37:16.213 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-06T20:37:16.213 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:16.213 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 249, 2023-07-06T20:37:16.214 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.094336985, 2023-07-06T20:37:16.214 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000378863 2023-07-06T20:37:16.214 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:16.214 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:16.214 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:16.214 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:16.214 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:16.214 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:16.215 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-07-06T20:37:16.215 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 195840 2023-07-06T20:37:16.215 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:16.215 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T20:37:16.215 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2023-07-06T20:37:16.215 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2023-07-06T20:37:16.215 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2023-07-06T20:37:16.215 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2023-07-06T20:37:16.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2023-07-06T20:37:16.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2023-07-06T20:37:16.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.000852234, 2023-07-06T20:37:16.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2023-07-06T20:37:16.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000720602, 2023-07-06T20:37:16.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-06T20:37:16.216 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:16.216 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:16.217 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:16.217 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:16.217 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:16.217 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:16.217 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:16.217 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:16.217 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:16.218 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:16.218 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-06T20:37:16.218 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2023-07-06T20:37:16.218 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:16.218 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2023-07-06T20:37:16.218 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2023-07-06T20:37:16.218 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2023-07-06T20:37:16.218 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:16.219 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2023-07-06T20:37:16.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2023-07-06T20:37:16.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2023-07-06T20:37:16.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T20:37:16.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2023-07-06T20:37:16.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T20:37:16.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2023-07-06T20:37:16.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T20:37:16.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T20:37:16.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T20:37:16.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T20:37:16.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T20:37:16.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2023-07-06T20:37:16.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T20:37:16.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T20:37:16.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2023-07-06T20:37:16.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2023-07-06T20:37:16.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2023-07-06T20:37:16.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2023-07-06T20:37:16.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T20:37:16.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T20:37:16.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T20:37:16.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T20:37:16.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T20:37:16.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2023-07-06T20:37:16.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2023-07-06T20:37:16.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2023-07-06T20:37:16.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T20:37:16.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T20:37:16.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2023-07-06T20:37:16.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2023-07-06T20:37:16.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2023-07-06T20:37:16.223 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2023-07-06T20:37:16.223 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T20:37:16.223 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2023-07-06T20:37:16.223 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T20:37:16.223 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2023-07-06T20:37:16.223 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148704, 2023-07-06T20:37:16.223 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 29, 2023-07-06T20:37:16.224 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2023-07-06T20:37:16.224 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2023-07-06T20:37:16.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2023-07-06T20:37:16.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2023-07-06T20:37:16.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2023-07-06T20:37:16.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2023-07-06T20:37:16.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2023-07-06T20:37:16.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2023-07-06T20:37:16.225 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2023-07-06T20:37:16.225 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2023-07-06T20:37:16.225 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T20:37:16.225 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2023-07-06T20:37:16.225 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2023-07-06T20:37:16.225 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2023-07-06T20:37:16.225 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2023-07-06T20:37:16.225 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2023-07-06T20:37:16.226 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2023-07-06T20:37:16.226 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2023-07-06T20:37:16.226 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2023-07-06T20:37:16.226 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2023-07-06T20:37:16.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-07-06T20:37:16.226 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T20:37:16.226 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-07-06T20:37:16.226 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2023-07-06T20:37:16.227 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-07-06T20:37:16.227 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 329, 2023-07-06T20:37:16.227 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 456366, 2023-07-06T20:37:16.227 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:16.227 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 329, 2023-07-06T20:37:16.227 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-07-06T20:37:16.227 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-07-06T20:37:16.227 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 329, 2023-07-06T20:37:16.228 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 456366, 2023-07-06T20:37:16.228 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-07-06T20:37:16.228 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-07-06T20:37:16.228 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-07-06T20:37:16.228 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-07-06T20:37:16.228 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:16.228 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-07-06T20:37:16.229 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-07-06T20:37:16.742 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-06T20:37:16.743 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-06T20:37:16.743 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-07-06T20:37:16.743 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-07-06T20:37:16.743 DEBUG:tasks.check_counter:Getting stats from b 2023-07-06T20:37:16.744 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-06T20:37:17.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:16 smithi069 ceph-mon[126399]: pgmap v866: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-07-06T20:37:17.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:16 smithi156 ceph-mon[129280]: pgmap v866: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-07-06T20:37:17.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:16 smithi098 ceph-mon[128968]: pgmap v866: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-07-06T20:37:18.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:18 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:37:18.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:18 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:37:18.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:18 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-06T20:37:19.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:19 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-07-06T20:37:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:19 smithi069 ceph-mon[126399]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-06T20:37:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:19 smithi069 ceph-mon[126399]: pgmap v867: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-07-06T20:37:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:19 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.593 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.b/config 2023-07-06T20:37:19.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:19 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-07-06T20:37:19.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:19 smithi156 ceph-mon[129280]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-06T20:37:19.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:19 smithi156 ceph-mon[129280]: pgmap v867: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-07-06T20:37:19.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:19 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:19 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-07-06T20:37:19.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:19 smithi098 ceph-mon[128968]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-06T20:37:19.626 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:19 smithi098 ceph-mon[128968]: pgmap v867: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-07-06T20:37:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:19.627 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:19 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:20 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi069 to 220.8M 2023-07-06T20:37:20.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:20 smithi069 ceph-mon[126399]: Unable to set osd_memory_target on smithi069 to 231614566: error parsing value: Value '231614566' is below minimum 939524096 2023-07-06T20:37:20.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:20 smithi069 ceph-mon[126399]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-07-06T20:37:20.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:20 smithi069 ceph-mon[126399]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-06T20:37:20.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:37:20.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:37:20.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:20 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:37:20.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:20 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi069 to 220.8M 2023-07-06T20:37:20.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:20 smithi156 ceph-mon[129280]: Unable to set osd_memory_target on smithi069 to 231614566: error parsing value: Value '231614566' is below minimum 939524096 2023-07-06T20:37:20.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:20 smithi156 ceph-mon[129280]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-07-06T20:37:20.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:20 smithi156 ceph-mon[129280]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-06T20:37:20.608 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:37:20.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:37:20.609 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:20 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:37:20.803 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:20 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi069 to 220.8M 2023-07-06T20:37:20.803 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:20 smithi098 ceph-mon[128968]: Unable to set osd_memory_target on smithi069 to 231614566: error parsing value: Value '231614566' is below minimum 939524096 2023-07-06T20:37:20.803 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.803 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.803 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.804 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-06T20:37:20.804 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:20 smithi098 ceph-mon[128968]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-07-06T20:37:20.804 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:20 smithi098 ceph-mon[128968]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-06T20:37:20.804 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-06T20:37:20.804 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-06T20:37:20.804 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:20 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' 2023-07-06T20:37:21.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:21 smithi098 ceph-mon[128968]: pgmap v868: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-07-06T20:37:21.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:21 smithi069 ceph-mon[126399]: pgmap v868: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-07-06T20:37:21.607 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:21 smithi156 ceph-mon[129280]: pgmap v868: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-07-06T20:37:23.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:22 smithi156 ceph-mon[129280]: pgmap v869: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-07-06T20:37:23.213 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:22 smithi098 ceph-mon[128968]: pgmap v869: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-07-06T20:37:23.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:22 smithi069 ceph-mon[126399]: pgmap v869: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-07-06T20:37:25.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:24 smithi069 ceph-mon[126399]: pgmap v870: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 9.5 KiB/s wr, 9 op/s 2023-07-06T20:37:25.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:24 smithi156 ceph-mon[129280]: pgmap v870: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 9.5 KiB/s wr, 9 op/s 2023-07-06T20:37:25.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:24 smithi098 ceph-mon[128968]: pgmap v870: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 9.5 KiB/s wr, 9 op/s 2023-07-06T20:37:26.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:26 smithi098 ceph-mon[128968]: pgmap v871: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7.2 KiB/s wr, 6 op/s 2023-07-06T20:37:27.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:26 smithi156 ceph-mon[129280]: pgmap v871: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7.2 KiB/s wr, 6 op/s 2023-07-06T20:37:27.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:26 smithi069 ceph-mon[126399]: pgmap v871: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7.2 KiB/s wr, 6 op/s 2023-07-06T20:37:29.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:28 smithi156 ceph-mon[129280]: pgmap v872: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2023-07-06T20:37:29.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:28 smithi098 ceph-mon[128968]: pgmap v872: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2023-07-06T20:37:29.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:28 smithi069 ceph-mon[126399]: pgmap v872: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2023-07-06T20:37:30.947 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-07-06T20:37:30.948 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T20:37:30.948 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 4788, 2023-07-06T20:37:30.948 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 6559, 2023-07-06T20:37:30.948 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1743539, 2023-07-06T20:37:30.948 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 6412904, 2023-07-06T20:37:30.948 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 78, 2023-07-06T20:37:30.949 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 3, 2023-07-06T20:37:30.949 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.450468271, 2023-07-06T20:37:30.949 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.165143791, 2023-07-06T20:37:30.949 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.189175470, 2023-07-06T20:37:30.949 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.105663365, 2023-07-06T20:37:30.949 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:30.949 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 6563, 2023-07-06T20:37:30.949 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 2.215637337, 2023-07-06T20:37:30.950 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000337595 2023-07-06T20:37:30.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.950 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:30.950 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 7329, 2023-07-06T20:37:30.950 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001628696, 2023-07-06T20:37:30.950 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000222 2023-07-06T20:37:30.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.951 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 569728, 2023-07-06T20:37:30.951 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 193520 2023-07-06T20:37:30.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.951 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T20:37:30.951 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 12562674, 2023-07-06T20:37:30.951 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 12564628, 2023-07-06T20:37:30.951 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1985496489, 2023-07-06T20:37:30.951 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 2041499452, 2023-07-06T20:37:30.952 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 9, 2023-07-06T20:37:30.952 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2023-07-06T20:37:30.952 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 377.035958477, 2023-07-06T20:37:30.952 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 175.252921181, 2023-07-06T20:37:30.952 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 264.470959375, 2023-07-06T20:37:30.952 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.067402331, 2023-07-06T20:37:30.952 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:30.952 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 12564628, 2023-07-06T20:37:30.953 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 324.641607148, 2023-07-06T20:37:30.953 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000025837 2023-07-06T20:37:30.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.953 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:30.953 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 22018166, 2023-07-06T20:37:30.953 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 5.409526911, 2023-07-06T20:37:30.953 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000245 2023-07-06T20:37:30.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.954 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-06T20:37:30.954 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 599616 2023-07-06T20:37:30.954 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.954 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T20:37:30.954 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 12554, 2023-07-06T20:37:30.954 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 13670, 2023-07-06T20:37:30.954 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 3687555, 2023-07-06T20:37:30.954 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 14010539, 2023-07-06T20:37:30.955 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 15, 2023-07-06T20:37:30.955 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 8, 2023-07-06T20:37:30.955 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.830683776, 2023-07-06T20:37:30.955 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.261941655, 2023-07-06T20:37:30.955 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.460373898, 2023-07-06T20:37:30.955 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.218776251, 2023-07-06T20:37:30.955 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:30.956 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 13662, 2023-07-06T20:37:30.956 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3.009811154, 2023-07-06T20:37:30.956 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000220305 2023-07-06T20:37:30.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.956 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:30.956 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 19543, 2023-07-06T20:37:30.956 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.003736910, 2023-07-06T20:37:30.956 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000191 2023-07-06T20:37:30.957 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.957 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 54464, 2023-07-06T20:37:30.957 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 29344 2023-07-06T20:37:30.957 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.957 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2023-07-06T20:37:30.958 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2023-07-06T20:37:30.958 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2023-07-06T20:37:30.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.958 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2023-07-06T20:37:30.958 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2023-07-06T20:37:30.958 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2023-07-06T20:37:30.958 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4770, 2023-07-06T20:37:30.959 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 4.232111664, 2023-07-06T20:37:30.959 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000887235 2023-07-06T20:37:30.959 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:30.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.959 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2023-07-06T20:37:30.959 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2023-07-06T20:37:30.959 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2023-07-06T20:37:30.960 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1313, 2023-07-06T20:37:30.960 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.076925471, 2023-07-06T20:37:30.960 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000058587 2023-07-06T20:37:30.960 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:30.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.960 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2023-07-06T20:37:30.960 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 4280, 2023-07-06T20:37:30.961 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 2795, 2023-07-06T20:37:30.961 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2023-07-06T20:37:30.961 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2795, 2023-07-06T20:37:30.961 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 58.505671806, 2023-07-06T20:37:30.961 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.020932261 2023-07-06T20:37:30.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.961 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2023-07-06T20:37:30.961 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 1485, 2023-07-06T20:37:30.962 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2023-07-06T20:37:30.962 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2023-07-06T20:37:30.962 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 39, 2023-07-06T20:37:30.962 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 3, 2023-07-06T20:37:30.962 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2023-07-06T20:37:30.962 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 591, 2023-07-06T20:37:30.962 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 3, 2023-07-06T20:37:30.962 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 545, 2023-07-06T20:37:30.963 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 43, 2023-07-06T20:37:30.963 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 586, 2023-07-06T20:37:30.963 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 524, 2023-07-06T20:37:30.963 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 507, 2023-07-06T20:37:30.963 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 507, 2023-07-06T20:37:30.963 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 16, 2023-07-06T20:37:30.963 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 5704, 2023-07-06T20:37:30.963 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 3346, 2023-07-06T20:37:30.964 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 1478, 2023-07-06T20:37:30.964 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 0, 2023-07-06T20:37:30.964 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2023-07-06T20:37:30.964 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2023-07-06T20:37:30.964 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 151, 2023-07-06T20:37:30.964 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 7513, 2023-07-06T20:37:30.964 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 2, 2023-07-06T20:37:30.964 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 17, 2023-07-06T20:37:30.965 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 526, 2023-07-06T20:37:30.965 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 4, 2023-07-06T20:37:30.965 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 6, 2023-07-06T20:37:30.965 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2023-07-06T20:37:30.965 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 508, 2023-07-06T20:37:30.965 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2023-07-06T20:37:30.965 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 1046, 2023-07-06T20:37:30.965 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 9135538132, 2023-07-06T20:37:30.966 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2023-07-06T20:37:30.966 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 508, 2023-07-06T20:37:30.966 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2023-07-06T20:37:30.966 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2023-07-06T20:37:30.966 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2023-07-06T20:37:30.967 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 35, 2023-07-06T20:37:30.967 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 2, 2023-07-06T20:37:30.967 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 33, 2023-07-06T20:37:30.967 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 473, 2023-07-06T20:37:30.967 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 9, 2023-07-06T20:37:30.967 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 403, 2023-07-06T20:37:30.967 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 2245, 2023-07-06T20:37:30.968 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2023-07-06T20:37:30.968 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-06T20:37:30.968 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-06T20:37:30.968 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 2923, 2023-07-06T20:37:30.968 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 2144, 2023-07-06T20:37:30.968 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 191, 2023-07-06T20:37:30.968 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 711 2023-07-06T20:37:30.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.969 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2023-07-06T20:37:30.969 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 98, 2023-07-06T20:37:30.969 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2023-07-06T20:37:30.969 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2023-07-06T20:37:30.969 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 2, 2023-07-06T20:37:30.969 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 322, 2023-07-06T20:37:30.969 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2023-07-06T20:37:30.969 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2023-07-06T20:37:30.970 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2023-07-06T20:37:30.970 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 283, 2023-07-06T20:37:30.970 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 270, 2023-07-06T20:37:30.970 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 52, 2023-07-06T20:37:30.970 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2023-07-06T20:37:30.970 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2023-07-06T20:37:30.970 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2023-07-06T20:37:30.970 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2023-07-06T20:37:30.971 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2023-07-06T20:37:30.971 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2023-07-06T20:37:30.971 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 2, 2023-07-06T20:37:30.971 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 17, 2023-07-06T20:37:30.971 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2023-07-06T20:37:30.971 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 3, 2023-07-06T20:37:30.971 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2023-07-06T20:37:30.971 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2023-07-06T20:37:30.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.972 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2023-07-06T20:37:30.972 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 5962, 2023-07-06T20:37:30.972 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2023-07-06T20:37:30.972 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2023-07-06T20:37:30.972 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 5962, 2023-07-06T20:37:30.972 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2023-07-06T20:37:30.972 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2023-07-06T20:37:30.973 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 8, 2023-07-06T20:37:30.973 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2023-07-06T20:37:30.973 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2023-07-06T20:37:30.973 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 8, 2023-07-06T20:37:30.973 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2023-07-06T20:37:30.973 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2023-07-06T20:37:30.973 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2023-07-06T20:37:30.973 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 20363899, 2023-07-06T20:37:30.974 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2023-07-06T20:37:30.974 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2023-07-06T20:37:30.974 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2579, 2023-07-06T20:37:30.974 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 55.490399210, 2023-07-06T20:37:30.974 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.021516246 2023-07-06T20:37:30.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.974 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2023-07-06T20:37:30.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.975 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2023-07-06T20:37:30.975 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 534, 2023-07-06T20:37:30.975 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 1317, 2023-07-06T20:37:30.975 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 783, 2023-07-06T20:37:30.975 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 66, 2023-07-06T20:37:30.975 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 73, 2023-07-06T20:37:30.975 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 7, 2023-07-06T20:37:30.976 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 591, 2023-07-06T20:37:30.976 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 1698, 2023-07-06T20:37:30.976 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 1107, 2023-07-06T20:37:30.976 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 507, 2023-07-06T20:37:30.976 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 1766, 2023-07-06T20:37:30.976 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 1259, 2023-07-06T20:37:30.976 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 66036, 2023-07-06T20:37:30.976 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 207156 2023-07-06T20:37:30.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.977 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2023-07-06T20:37:30.977 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 5052, 2023-07-06T20:37:30.977 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 168, 2023-07-06T20:37:30.977 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 4280, 2023-07-06T20:37:30.977 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 1547, 2023-07-06T20:37:30.977 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 336, 2023-07-06T20:37:30.977 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2023-07-06T20:37:30.978 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1261, 2023-07-06T20:37:30.978 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 51.895104846, 2023-07-06T20:37:30.978 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.041153929 2023-07-06T20:37:30.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.978 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2023-07-06T20:37:30.978 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 162, 2023-07-06T20:37:30.978 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.535947498, 2023-07-06T20:37:30.978 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.003308317 2023-07-06T20:37:30.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.979 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2023-07-06T20:37:30.979 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.979 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.979 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.979 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2023-07-06T20:37:30.979 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 13, 2023-07-06T20:37:30.980 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.024373637, 2023-07-06T20:37:30.980 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001874895 2023-07-06T20:37:30.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.980 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2023-07-06T20:37:30.980 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 321, 2023-07-06T20:37:30.980 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.232076755, 2023-07-06T20:37:30.980 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000722980 2023-07-06T20:37:30.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.981 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2023-07-06T20:37:30.981 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.981 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.981 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.981 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2023-07-06T20:37:30.981 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.982 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.982 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.982 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2023-07-06T20:37:30.982 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.982 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.982 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.983 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2023-07-06T20:37:30.983 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.983 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.983 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.983 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2023-07-06T20:37:30.983 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.983 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.984 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.984 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2023-07-06T20:37:30.984 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.984 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.984 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.985 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2023-07-06T20:37:30.985 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 35, 2023-07-06T20:37:30.985 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.274155105, 2023-07-06T20:37:30.985 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.007833003 2023-07-06T20:37:30.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.985 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2023-07-06T20:37:30.985 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.985 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.986 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.986 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2023-07-06T20:37:30.986 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.986 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.986 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.986 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2023-07-06T20:37:30.987 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 535, 2023-07-06T20:37:30.987 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.560435181, 2023-07-06T20:37:30.987 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002916701 2023-07-06T20:37:30.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.987 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2023-07-06T20:37:30.987 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 112, 2023-07-06T20:37:30.987 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.202997481, 2023-07-06T20:37:30.987 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001812477 2023-07-06T20:37:30.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.988 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2023-07-06T20:37:30.988 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 286, 2023-07-06T20:37:30.988 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3.554699080, 2023-07-06T20:37:30.988 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.012429017 2023-07-06T20:37:30.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.988 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2023-07-06T20:37:30.988 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.989 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.989 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.989 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2023-07-06T20:37:30.989 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.989 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.989 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.990 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2023-07-06T20:37:30.990 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.990 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.990 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.990 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2023-07-06T20:37:30.990 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.991 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.991 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.991 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2023-07-06T20:37:30.991 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.991 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.991 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.992 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2023-07-06T20:37:30.992 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2023-07-06T20:37:30.992 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001638543, 2023-07-06T20:37:30.992 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001638543 2023-07-06T20:37:30.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.992 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2023-07-06T20:37:30.992 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.992 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.993 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.993 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2023-07-06T20:37:30.993 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.993 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.993 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.993 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2023-07-06T20:37:30.994 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.994 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.994 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.994 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2023-07-06T20:37:30.994 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 6, 2023-07-06T20:37:30.994 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.017392829, 2023-07-06T20:37:30.995 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002898804 2023-07-06T20:37:30.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.995 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2023-07-06T20:37:30.995 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.995 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.995 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.995 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2023-07-06T20:37:30.996 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 63, 2023-07-06T20:37:30.996 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.206850851, 2023-07-06T20:37:30.996 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.003283346 2023-07-06T20:37:30.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.996 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2023-07-06T20:37:30.997 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2023-07-06T20:37:30.997 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2023-07-06T20:37:30.997 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:30.997 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:30.997 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:30.997 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:30.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.997 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2023-07-06T20:37:30.998 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2023-07-06T20:37:30.998 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 2, 2023-07-06T20:37:30.998 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 1, 2023-07-06T20:37:30.998 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2023-07-06T20:37:30.998 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2023-07-06T20:37:30.998 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 113, 2023-07-06T20:37:30.998 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 113, 2023-07-06T20:37:30.998 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 1102 2023-07-06T20:37:30.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:30.999 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2023-07-06T20:37:30.999 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2023-07-06T20:37:30.999 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2023-07-06T20:37:30.999 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T20:37:30.999 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2023-07-06T20:37:30.999 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T20:37:30.999 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2023-07-06T20:37:31.000 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T20:37:31.000 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T20:37:31.000 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T20:37:31.000 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T20:37:31.000 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T20:37:31.000 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2023-07-06T20:37:31.000 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T20:37:31.000 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T20:37:31.001 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2023-07-06T20:37:31.001 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2023-07-06T20:37:31.001 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2023-07-06T20:37:31.001 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2023-07-06T20:37:31.001 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T20:37:31.001 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T20:37:31.001 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T20:37:31.001 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T20:37:31.002 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T20:37:31.002 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2023-07-06T20:37:31.002 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2023-07-06T20:37:31.002 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2023-07-06T20:37:31.002 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T20:37:31.002 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T20:37:31.002 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2023-07-06T20:37:31.002 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2023-07-06T20:37:31.003 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2023-07-06T20:37:31.003 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2023-07-06T20:37:31.003 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T20:37:31.003 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2023-07-06T20:37:31.003 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T20:37:31.003 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2023-07-06T20:37:31.003 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 311046, 2023-07-06T20:37:31.003 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 630, 2023-07-06T20:37:31.004 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 440, 2023-07-06T20:37:31.004 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 5, 2023-07-06T20:37:31.004 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2023-07-06T20:37:31.004 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2023-07-06T20:37:31.004 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2023-07-06T20:37:31.004 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2023-07-06T20:37:31.004 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2023-07-06T20:37:31.005 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2023-07-06T20:37:31.005 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 7728, 2023-07-06T20:37:31.005 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 163, 2023-07-06T20:37:31.005 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T20:37:31.005 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2023-07-06T20:37:31.005 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2023-07-06T20:37:31.005 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2023-07-06T20:37:31.005 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 1934825, 2023-07-06T20:37:31.006 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 22084, 2023-07-06T20:37:31.006 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2023-07-06T20:37:31.006 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2023-07-06T20:37:31.006 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2023-07-06T20:37:31.006 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2023-07-06T20:37:31.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:31.006 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2023-07-06T20:37:31.006 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2023-07-06T20:37:31.007 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2023-07-06T20:37:31.007 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 4929, 2023-07-06T20:37:31.007 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 16829244, 2023-07-06T20:37:31.007 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 31, 2023-07-06T20:37:31.007 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 4929, 2023-07-06T20:37:31.007 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2023-07-06T20:37:31.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4929, 2023-07-06T20:37:31.007 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 97.824818803, 2023-07-06T20:37:31.008 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.019846788 2023-07-06T20:37:31.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:31.008 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2023-07-06T20:37:31.008 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2023-07-06T20:37:31.008 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4929, 2023-07-06T20:37:31.008 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 5125 2023-07-06T20:37:31.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:31.009 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 4929, 2023-07-06T20:37:31.009 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 622, 2023-07-06T20:37:31.009 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 4307, 2023-07-06T20:37:31.009 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2023-07-06T20:37:31.009 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2023-07-06T20:37:31.009 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2023-07-06T20:37:31.009 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 11, 2023-07-06T20:37:31.009 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 112, 2023-07-06T20:37:31.010 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 2689, 2023-07-06T20:37:31.010 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 173, 2023-07-06T20:37:31.010 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2023-07-06T20:37:31.010 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2023-07-06T20:37:31.010 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2023-07-06T20:37:31.010 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 3, 2023-07-06T20:37:31.010 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 1262, 2023-07-06T20:37:31.010 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2023-07-06T20:37:31.011 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2023-07-06T20:37:31.011 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2023-07-06T20:37:31.011 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 510, 2023-07-06T20:37:31.011 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 22, 2023-07-06T20:37:31.011 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2023-07-06T20:37:31.011 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2023-07-06T20:37:31.011 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2023-07-06T20:37:31.011 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2023-07-06T20:37:31.012 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2023-07-06T20:37:31.012 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2023-07-06T20:37:31.012 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2023-07-06T20:37:31.012 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2023-07-06T20:37:31.012 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2023-07-06T20:37:31.012 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2023-07-06T20:37:31.012 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2023-07-06T20:37:31.012 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2023-07-06T20:37:31.013 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2023-07-06T20:37:31.013 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2023-07-06T20:37:31.013 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2023-07-06T20:37:31.013 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2023-07-06T20:37:31.013 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2023-07-06T20:37:31.013 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2023-07-06T20:37:31.013 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2023-07-06T20:37:31.013 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2023-07-06T20:37:31.014 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2023-07-06T20:37:31.014 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2023-07-06T20:37:31.014 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2023-07-06T20:37:31.014 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2023-07-06T20:37:31.014 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2023-07-06T20:37:31.014 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2023-07-06T20:37:31.014 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 87, 2023-07-06T20:37:31.014 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2023-07-06T20:37:31.015 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 5, 2023-07-06T20:37:31.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 12, 2023-07-06T20:37:31.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 12, 2023-07-06T20:37:31.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2023-07-06T20:37:31.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2023-07-06T20:37:31.015 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 269, 2023-07-06T20:37:31.015 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2023-07-06T20:37:31.016 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 74 2023-07-06T20:37:31.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:31.016 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2023-07-06T20:37:31.016 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2023-07-06T20:37:31.016 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2023-07-06T20:37:31.016 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2023-07-06T20:37:31.016 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2023-07-06T20:37:31.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:31.017 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2023-07-06T20:37:31.017 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2023-07-06T20:37:31.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:31.017 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2023-07-06T20:37:31.017 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2023-07-06T20:37:31.017 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 478, 2023-07-06T20:37:31.017 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2023-07-06T20:37:31.017 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 10, 2023-07-06T20:37:31.018 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 270, 2023-07-06T20:37:31.018 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2023-07-06T20:37:31.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:31.018 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T20:37:31.018 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 136, 2023-07-06T20:37:31.018 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-07-06T20:37:31.018 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:31.018 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 12580021, 2023-07-06T20:37:31.019 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 858698709, 2023-07-06T20:37:31.019 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:31.019 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 12580021, 2023-07-06T20:37:31.019 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-07-06T20:37:31.019 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-07-06T20:37:31.019 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 12580019, 2023-07-06T20:37:31.019 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 858698641, 2023-07-06T20:37:31.019 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:31.020 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:31.020 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:31.020 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:31.020 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:31.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:31.020 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2023-07-06T20:37:31.020 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:31.020 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-07-06T20:37:31.021 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:31.021 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-07-06T20:37:31.021 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-07-06T20:37:31.021 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:31.021 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:31.021 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 4929, 2023-07-06T20:37:31.021 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 16856514, 2023-07-06T20:37:31.022 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 4929, 2023-07-06T20:37:31.022 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 16856514, 2023-07-06T20:37:31.022 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:31.022 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:31.022 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:31.022 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:31.022 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:31.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:31.023 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2023-07-06T20:37:31.023 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:31.023 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2023-07-06T20:37:31.023 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:31.023 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-07-06T20:37:31.023 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-07-06T20:37:31.023 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:31.023 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:31.024 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 4929, 2023-07-06T20:37:31.024 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 4929, 2023-07-06T20:37:31.024 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 4929, 2023-07-06T20:37:31.024 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 4929, 2023-07-06T20:37:31.024 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:31.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:31.024 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:31.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:31.025 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:31.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:31.025 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2023-07-06T20:37:31.025 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:31.025 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2023-07-06T20:37:31.025 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:31.025 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 270, 2023-07-06T20:37:31.026 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 27270, 2023-07-06T20:37:31.026 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:31.026 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 270, 2023-07-06T20:37:31.026 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-07-06T20:37:31.026 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-07-06T20:37:31.026 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 110, 2023-07-06T20:37:31.026 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 27270, 2023-07-06T20:37:31.026 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:31.027 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:31.027 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:31.027 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:31.027 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:31.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:31.027 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55f197b860a0": { 2023-07-06T20:37:31.027 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:31.027 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2023-07-06T20:37:31.028 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:31.028 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 5962, 2023-07-06T20:37:31.028 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 16169595, 2023-07-06T20:37:31.028 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:31.028 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 5962, 2023-07-06T20:37:31.028 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-07-06T20:37:31.028 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-07-06T20:37:31.028 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 2579, 2023-07-06T20:37:31.029 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 16169595, 2023-07-06T20:37:31.029 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:31.029 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:31.029 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:31.029 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:31.029 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:31.029 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:31.029 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-07-06T20:37:31.084 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:30 smithi098 ceph-mon[128968]: pgmap v873: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2023-07-06T20:37:31.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:30 smithi156 ceph-mon[129280]: pgmap v873: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2023-07-06T20:37:31.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:30 smithi069 ceph-mon[126399]: pgmap v873: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2023-07-06T20:37:31.355 INFO:tasks.check_counter:Daemon mds.b mds.exported=17 2023-07-06T20:37:31.356 INFO:tasks.check_counter:Daemon mds.b mds.imported=4 2023-07-06T20:37:31.356 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=98 2023-07-06T20:37:31.356 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2023-07-06T20:37:31.356 DEBUG:tasks.check_counter:Getting stats from e 2023-07-06T20:37:31.357 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-06T20:37:32.227 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.b/config 2023-07-06T20:37:33.038 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:32 smithi098 ceph-mon[128968]: pgmap v874: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-06T20:37:33.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:32 smithi156 ceph-mon[129280]: pgmap v874: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-06T20:37:33.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:32 smithi069 ceph-mon[126399]: pgmap v874: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-06T20:37:33.863 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-07-06T20:37:33.863 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T20:37:33.864 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 3952, 2023-07-06T20:37:33.864 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 4665, 2023-07-06T20:37:33.864 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1170841, 2023-07-06T20:37:33.864 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 5015426, 2023-07-06T20:37:33.864 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 13, 2023-07-06T20:37:33.864 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 3, 2023-07-06T20:37:33.864 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.273320113, 2023-07-06T20:37:33.865 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.095683883, 2023-07-06T20:37:33.865 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.174782531, 2023-07-06T20:37:33.865 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.030761641, 2023-07-06T20:37:33.865 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:33.865 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4664, 2023-07-06T20:37:33.865 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.686976280, 2023-07-06T20:37:33.865 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000147293 2023-07-06T20:37:33.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.866 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:33.866 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 5712, 2023-07-06T20:37:33.866 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001262653, 2023-07-06T20:37:33.866 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000221 2023-07-06T20:37:33.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.866 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 4528, 2023-07-06T20:37:33.866 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 1840 2023-07-06T20:37:33.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.867 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T20:37:33.867 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 370, 2023-07-06T20:37:33.867 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 618, 2023-07-06T20:37:33.867 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 99587, 2023-07-06T20:37:33.867 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 750369, 2023-07-06T20:37:33.867 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 13, 2023-07-06T20:37:33.867 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 6, 2023-07-06T20:37:33.868 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.038444759, 2023-07-06T20:37:33.868 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.015304490, 2023-07-06T20:37:33.868 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.009303707, 2023-07-06T20:37:33.868 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.010060814, 2023-07-06T20:37:33.868 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:33.868 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 616, 2023-07-06T20:37:33.868 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.167860968, 2023-07-06T20:37:33.868 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000272501 2023-07-06T20:37:33.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.869 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:33.869 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4, 2023-07-06T20:37:33.869 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000248, 2023-07-06T20:37:33.869 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000062 2023-07-06T20:37:33.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.869 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-06T20:37:33.870 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 511024 2023-07-06T20:37:33.870 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.870 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T20:37:33.870 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 12619130, 2023-07-06T20:37:33.870 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 12617931, 2023-07-06T20:37:33.870 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 2045918615, 2023-07-06T20:37:33.870 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 1994517725, 2023-07-06T20:37:33.870 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 7, 2023-07-06T20:37:33.871 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 6, 2023-07-06T20:37:33.871 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 404.393303648, 2023-07-06T20:37:33.871 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 195.588119399, 2023-07-06T20:37:33.871 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 263.204125411, 2023-07-06T20:37:33.871 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.031689723, 2023-07-06T20:37:33.871 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:33.871 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 12617930, 2023-07-06T20:37:33.871 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 224.184317961, 2023-07-06T20:37:33.872 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000017767 2023-07-06T20:37:33.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.872 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:33.872 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 19738535, 2023-07-06T20:37:33.872 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3.471232786, 2023-07-06T20:37:33.872 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000175 2023-07-06T20:37:33.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.872 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 557760, 2023-07-06T20:37:33.873 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 194848 2023-07-06T20:37:33.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.873 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2023-07-06T20:37:33.873 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2023-07-06T20:37:33.873 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2023-07-06T20:37:33.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.873 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2023-07-06T20:37:33.873 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2023-07-06T20:37:33.874 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2023-07-06T20:37:33.874 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2037, 2023-07-06T20:37:33.874 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.680216543, 2023-07-06T20:37:33.874 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000333930 2023-07-06T20:37:33.874 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:33.874 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.874 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2023-07-06T20:37:33.874 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2023-07-06T20:37:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2023-07-06T20:37:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 463, 2023-07-06T20:37:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.015266834, 2023-07-06T20:37:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000032973 2023-07-06T20:37:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2023-07-06T20:37:33.875 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 1412, 2023-07-06T20:37:33.876 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 793, 2023-07-06T20:37:33.876 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2023-07-06T20:37:33.876 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 793, 2023-07-06T20:37:33.876 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.378428180, 2023-07-06T20:37:33.876 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001738244 2023-07-06T20:37:33.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.876 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2023-07-06T20:37:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 619, 2023-07-06T20:37:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2023-07-06T20:37:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2023-07-06T20:37:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 11, 2023-07-06T20:37:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2023-07-06T20:37:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2023-07-06T20:37:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 542, 2023-07-06T20:37:33.877 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 6, 2023-07-06T20:37:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 536, 2023-07-06T20:37:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2023-07-06T20:37:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 533, 2023-07-06T20:37:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 0, 2023-07-06T20:37:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 508, 2023-07-06T20:37:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 508, 2023-07-06T20:37:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 15, 2023-07-06T20:37:33.878 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 2580, 2023-07-06T20:37:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 1503, 2023-07-06T20:37:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 618, 2023-07-06T20:37:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 25, 2023-07-06T20:37:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2023-07-06T20:37:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2023-07-06T20:37:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 64, 2023-07-06T20:37:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 638, 2023-07-06T20:37:33.879 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2023-07-06T20:37:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2023-07-06T20:37:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2023-07-06T20:37:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 13, 2023-07-06T20:37:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 504, 2023-07-06T20:37:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2023-07-06T20:37:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 237, 2023-07-06T20:37:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2023-07-06T20:37:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 1046, 2023-07-06T20:37:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 9135538132, 2023-07-06T20:37:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2023-07-06T20:37:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 237, 2023-07-06T20:37:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2023-07-06T20:37:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2023-07-06T20:37:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2023-07-06T20:37:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 2, 2023-07-06T20:37:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2023-07-06T20:37:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 2, 2023-07-06T20:37:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 235, 2023-07-06T20:37:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2023-07-06T20:37:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 4, 2023-07-06T20:37:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 368, 2023-07-06T20:37:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2023-07-06T20:37:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-06T20:37:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-06T20:37:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 833, 2023-07-06T20:37:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 722, 2023-07-06T20:37:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 72, 2023-07-06T20:37:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 337 2023-07-06T20:37:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2023-07-06T20:37:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 99, 2023-07-06T20:37:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 253, 2023-07-06T20:37:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 136, 2023-07-06T20:37:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 222, 2023-07-06T20:37:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 165, 2023-07-06T20:37:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2023-07-06T20:37:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2023-07-06T20:37:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2023-07-06T20:37:33.884 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 65, 2023-07-06T20:37:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 65, 2023-07-06T20:37:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2023-07-06T20:37:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2023-07-06T20:37:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2023-07-06T20:37:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2023-07-06T20:37:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2023-07-06T20:37:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2023-07-06T20:37:33.885 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2023-07-06T20:37:33.886 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2023-07-06T20:37:33.886 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2023-07-06T20:37:33.886 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2023-07-06T20:37:33.886 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2023-07-06T20:37:33.886 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2023-07-06T20:37:33.886 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2023-07-06T20:37:33.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2023-07-06T20:37:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 2126, 2023-07-06T20:37:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2023-07-06T20:37:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2023-07-06T20:37:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 2126, 2023-07-06T20:37:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2023-07-06T20:37:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2023-07-06T20:37:33.887 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 3, 2023-07-06T20:37:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2023-07-06T20:37:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2023-07-06T20:37:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 3, 2023-07-06T20:37:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2023-07-06T20:37:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2023-07-06T20:37:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2023-07-06T20:37:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 7100528, 2023-07-06T20:37:33.888 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2023-07-06T20:37:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2023-07-06T20:37:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 858, 2023-07-06T20:37:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 11.789669006, 2023-07-06T20:37:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.013740872 2023-07-06T20:37:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2023-07-06T20:37:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.889 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2023-07-06T20:37:33.890 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 536, 2023-07-06T20:37:33.890 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 601, 2023-07-06T20:37:33.890 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 65, 2023-07-06T20:37:33.890 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 57, 2023-07-06T20:37:33.890 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 146, 2023-07-06T20:37:33.890 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 89, 2023-07-06T20:37:33.890 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 542, 2023-07-06T20:37:33.890 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 615, 2023-07-06T20:37:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 73, 2023-07-06T20:37:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 508, 2023-07-06T20:37:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 1061, 2023-07-06T20:37:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 553, 2023-07-06T20:37:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 57968, 2023-07-06T20:37:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 198964 2023-07-06T20:37:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.891 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2023-07-06T20:37:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 1976, 2023-07-06T20:37:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2023-07-06T20:37:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 1412, 2023-07-06T20:37:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 124, 2023-07-06T20:37:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 187, 2023-07-06T20:37:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2023-07-06T20:37:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 69, 2023-07-06T20:37:33.892 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.362055650, 2023-07-06T20:37:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005247183 2023-07-06T20:37:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2023-07-06T20:37:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2023-07-06T20:37:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000250532, 2023-07-06T20:37:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000250532 2023-07-06T20:37:33.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2023-07-06T20:37:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2023-07-06T20:37:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.894 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.895 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.895 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2023-07-06T20:37:33.895 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 594, 2023-07-06T20:37:33.895 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.627126460, 2023-07-06T20:37:33.895 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001055768 2023-07-06T20:37:33.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.895 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2023-07-06T20:37:33.896 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.896 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.896 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.896 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.896 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2023-07-06T20:37:33.896 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2023-07-06T20:37:33.896 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000283386, 2023-07-06T20:37:33.896 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000141693 2023-07-06T20:37:33.897 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.897 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2023-07-06T20:37:33.897 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.897 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.897 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.897 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.897 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2023-07-06T20:37:33.897 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.898 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.898 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.898 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.898 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2023-07-06T20:37:33.898 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.898 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.898 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.898 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.899 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2023-07-06T20:37:33.899 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.899 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.899 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.899 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2023-07-06T20:37:33.899 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.900 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.900 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.900 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.900 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2023-07-06T20:37:33.900 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.900 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.900 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.900 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.901 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2023-07-06T20:37:33.901 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.901 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.901 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.901 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.901 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2023-07-06T20:37:33.901 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 49, 2023-07-06T20:37:33.901 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.027360726, 2023-07-06T20:37:33.902 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000558382 2023-07-06T20:37:33.902 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.902 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2023-07-06T20:37:33.902 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 13, 2023-07-06T20:37:33.902 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.012657475, 2023-07-06T20:37:33.902 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000973651 2023-07-06T20:37:33.902 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.902 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2023-07-06T20:37:33.903 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 64, 2023-07-06T20:37:33.903 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.347372156, 2023-07-06T20:37:33.903 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005427689 2023-07-06T20:37:33.903 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.903 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2023-07-06T20:37:33.903 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.903 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.903 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.904 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.904 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2023-07-06T20:37:33.904 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.904 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.904 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.904 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.904 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2023-07-06T20:37:33.904 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.905 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.905 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.905 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.905 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2023-07-06T20:37:33.905 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.905 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.905 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.906 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2023-07-06T20:37:33.906 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.906 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.906 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.906 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2023-07-06T20:37:33.906 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.907 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.907 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.907 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2023-07-06T20:37:33.907 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.907 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.907 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.908 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2023-07-06T20:37:33.908 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.908 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.908 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.908 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2023-07-06T20:37:33.908 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.908 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.909 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.909 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2023-07-06T20:37:33.909 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.909 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.909 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.909 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2023-07-06T20:37:33.910 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.910 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.910 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.910 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.910 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2023-07-06T20:37:33.910 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2023-07-06T20:37:33.910 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001321795, 2023-07-06T20:37:33.910 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001321795 2023-07-06T20:37:33.911 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.911 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2023-07-06T20:37:33.911 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2023-07-06T20:37:33.911 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2023-07-06T20:37:33.911 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.911 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.911 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.911 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:33.912 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.912 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2023-07-06T20:37:33.912 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2023-07-06T20:37:33.912 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 2, 2023-07-06T20:37:33.912 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 1, 2023-07-06T20:37:33.912 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2023-07-06T20:37:33.912 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2023-07-06T20:37:33.912 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 69, 2023-07-06T20:37:33.913 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 69, 2023-07-06T20:37:33.913 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 1104 2023-07-06T20:37:33.913 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.913 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2023-07-06T20:37:33.913 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2023-07-06T20:37:33.913 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2023-07-06T20:37:33.913 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T20:37:33.913 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2023-07-06T20:37:33.914 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T20:37:33.914 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2023-07-06T20:37:33.914 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T20:37:33.914 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T20:37:33.914 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T20:37:33.914 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T20:37:33.914 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T20:37:33.914 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2023-07-06T20:37:33.915 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T20:37:33.915 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T20:37:33.915 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2023-07-06T20:37:33.915 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2023-07-06T20:37:33.915 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2023-07-06T20:37:33.915 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2023-07-06T20:37:33.915 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T20:37:33.916 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T20:37:33.916 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T20:37:33.916 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T20:37:33.916 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T20:37:33.916 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2023-07-06T20:37:33.916 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2023-07-06T20:37:33.916 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2023-07-06T20:37:33.916 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T20:37:33.917 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T20:37:33.917 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2023-07-06T20:37:33.917 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2023-07-06T20:37:33.917 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2023-07-06T20:37:33.917 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2023-07-06T20:37:33.917 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T20:37:33.917 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2023-07-06T20:37:33.917 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T20:37:33.918 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2023-07-06T20:37:33.918 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 327819, 2023-07-06T20:37:33.918 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 654, 2023-07-06T20:37:33.918 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 1320, 2023-07-06T20:37:33.918 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 15, 2023-07-06T20:37:33.918 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2023-07-06T20:37:33.918 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2023-07-06T20:37:33.918 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2023-07-06T20:37:33.919 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2023-07-06T20:37:33.919 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2023-07-06T20:37:33.919 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2023-07-06T20:37:33.919 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 7728, 2023-07-06T20:37:33.919 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 163, 2023-07-06T20:37:33.919 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T20:37:33.919 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2023-07-06T20:37:33.919 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2023-07-06T20:37:33.920 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2023-07-06T20:37:33.920 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 1871974, 2023-07-06T20:37:33.920 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 19867, 2023-07-06T20:37:33.920 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2023-07-06T20:37:33.920 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2023-07-06T20:37:33.920 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2023-07-06T20:37:33.920 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2023-07-06T20:37:33.920 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.921 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2023-07-06T20:37:33.921 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2023-07-06T20:37:33.921 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2023-07-06T20:37:33.921 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 1610, 2023-07-06T20:37:33.921 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 3002246, 2023-07-06T20:37:33.921 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 33, 2023-07-06T20:37:33.921 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 1610, 2023-07-06T20:37:33.921 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2023-07-06T20:37:33.922 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1610, 2023-07-06T20:37:33.922 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 19.544766831, 2023-07-06T20:37:33.922 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.012139606 2023-07-06T20:37:33.922 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.922 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2023-07-06T20:37:33.922 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2023-07-06T20:37:33.922 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1610, 2023-07-06T20:37:33.923 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1800 2023-07-06T20:37:33.923 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.923 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 1610, 2023-07-06T20:37:33.923 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 302, 2023-07-06T20:37:33.923 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 1308, 2023-07-06T20:37:33.923 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2023-07-06T20:37:33.923 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2023-07-06T20:37:33.923 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2023-07-06T20:37:33.924 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 10, 2023-07-06T20:37:33.924 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 65, 2023-07-06T20:37:33.924 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 923, 2023-07-06T20:37:33.924 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 134, 2023-07-06T20:37:33.924 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2023-07-06T20:37:33.924 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2023-07-06T20:37:33.924 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2023-07-06T20:37:33.924 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2023-07-06T20:37:33.925 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 77, 2023-07-06T20:37:33.925 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2023-07-06T20:37:33.925 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2023-07-06T20:37:33.925 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2023-07-06T20:37:33.925 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 237, 2023-07-06T20:37:33.925 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 20, 2023-07-06T20:37:33.925 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2023-07-06T20:37:33.925 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2023-07-06T20:37:33.926 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2023-07-06T20:37:33.926 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2023-07-06T20:37:33.926 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2023-07-06T20:37:33.926 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2023-07-06T20:37:33.926 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2023-07-06T20:37:33.926 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2023-07-06T20:37:33.926 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2023-07-06T20:37:33.926 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2023-07-06T20:37:33.927 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2023-07-06T20:37:33.927 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2023-07-06T20:37:33.927 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2023-07-06T20:37:33.927 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2023-07-06T20:37:33.927 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2023-07-06T20:37:33.927 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2023-07-06T20:37:33.927 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2023-07-06T20:37:33.927 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2023-07-06T20:37:33.928 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2023-07-06T20:37:33.928 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2023-07-06T20:37:33.928 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2023-07-06T20:37:33.928 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2023-07-06T20:37:33.928 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2023-07-06T20:37:33.928 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2023-07-06T20:37:33.928 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2023-07-06T20:37:33.928 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2023-07-06T20:37:33.929 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 87, 2023-07-06T20:37:33.929 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2023-07-06T20:37:33.929 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 4, 2023-07-06T20:37:33.929 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 12, 2023-07-06T20:37:33.929 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 12, 2023-07-06T20:37:33.929 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2023-07-06T20:37:33.929 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2023-07-06T20:37:33.930 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 268, 2023-07-06T20:37:33.930 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2023-07-06T20:37:33.930 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 66 2023-07-06T20:37:33.930 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.930 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2023-07-06T20:37:33.930 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2023-07-06T20:37:33.930 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2023-07-06T20:37:33.930 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2023-07-06T20:37:33.931 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2023-07-06T20:37:33.931 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.931 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2023-07-06T20:37:33.931 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2023-07-06T20:37:33.931 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.931 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2023-07-06T20:37:33.931 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2023-07-06T20:37:33.931 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 2, 2023-07-06T20:37:33.932 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2023-07-06T20:37:33.932 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 2, 2023-07-06T20:37:33.932 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 65, 2023-07-06T20:37:33.932 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2023-07-06T20:37:33.932 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.932 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T20:37:33.932 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 72, 2023-07-06T20:37:33.932 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-07-06T20:37:33.933 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:33.933 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 12623455, 2023-07-06T20:37:33.933 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 911056683, 2023-07-06T20:37:33.933 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:33.933 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 12623455, 2023-07-06T20:37:33.933 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-07-06T20:37:33.933 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-07-06T20:37:33.933 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 12623454, 2023-07-06T20:37:33.934 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 911056611, 2023-07-06T20:37:33.934 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:33.934 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.934 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.934 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.934 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:33.934 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.934 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2023-07-06T20:37:33.935 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:33.935 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-07-06T20:37:33.935 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:33.935 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-07-06T20:37:33.935 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-07-06T20:37:33.935 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:33.935 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:33.935 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 1610, 2023-07-06T20:37:33.936 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 3008811, 2023-07-06T20:37:33.936 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 1610, 2023-07-06T20:37:33.936 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 3008811, 2023-07-06T20:37:33.936 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:33.936 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.936 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.936 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.937 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:33.937 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.937 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2023-07-06T20:37:33.937 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:33.937 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2023-07-06T20:37:33.937 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:33.937 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-07-06T20:37:33.937 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-07-06T20:37:33.938 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:33.938 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:33.938 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 1610, 2023-07-06T20:37:33.938 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 1610, 2023-07-06T20:37:33.938 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 1610, 2023-07-06T20:37:33.938 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1610, 2023-07-06T20:37:33.938 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:33.938 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.939 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.939 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.939 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:33.939 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.939 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2023-07-06T20:37:33.939 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:33.939 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2023-07-06T20:37:33.939 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:33.940 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 65, 2023-07-06T20:37:33.940 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 6565, 2023-07-06T20:37:33.940 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:33.940 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 65, 2023-07-06T20:37:33.940 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-07-06T20:37:33.940 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-07-06T20:37:33.940 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 65, 2023-07-06T20:37:33.941 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 6565, 2023-07-06T20:37:33.941 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:33.941 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.941 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.941 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.941 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:33.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:33.941 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55a28c9760a0": { 2023-07-06T20:37:33.942 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:33.942 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2023-07-06T20:37:33.942 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:33.942 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 2126, 2023-07-06T20:37:33.942 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 2906224, 2023-07-06T20:37:33.942 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:33.942 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 2126, 2023-07-06T20:37:33.942 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-07-06T20:37:33.943 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-07-06T20:37:33.943 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 858, 2023-07-06T20:37:33.943 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 2906224, 2023-07-06T20:37:33.943 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:33.943 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:33.943 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:33.943 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:33.943 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:33.944 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:33.944 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-07-06T20:37:34.258 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-07-06T20:37:34.259 INFO:tasks.check_counter:Daemon mds.e mds.imported=13 2023-07-06T20:37:34.259 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=99 2023-07-06T20:37:34.259 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=253 2023-07-06T20:37:34.259 DEBUG:tasks.check_counter:Getting stats from h 2023-07-06T20:37:34.259 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-06T20:37:35.009 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.b/config 2023-07-06T20:37:35.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:34 smithi156 ceph-mon[129280]: pgmap v875: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2023-07-06T20:37:35.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:34 smithi098 ceph-mon[128968]: pgmap v875: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2023-07-06T20:37:35.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:34 smithi069 ceph-mon[126399]: pgmap v875: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2023-07-06T20:37:36.590 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-07-06T20:37:36.591 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T20:37:36.591 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 11399, 2023-07-06T20:37:36.591 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 9075, 2023-07-06T20:37:36.591 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 3316353, 2023-07-06T20:37:36.591 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 15447213, 2023-07-06T20:37:36.591 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 9, 2023-07-06T20:37:36.592 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 3, 2023-07-06T20:37:36.592 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.723785722, 2023-07-06T20:37:36.592 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.265689300, 2023-07-06T20:37:36.592 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.405162796, 2023-07-06T20:37:36.592 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.153207969, 2023-07-06T20:37:36.592 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:36.592 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 9075, 2023-07-06T20:37:36.593 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.137808323, 2023-07-06T20:37:36.593 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000125378 2023-07-06T20:37:36.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.593 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:36.593 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 9818, 2023-07-06T20:37:36.593 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001854024, 2023-07-06T20:37:36.593 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000188 2023-07-06T20:37:36.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-06T20:37:36.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2023-07-06T20:37:36.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.594 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T20:37:36.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 18257, 2023-07-06T20:37:36.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 20313, 2023-07-06T20:37:36.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 5498844, 2023-07-06T20:37:36.594 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 11724184, 2023-07-06T20:37:36.595 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 13, 2023-07-06T20:37:36.595 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 6, 2023-07-06T20:37:36.595 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 1.021776944, 2023-07-06T20:37:36.595 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.420099937, 2023-07-06T20:37:36.595 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.768874556, 2023-07-06T20:37:36.595 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.013047971, 2023-07-06T20:37:36.595 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:36.595 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 20313, 2023-07-06T20:37:36.596 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 2.487950180, 2023-07-06T20:37:36.596 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000122480 2023-07-06T20:37:36.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.596 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:36.596 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 31459, 2023-07-06T20:37:36.596 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.007224347, 2023-07-06T20:37:36.596 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000229 2023-07-06T20:37:36.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.597 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 565024, 2023-07-06T20:37:36.597 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 194608 2023-07-06T20:37:36.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.597 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T20:37:36.598 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1323, 2023-07-06T20:37:36.598 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1572, 2023-07-06T20:37:36.598 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 342130, 2023-07-06T20:37:36.598 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 3783855, 2023-07-06T20:37:36.598 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 12, 2023-07-06T20:37:36.598 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 7, 2023-07-06T20:37:36.599 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.117115103, 2023-07-06T20:37:36.599 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.039782589, 2023-07-06T20:37:36.599 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.025460415, 2023-07-06T20:37:36.599 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.042451604, 2023-07-06T20:37:36.599 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:36.599 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1568, 2023-07-06T20:37:36.599 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.420666838, 2023-07-06T20:37:36.599 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000268282 2023-07-06T20:37:36.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.600 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:36.600 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2023-07-06T20:37:36.600 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000174, 2023-07-06T20:37:36.600 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000087 2023-07-06T20:37:36.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.600 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-06T20:37:36.600 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 597424 2023-07-06T20:37:36.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.601 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2023-07-06T20:37:36.601 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2023-07-06T20:37:36.601 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2023-07-06T20:37:36.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.601 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2023-07-06T20:37:36.601 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2023-07-06T20:37:36.602 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2023-07-06T20:37:36.602 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 11246, 2023-07-06T20:37:36.602 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 11.565787943, 2023-07-06T20:37:36.602 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001028435 2023-07-06T20:37:36.602 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:36.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.602 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2023-07-06T20:37:36.602 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2023-07-06T20:37:36.603 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2023-07-06T20:37:36.603 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 492, 2023-07-06T20:37:36.603 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.013940201, 2023-07-06T20:37:36.603 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000028333 2023-07-06T20:37:36.603 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:36.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.603 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2023-07-06T20:37:36.603 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 15061, 2023-07-06T20:37:36.604 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 15021, 2023-07-06T20:37:36.604 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2023-07-06T20:37:36.604 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 15021, 2023-07-06T20:37:36.604 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 94.041306075, 2023-07-06T20:37:36.604 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.006260655 2023-07-06T20:37:36.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.604 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2023-07-06T20:37:36.605 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 40, 2023-07-06T20:37:36.605 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2023-07-06T20:37:36.605 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2023-07-06T20:37:36.605 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 11, 2023-07-06T20:37:36.605 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2023-07-06T20:37:36.605 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2023-07-06T20:37:36.605 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 2341, 2023-07-06T20:37:36.605 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 9, 2023-07-06T20:37:36.606 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 2316, 2023-07-06T20:37:36.606 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 16, 2023-07-06T20:37:36.606 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 2323, 2023-07-06T20:37:36.606 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 335, 2023-07-06T20:37:36.606 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 71, 2023-07-06T20:37:36.606 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 71, 2023-07-06T20:37:36.606 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 7, 2023-07-06T20:37:36.606 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 54575, 2023-07-06T20:37:36.607 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 40853, 2023-07-06T20:37:36.607 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 39, 2023-07-06T20:37:36.607 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 88, 2023-07-06T20:37:36.607 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2023-07-06T20:37:36.607 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2023-07-06T20:37:36.607 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 9974, 2023-07-06T20:37:36.607 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 702, 2023-07-06T20:37:36.608 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2023-07-06T20:37:36.608 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 10, 2023-07-06T20:37:36.608 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 206, 2023-07-06T20:37:36.608 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 10, 2023-07-06T20:37:36.608 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 222, 2023-07-06T20:37:36.608 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2023-07-06T20:37:36.608 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 70, 2023-07-06T20:37:36.608 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2023-07-06T20:37:36.609 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 1046, 2023-07-06T20:37:36.609 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 9135538132, 2023-07-06T20:37:36.609 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2023-07-06T20:37:36.609 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 70, 2023-07-06T20:37:36.609 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2023-07-06T20:37:36.609 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2023-07-06T20:37:36.609 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2023-07-06T20:37:36.610 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 3, 2023-07-06T20:37:36.610 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2023-07-06T20:37:36.610 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 3, 2023-07-06T20:37:36.610 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 67, 2023-07-06T20:37:36.610 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2023-07-06T20:37:36.610 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 793, 2023-07-06T20:37:36.610 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 2037, 2023-07-06T20:37:36.610 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2023-07-06T20:37:36.611 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-06T20:37:36.611 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-06T20:37:36.611 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 3444, 2023-07-06T20:37:36.611 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 2093, 2023-07-06T20:37:36.611 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 139, 2023-07-06T20:37:36.611 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 6860 2023-07-06T20:37:36.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.611 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2023-07-06T20:37:36.612 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 74, 2023-07-06T20:37:36.612 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 331, 2023-07-06T20:37:36.612 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 177, 2023-07-06T20:37:36.612 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 343, 2023-07-06T20:37:36.612 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 80, 2023-07-06T20:37:36.612 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2023-07-06T20:37:36.612 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2023-07-06T20:37:36.612 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2023-07-06T20:37:36.613 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 1696, 2023-07-06T20:37:36.613 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 45, 2023-07-06T20:37:36.613 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 6623, 2023-07-06T20:37:36.614 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2023-07-06T20:37:36.614 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2023-07-06T20:37:36.615 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2023-07-06T20:37:36.615 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2023-07-06T20:37:36.615 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2023-07-06T20:37:36.615 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2023-07-06T20:37:36.616 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2023-07-06T20:37:36.616 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 10, 2023-07-06T20:37:36.617 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2023-07-06T20:37:36.617 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2023-07-06T20:37:36.617 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2023-07-06T20:37:36.617 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2023-07-06T20:37:36.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.617 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2023-07-06T20:37:36.618 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 8195, 2023-07-06T20:37:36.618 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2023-07-06T20:37:36.618 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2023-07-06T20:37:36.618 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 8195, 2023-07-06T20:37:36.618 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2023-07-06T20:37:36.619 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2023-07-06T20:37:36.619 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 11, 2023-07-06T20:37:36.619 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2023-07-06T20:37:36.619 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2023-07-06T20:37:36.619 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 11, 2023-07-06T20:37:36.619 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2023-07-06T20:37:36.620 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2023-07-06T20:37:36.620 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2023-07-06T20:37:36.620 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 19341467, 2023-07-06T20:37:36.620 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2023-07-06T20:37:36.620 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2023-07-06T20:37:36.620 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 5790, 2023-07-06T20:37:36.621 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 42.283778680, 2023-07-06T20:37:36.621 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.007302897 2023-07-06T20:37:36.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.621 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2023-07-06T20:37:36.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.621 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2023-07-06T20:37:36.621 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 111, 2023-07-06T20:37:36.622 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 419, 2023-07-06T20:37:36.622 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 308, 2023-07-06T20:37:36.622 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 53, 2023-07-06T20:37:36.622 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 184, 2023-07-06T20:37:36.622 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 131, 2023-07-06T20:37:36.622 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 2341, 2023-07-06T20:37:36.622 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 2722, 2023-07-06T20:37:36.622 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 381, 2023-07-06T20:37:36.623 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 71, 2023-07-06T20:37:36.623 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 397, 2023-07-06T20:37:36.623 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 326, 2023-07-06T20:37:36.623 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 57516, 2023-07-06T20:37:36.623 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 198964 2023-07-06T20:37:36.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.623 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2023-07-06T20:37:36.624 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 31694, 2023-07-06T20:37:36.624 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 128, 2023-07-06T20:37:36.624 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 15061, 2023-07-06T20:37:36.624 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 3154, 2023-07-06T20:37:36.624 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 237, 2023-07-06T20:37:36.625 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2023-07-06T20:37:36.625 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 96, 2023-07-06T20:37:36.625 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.210595569, 2023-07-06T20:37:36.625 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002193703 2023-07-06T20:37:36.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.625 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2023-07-06T20:37:36.625 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 21, 2023-07-06T20:37:36.625 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.034997995, 2023-07-06T20:37:36.626 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001666571 2023-07-06T20:37:36.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.626 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2023-07-06T20:37:36.626 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.626 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.626 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.627 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2023-07-06T20:37:36.627 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1651, 2023-07-06T20:37:36.627 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 4.210199820, 2023-07-06T20:37:36.627 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002550090 2023-07-06T20:37:36.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.628 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2023-07-06T20:37:36.628 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4660, 2023-07-06T20:37:36.628 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 9.056552979, 2023-07-06T20:37:36.629 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001943466 2023-07-06T20:37:36.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.629 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2023-07-06T20:37:36.629 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.630 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.630 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.631 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2023-07-06T20:37:36.631 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.632 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.632 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.632 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2023-07-06T20:37:36.633 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.633 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.633 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.634 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2023-07-06T20:37:36.634 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.635 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.635 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.635 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2023-07-06T20:37:36.636 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.636 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.636 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.637 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2023-07-06T20:37:36.637 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.637 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.637 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.638 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2023-07-06T20:37:36.638 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.638 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.639 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.639 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2023-07-06T20:37:36.639 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.639 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.640 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.640 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.640 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2023-07-06T20:37:36.640 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.641 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.641 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.641 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.641 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2023-07-06T20:37:36.641 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 207, 2023-07-06T20:37:36.641 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.477612690, 2023-07-06T20:37:36.641 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002307307 2023-07-06T20:37:36.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.642 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2023-07-06T20:37:36.642 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 67, 2023-07-06T20:37:36.642 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.162795659, 2023-07-06T20:37:36.642 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002429785 2023-07-06T20:37:36.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.643 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2023-07-06T20:37:36.643 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 6624, 2023-07-06T20:37:36.643 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 76.981353480, 2023-07-06T20:37:36.643 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.011621581 2023-07-06T20:37:36.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.643 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2023-07-06T20:37:36.644 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.644 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.644 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.644 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2023-07-06T20:37:36.645 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.645 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.645 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.645 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2023-07-06T20:37:36.645 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.646 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.646 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.646 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2023-07-06T20:37:36.646 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.646 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.646 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.647 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2023-07-06T20:37:36.647 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.647 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.647 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.648 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2023-07-06T20:37:36.648 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.648 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.648 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.648 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2023-07-06T20:37:36.648 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.648 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.649 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.649 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2023-07-06T20:37:36.649 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.649 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.649 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.650 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2023-07-06T20:37:36.650 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.650 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.650 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.651 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2023-07-06T20:37:36.651 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.651 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.651 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.652 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2023-07-06T20:37:36.652 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.652 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.652 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.652 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.652 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2023-07-06T20:37:36.652 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1695, 2023-07-06T20:37:36.653 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 2.907197883, 2023-07-06T20:37:36.653 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001715160 2023-07-06T20:37:36.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.653 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2023-07-06T20:37:36.653 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2023-07-06T20:37:36.653 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2023-07-06T20:37:36.653 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.653 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.654 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.654 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:36.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.654 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2023-07-06T20:37:36.654 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2023-07-06T20:37:36.654 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 2, 2023-07-06T20:37:36.654 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 1, 2023-07-06T20:37:36.655 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2023-07-06T20:37:36.655 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2023-07-06T20:37:36.655 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 613, 2023-07-06T20:37:36.655 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 613, 2023-07-06T20:37:36.655 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 1093 2023-07-06T20:37:36.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.655 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2023-07-06T20:37:36.655 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2023-07-06T20:37:36.656 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2023-07-06T20:37:36.656 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T20:37:36.656 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2023-07-06T20:37:36.656 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T20:37:36.656 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2023-07-06T20:37:36.656 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T20:37:36.656 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T20:37:36.657 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T20:37:36.657 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T20:37:36.657 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T20:37:36.657 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2023-07-06T20:37:36.657 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T20:37:36.657 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T20:37:36.658 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2023-07-06T20:37:36.658 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2023-07-06T20:37:36.658 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2023-07-06T20:37:36.658 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2023-07-06T20:37:36.658 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T20:37:36.658 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T20:37:36.658 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T20:37:36.659 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T20:37:36.659 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T20:37:36.659 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2023-07-06T20:37:36.659 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2023-07-06T20:37:36.659 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2023-07-06T20:37:36.659 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T20:37:36.659 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T20:37:36.659 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2023-07-06T20:37:36.660 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2023-07-06T20:37:36.660 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2023-07-06T20:37:36.660 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2023-07-06T20:37:36.660 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T20:37:36.660 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2023-07-06T20:37:36.661 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T20:37:36.661 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2023-07-06T20:37:36.661 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 292756, 2023-07-06T20:37:36.661 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 194, 2023-07-06T20:37:36.662 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 176, 2023-07-06T20:37:36.662 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 2, 2023-07-06T20:37:36.662 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2023-07-06T20:37:36.663 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2023-07-06T20:37:36.663 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2023-07-06T20:37:36.663 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2023-07-06T20:37:36.663 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2023-07-06T20:37:36.664 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2023-07-06T20:37:36.664 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 7728, 2023-07-06T20:37:36.664 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 163, 2023-07-06T20:37:36.665 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T20:37:36.665 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2023-07-06T20:37:36.665 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2023-07-06T20:37:36.666 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2023-07-06T20:37:36.666 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 2164691, 2023-07-06T20:37:36.666 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 75437, 2023-07-06T20:37:36.666 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2023-07-06T20:37:36.667 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2023-07-06T20:37:36.667 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2023-07-06T20:37:36.667 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2023-07-06T20:37:36.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.668 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2023-07-06T20:37:36.668 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2023-07-06T20:37:36.669 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2023-07-06T20:37:36.669 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 6472, 2023-07-06T20:37:36.669 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 15342822, 2023-07-06T20:37:36.669 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 27, 2023-07-06T20:37:36.669 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 6472, 2023-07-06T20:37:36.670 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2023-07-06T20:37:36.670 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 6472, 2023-07-06T20:37:36.670 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 59.949278764, 2023-07-06T20:37:36.670 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.009262867 2023-07-06T20:37:36.670 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.671 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2023-07-06T20:37:36.671 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2023-07-06T20:37:36.671 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 6472, 2023-07-06T20:37:36.671 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 6753 2023-07-06T20:37:36.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.672 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 6472, 2023-07-06T20:37:36.672 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 113, 2023-07-06T20:37:36.673 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 6359, 2023-07-06T20:37:36.673 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2023-07-06T20:37:36.673 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2023-07-06T20:37:36.673 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2023-07-06T20:37:36.673 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 10, 2023-07-06T20:37:36.673 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 43, 2023-07-06T20:37:36.673 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 5833, 2023-07-06T20:37:36.674 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 114, 2023-07-06T20:37:36.674 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2023-07-06T20:37:36.674 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2023-07-06T20:37:36.674 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2023-07-06T20:37:36.674 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2023-07-06T20:37:36.674 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 197, 2023-07-06T20:37:36.674 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2023-07-06T20:37:36.674 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2023-07-06T20:37:36.675 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2023-07-06T20:37:36.675 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 70, 2023-07-06T20:37:36.675 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 20, 2023-07-06T20:37:36.675 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2023-07-06T20:37:36.675 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2023-07-06T20:37:36.675 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2023-07-06T20:37:36.675 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2023-07-06T20:37:36.676 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2023-07-06T20:37:36.676 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2023-07-06T20:37:36.676 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2023-07-06T20:37:36.676 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2023-07-06T20:37:36.676 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2023-07-06T20:37:36.676 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2023-07-06T20:37:36.677 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2023-07-06T20:37:36.677 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2023-07-06T20:37:36.677 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2023-07-06T20:37:36.677 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2023-07-06T20:37:36.677 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2023-07-06T20:37:36.677 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2023-07-06T20:37:36.677 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2023-07-06T20:37:36.678 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2023-07-06T20:37:36.678 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2023-07-06T20:37:36.678 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2023-07-06T20:37:36.678 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2023-07-06T20:37:36.678 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2023-07-06T20:37:36.678 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2023-07-06T20:37:36.678 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2023-07-06T20:37:36.678 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2023-07-06T20:37:36.679 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2023-07-06T20:37:36.679 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 87, 2023-07-06T20:37:36.679 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2023-07-06T20:37:36.679 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 4, 2023-07-06T20:37:36.679 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 12, 2023-07-06T20:37:36.679 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 12, 2023-07-06T20:37:36.679 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2023-07-06T20:37:36.679 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2023-07-06T20:37:36.680 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 306, 2023-07-06T20:37:36.680 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2023-07-06T20:37:36.680 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 160 2023-07-06T20:37:36.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.680 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2023-07-06T20:37:36.680 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2023-07-06T20:37:36.680 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2023-07-06T20:37:36.681 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2023-07-06T20:37:36.681 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2023-07-06T20:37:36.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.681 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2023-07-06T20:37:36.681 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2023-07-06T20:37:36.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.682 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2023-07-06T20:37:36.682 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2023-07-06T20:37:36.682 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 12, 2023-07-06T20:37:36.682 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2023-07-06T20:37:36.682 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 3, 2023-07-06T20:37:36.682 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 45, 2023-07-06T20:37:36.682 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2023-07-06T20:37:36.682 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.683 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T20:37:36.683 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:36.683 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-07-06T20:37:36.683 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:36.683 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 30975, 2023-07-06T20:37:36.683 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 6347319, 2023-07-06T20:37:36.683 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:36.684 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 30975, 2023-07-06T20:37:36.684 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-07-06T20:37:36.684 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-07-06T20:37:36.684 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 30975, 2023-07-06T20:37:36.684 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 6347319, 2023-07-06T20:37:36.684 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:36.685 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.685 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.685 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.685 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:36.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.685 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2023-07-06T20:37:36.685 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:36.686 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-07-06T20:37:36.686 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:36.686 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-07-06T20:37:36.686 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-07-06T20:37:36.686 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:36.687 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:36.687 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 6472, 2023-07-06T20:37:36.687 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 15347367, 2023-07-06T20:37:36.687 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 6472, 2023-07-06T20:37:36.688 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 15347367, 2023-07-06T20:37:36.688 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:36.688 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.689 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.689 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.689 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:36.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.690 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2023-07-06T20:37:36.690 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:36.690 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2023-07-06T20:37:36.691 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:36.691 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2023-07-06T20:37:36.691 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2023-07-06T20:37:36.691 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:36.692 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:36.692 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 6472, 2023-07-06T20:37:36.692 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 6472, 2023-07-06T20:37:36.692 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 6472, 2023-07-06T20:37:36.692 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 6472, 2023-07-06T20:37:36.693 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:36.693 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.693 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.693 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.693 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:36.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.693 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2023-07-06T20:37:36.693 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:36.694 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2023-07-06T20:37:36.694 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:36.694 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 45, 2023-07-06T20:37:36.694 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 4545, 2023-07-06T20:37:36.694 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:36.694 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 45, 2023-07-06T20:37:36.694 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-07-06T20:37:36.695 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-07-06T20:37:36.695 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 43, 2023-07-06T20:37:36.695 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 4545, 2023-07-06T20:37:36.695 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:36.695 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.695 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.696 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.696 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:36.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:36.696 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55aa3c3ca0a0": { 2023-07-06T20:37:36.696 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:36.696 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2023-07-06T20:37:36.697 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:36.697 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 8195, 2023-07-06T20:37:36.697 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 15147163, 2023-07-06T20:37:36.697 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:36.697 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 8195, 2023-07-06T20:37:36.698 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-07-06T20:37:36.698 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-07-06T20:37:36.698 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 5790, 2023-07-06T20:37:36.698 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 15147163, 2023-07-06T20:37:36.699 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:36.699 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:36.699 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:36.699 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:36.700 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:36.700 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:36.700 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-07-06T20:37:36.875 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:36 smithi098 ceph-mon[128968]: pgmap v876: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2023-07-06T20:37:37.013 INFO:tasks.check_counter:Daemon mds.h mds.exported=10 2023-07-06T20:37:37.014 INFO:tasks.check_counter:Daemon mds.h mds.imported=10 2023-07-06T20:37:37.014 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=74 2023-07-06T20:37:37.014 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=331 2023-07-06T20:37:37.014 DEBUG:tasks.check_counter:Getting stats from k 2023-07-06T20:37:37.014 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-06T20:37:37.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:36 smithi156 ceph-mon[129280]: pgmap v876: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2023-07-06T20:37:37.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:36 smithi069 ceph-mon[126399]: pgmap v876: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2023-07-06T20:37:37.666 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.b/config 2023-07-06T20:37:39.102 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:38 smithi098 ceph-mon[128968]: pgmap v877: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-07-06T20:37:39.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:38 smithi156 ceph-mon[129280]: pgmap v877: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-07-06T20:37:39.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:38 smithi069 ceph-mon[126399]: pgmap v877: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-07-06T20:37:39.455 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-07-06T20:37:39.456 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T20:37:39.456 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 0, 2023-07-06T20:37:39.456 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2, 2023-07-06T20:37:39.456 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 0, 2023-07-06T20:37:39.456 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 0, 2023-07-06T20:37:39.457 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:39.457 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 0, 2023-07-06T20:37:39.457 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.001228123, 2023-07-06T20:37:39.457 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.000000000, 2023-07-06T20:37:39.457 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.000363674, 2023-07-06T20:37:39.458 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-06T20:37:39.458 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:39.458 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:39.458 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:39.458 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:39.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:39.458 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:39.458 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:39.459 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:39.459 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:39.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:39.459 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-06T20:37:39.459 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2023-07-06T20:37:39.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:39.459 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T20:37:39.459 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 6, 2023-07-06T20:37:39.460 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 255, 2023-07-06T20:37:39.460 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 3312, 2023-07-06T20:37:39.460 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 196720, 2023-07-06T20:37:39.460 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:39.460 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2023-07-06T20:37:39.460 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.011816986, 2023-07-06T20:37:39.460 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.008336717, 2023-07-06T20:37:39.461 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.001609366, 2023-07-06T20:37:39.461 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-06T20:37:39.461 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:39.461 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 254, 2023-07-06T20:37:39.461 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.060763625, 2023-07-06T20:37:39.461 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000239226 2023-07-06T20:37:39.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:39.462 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:39.462 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:39.462 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:39.462 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:39.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:39.462 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-07-06T20:37:39.462 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 196720 2023-07-06T20:37:39.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:39.463 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T20:37:39.463 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 333, 2023-07-06T20:37:39.463 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 330, 2023-07-06T20:37:39.463 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 504240, 2023-07-06T20:37:39.463 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 189120, 2023-07-06T20:37:39.463 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 2, 2023-07-06T20:37:39.463 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 1, 2023-07-06T20:37:39.463 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.043340139, 2023-07-06T20:37:39.464 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.015413564, 2023-07-06T20:37:39.464 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.017501840, 2023-07-06T20:37:39.464 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.009372950, 2023-07-06T20:37:39.464 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:39.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 329, 2023-07-06T20:37:39.464 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.087510843, 2023-07-06T20:37:39.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000265990 2023-07-06T20:37:39.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:39.465 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:39.465 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:39.465 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:39.465 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:39.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:39.465 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 504240, 2023-07-06T20:37:39.465 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 189120 2023-07-06T20:37:39.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:39.466 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2023-07-06T20:37:39.466 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 0, 2023-07-06T20:37:39.466 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2023-07-06T20:37:39.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:39.466 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2023-07-06T20:37:39.466 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2023-07-06T20:37:39.466 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2023-07-06T20:37:39.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T20:37:39.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2023-07-06T20:37:39.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T20:37:39.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2023-07-06T20:37:39.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T20:37:39.467 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T20:37:39.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T20:37:39.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T20:37:39.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T20:37:39.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2023-07-06T20:37:39.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T20:37:39.468 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T20:37:39.469 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2023-07-06T20:37:39.469 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2023-07-06T20:37:39.469 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2023-07-06T20:37:39.469 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2023-07-06T20:37:39.469 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T20:37:39.469 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T20:37:39.469 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T20:37:39.469 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T20:37:39.470 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T20:37:39.470 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2023-07-06T20:37:39.470 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2023-07-06T20:37:39.470 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2023-07-06T20:37:39.470 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T20:37:39.470 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T20:37:39.470 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2023-07-06T20:37:39.470 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2023-07-06T20:37:39.471 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2023-07-06T20:37:39.471 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2023-07-06T20:37:39.471 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T20:37:39.471 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2023-07-06T20:37:39.471 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T20:37:39.471 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2023-07-06T20:37:39.471 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 148761, 2023-07-06T20:37:39.471 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 31, 2023-07-06T20:37:39.472 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2023-07-06T20:37:39.472 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2023-07-06T20:37:39.472 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2023-07-06T20:37:39.472 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2023-07-06T20:37:39.472 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2023-07-06T20:37:39.472 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2023-07-06T20:37:39.472 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2023-07-06T20:37:39.473 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2023-07-06T20:37:39.473 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 0, 2023-07-06T20:37:39.473 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 0, 2023-07-06T20:37:39.473 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T20:37:39.473 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2023-07-06T20:37:39.473 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2023-07-06T20:37:39.473 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2023-07-06T20:37:39.473 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 928, 2023-07-06T20:37:39.474 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 2, 2023-07-06T20:37:39.474 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2023-07-06T20:37:39.474 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2023-07-06T20:37:39.474 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2023-07-06T20:37:39.474 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2023-07-06T20:37:39.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2023-07-06T20:37:39.474 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T20:37:39.474 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2023-07-06T20:37:39.475 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2023-07-06T20:37:39.475 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2023-07-06T20:37:39.475 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 335, 2023-07-06T20:37:39.475 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 457146, 2023-07-06T20:37:39.475 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:39.475 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 335, 2023-07-06T20:37:39.475 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2023-07-06T20:37:39.475 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2023-07-06T20:37:39.476 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 335, 2023-07-06T20:37:39.476 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 457146, 2023-07-06T20:37:39.476 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2023-07-06T20:37:39.476 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2023-07-06T20:37:39.476 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2023-07-06T20:37:39.476 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2023-07-06T20:37:39.476 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:39.477 INFO:teuthology.orchestra.run.smithi098.stdout: } 2023-07-06T20:37:39.477 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-07-06T20:37:39.871 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-07-06T20:37:39.871 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-07-06T20:37:39.872 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-07-06T20:37:39.872 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-07-06T20:37:39.872 DEBUG:tasks.check_counter:Getting stats from c 2023-07-06T20:37:39.873 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-06T20:37:40.448 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:37:40.954 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:40 smithi156 ceph-mon[129280]: pgmap v878: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2023-07-06T20:37:41.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:40 smithi098 ceph-mon[128968]: pgmap v878: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2023-07-06T20:37:41.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:40 smithi069 ceph-mon[126399]: pgmap v878: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2023-07-06T20:37:41.688 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-07-06T20:37:41.689 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T20:37:41.689 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1519, 2023-07-06T20:37:41.689 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1506, 2023-07-06T20:37:41.689 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 948672, 2023-07-06T20:37:41.689 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 553031, 2023-07-06T20:37:41.690 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 5, 2023-07-06T20:37:41.690 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 1, 2023-07-06T20:37:41.690 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.172493855, 2023-07-06T20:37:41.690 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.050592129, 2023-07-06T20:37:41.690 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.050627475, 2023-07-06T20:37:41.690 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.057307595, 2023-07-06T20:37:41.690 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:41.690 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1505, 2023-07-06T20:37:41.691 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.406932072, 2023-07-06T20:37:41.691 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000270386 2023-07-06T20:37:41.691 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.691 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:41.691 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.691 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.691 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.692 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.692 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 555888, 2023-07-06T20:37:41.692 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 189472 2023-07-06T20:37:41.692 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.692 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T20:37:41.692 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 711, 2023-07-06T20:37:41.692 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 960, 2023-07-06T20:37:41.692 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 2743487, 2023-07-06T20:37:41.693 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 724952, 2023-07-06T20:37:41.693 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 5, 2023-07-06T20:37:41.693 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2023-07-06T20:37:41.693 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.077202374, 2023-07-06T20:37:41.693 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.022991987, 2023-07-06T20:37:41.693 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.019947824, 2023-07-06T20:37:41.694 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.027281906, 2023-07-06T20:37:41.694 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:41.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 958, 2023-07-06T20:37:41.694 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.255141999, 2023-07-06T20:37:41.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000266327 2023-07-06T20:37:41.694 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.695 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:41.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.695 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.696 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.696 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-06T20:37:41.696 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 504080 2023-07-06T20:37:41.696 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.697 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T20:37:41.697 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1895, 2023-07-06T20:37:41.697 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1895, 2023-07-06T20:37:41.698 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 13068738, 2023-07-06T20:37:41.698 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 588556, 2023-07-06T20:37:41.698 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:41.698 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2023-07-06T20:37:41.698 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.149120167, 2023-07-06T20:37:41.699 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.042950417, 2023-07-06T20:37:41.699 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.041243871, 2023-07-06T20:37:41.699 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.054722475, 2023-07-06T20:37:41.699 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:41.699 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1894, 2023-07-06T20:37:41.699 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.281414682, 2023-07-06T20:37:41.699 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000148582 2023-07-06T20:37:41.700 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.700 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:41.700 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.700 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.700 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.701 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.701 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-06T20:37:41.701 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 448 2023-07-06T20:37:41.701 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.701 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-07-06T20:37:41.701 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-07-06T20:37:41.702 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-07-06T20:37:41.702 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.702 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-07-06T20:37:41.702 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-07-06T20:37:41.702 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-07-06T20:37:41.702 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6397, 2023-07-06T20:37:41.702 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.165412503, 2023-07-06T20:37:41.702 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000025857 2023-07-06T20:37:41.703 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:41.703 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.703 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-07-06T20:37:41.703 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-07-06T20:37:41.703 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-07-06T20:37:41.704 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.704 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.704 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.704 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:41.705 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.705 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-07-06T20:37:41.705 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2023-07-06T20:37:41.705 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2023-07-06T20:37:41.705 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-07-06T20:37:41.706 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.706 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.706 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.706 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.706 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-07-06T20:37:41.707 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2023-07-06T20:37:41.707 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2023-07-06T20:37:41.707 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-07-06T20:37:41.707 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2023-07-06T20:37:41.707 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2023-07-06T20:37:41.707 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-07-06T20:37:41.708 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 2362, 2023-07-06T20:37:41.708 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 0, 2023-07-06T20:37:41.708 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 0, 2023-07-06T20:37:41.708 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 2362, 2023-07-06T20:37:41.708 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 2362, 2023-07-06T20:37:41.708 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 0, 2023-07-06T20:37:41.708 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-07-06T20:37:41.709 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-07-06T20:37:41.709 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 3, 2023-07-06T20:37:41.709 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2023-07-06T20:37:41.709 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2023-07-06T20:37:41.709 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2023-07-06T20:37:41.709 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2023-07-06T20:37:41.710 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2023-07-06T20:37:41.710 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-07-06T20:37:41.710 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2023-07-06T20:37:41.710 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-07-06T20:37:41.710 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-07-06T20:37:41.710 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2023-07-06T20:37:41.710 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2023-07-06T20:37:41.711 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2023-07-06T20:37:41.711 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2023-07-06T20:37:41.711 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-07-06T20:37:41.711 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2023-07-06T20:37:41.711 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-07-06T20:37:41.711 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2023-07-06T20:37:41.711 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2023-07-06T20:37:41.712 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2023-07-06T20:37:41.712 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2023-07-06T20:37:41.712 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-07-06T20:37:41.712 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-07-06T20:37:41.712 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-07-06T20:37:41.712 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2023-07-06T20:37:41.712 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-07-06T20:37:41.713 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2023-07-06T20:37:41.713 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2023-07-06T20:37:41.713 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2023-07-06T20:37:41.713 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2023-07-06T20:37:41.713 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2023-07-06T20:37:41.713 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-07-06T20:37:41.713 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-06T20:37:41.714 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-06T20:37:41.714 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2023-07-06T20:37:41.714 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2023-07-06T20:37:41.714 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2023-07-06T20:37:41.714 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2023-07-06T20:37:41.714 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.714 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-07-06T20:37:41.715 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-07-06T20:37:41.715 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2023-07-06T20:37:41.715 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2023-07-06T20:37:41.715 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2023-07-06T20:37:41.715 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2023-07-06T20:37:41.715 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2023-07-06T20:37:41.715 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-07-06T20:37:41.716 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-07-06T20:37:41.716 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2023-07-06T20:37:41.716 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2023-07-06T20:37:41.716 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-07-06T20:37:41.716 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-07-06T20:37:41.717 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-07-06T20:37:41.717 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-07-06T20:37:41.717 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-07-06T20:37:41.717 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-07-06T20:37:41.718 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-07-06T20:37:41.718 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-07-06T20:37:41.718 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2023-07-06T20:37:41.718 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-07-06T20:37:41.719 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2023-07-06T20:37:41.719 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-07-06T20:37:41.719 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-07-06T20:37:41.719 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.720 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-07-06T20:37:41.720 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2023-07-06T20:37:41.720 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-07-06T20:37:41.721 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-07-06T20:37:41.721 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 0, 2023-07-06T20:37:41.721 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-07-06T20:37:41.721 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-07-06T20:37:41.722 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2023-07-06T20:37:41.722 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-07-06T20:37:41.722 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-07-06T20:37:41.722 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 11, 2023-07-06T20:37:41.722 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-07-06T20:37:41.722 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-07-06T20:37:41.723 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2023-07-06T20:37:41.723 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 0, 2023-07-06T20:37:41.723 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 19340893, 2023-07-06T20:37:41.723 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-07-06T20:37:41.723 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.723 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.723 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.724 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.724 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 8195 2023-07-06T20:37:41.724 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.724 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-07-06T20:37:41.724 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 85, 2023-07-06T20:37:41.724 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 337, 2023-07-06T20:37:41.724 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 252, 2023-07-06T20:37:41.725 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 20, 2023-07-06T20:37:41.725 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 34, 2023-07-06T20:37:41.725 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 14, 2023-07-06T20:37:41.725 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 2362, 2023-07-06T20:37:41.725 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 2576, 2023-07-06T20:37:41.725 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 214, 2023-07-06T20:37:41.725 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-07-06T20:37:41.725 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2023-07-06T20:37:41.726 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2023-07-06T20:37:41.726 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 50892, 2023-07-06T20:37:41.726 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207156 2023-07-06T20:37:41.726 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.726 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-07-06T20:37:41.726 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2023-07-06T20:37:41.726 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2023-07-06T20:37:41.726 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2023-07-06T20:37:41.727 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2023-07-06T20:37:41.727 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2023-07-06T20:37:41.727 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-07-06T20:37:41.727 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.727 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.727 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.727 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.728 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-07-06T20:37:41.728 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.728 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.728 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.728 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.728 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-07-06T20:37:41.728 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.728 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.729 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.729 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.729 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-07-06T20:37:41.729 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.729 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.729 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.729 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.730 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-07-06T20:37:41.730 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.730 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.730 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.730 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.730 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-07-06T20:37:41.730 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.730 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.731 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.731 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.731 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-07-06T20:37:41.731 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.731 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.731 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.731 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.731 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-07-06T20:37:41.732 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.732 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.732 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.732 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.732 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-07-06T20:37:41.732 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.732 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.733 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.733 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.733 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-07-06T20:37:41.733 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.733 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.733 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.733 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.734 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-07-06T20:37:41.734 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.734 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.734 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.734 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.734 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-07-06T20:37:41.734 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.734 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.735 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.735 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.735 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-07-06T20:37:41.735 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.735 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.735 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.735 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.735 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-07-06T20:37:41.736 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.736 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.736 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.736 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.736 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-07-06T20:37:41.736 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.736 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.737 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.737 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.737 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-07-06T20:37:41.737 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.737 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.737 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.737 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.737 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-07-06T20:37:41.738 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.738 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.738 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.738 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.738 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-07-06T20:37:41.738 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.738 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.738 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.739 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.739 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-07-06T20:37:41.739 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.739 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.739 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.739 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.739 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-07-06T20:37:41.740 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.740 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.740 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.740 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.740 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-07-06T20:37:41.740 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.740 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.740 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.741 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.741 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-07-06T20:37:41.741 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.741 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.741 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.741 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.741 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-07-06T20:37:41.742 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.742 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.742 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.742 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.742 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-07-06T20:37:41.742 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.742 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.742 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.743 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.743 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-07-06T20:37:41.743 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.743 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.743 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.743 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.743 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-07-06T20:37:41.743 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.744 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.744 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.744 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-07-06T20:37:41.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.744 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.745 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.745 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-07-06T20:37:41.745 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.745 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.745 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.745 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.745 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-07-06T20:37:41.745 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.746 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.746 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.746 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.746 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-07-06T20:37:41.746 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-07-06T20:37:41.746 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-07-06T20:37:41.746 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.746 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.747 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.747 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:41.747 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.747 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-07-06T20:37:41.747 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 1, 2023-07-06T20:37:41.747 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 2, 2023-07-06T20:37:41.747 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 1, 2023-07-06T20:37:41.748 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 1, 2023-07-06T20:37:41.748 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-07-06T20:37:41.748 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2023-07-06T20:37:41.748 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2023-07-06T20:37:41.748 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 3321 2023-07-06T20:37:41.748 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.748 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-07-06T20:37:41.748 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-07-06T20:37:41.749 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-07-06T20:37:41.749 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T20:37:41.749 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-07-06T20:37:41.749 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T20:37:41.749 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-07-06T20:37:41.749 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T20:37:41.749 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T20:37:41.749 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T20:37:41.750 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T20:37:41.750 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T20:37:41.750 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-07-06T20:37:41.750 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T20:37:41.750 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T20:37:41.750 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-07-06T20:37:41.751 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-07-06T20:37:41.751 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-07-06T20:37:41.751 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-07-06T20:37:41.751 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T20:37:41.751 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T20:37:41.751 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T20:37:41.752 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T20:37:41.752 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T20:37:41.752 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-07-06T20:37:41.752 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-07-06T20:37:41.752 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-07-06T20:37:41.752 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T20:37:41.752 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T20:37:41.753 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-07-06T20:37:41.753 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-07-06T20:37:41.753 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-07-06T20:37:41.753 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-07-06T20:37:41.753 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T20:37:41.753 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-07-06T20:37:41.753 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T20:37:41.754 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-07-06T20:37:41.754 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 192723, 2023-07-06T20:37:41.754 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 121, 2023-07-06T20:37:41.754 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-07-06T20:37:41.754 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-07-06T20:37:41.754 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-07-06T20:37:41.754 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-07-06T20:37:41.754 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-07-06T20:37:41.755 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-07-06T20:37:41.755 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-07-06T20:37:41.755 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-07-06T20:37:41.755 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 7728, 2023-07-06T20:37:41.755 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 163, 2023-07-06T20:37:41.755 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T20:37:41.755 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-07-06T20:37:41.755 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-07-06T20:37:41.756 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-07-06T20:37:41.756 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 1929425, 2023-07-06T20:37:41.756 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 59664, 2023-07-06T20:37:41.756 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-07-06T20:37:41.756 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-07-06T20:37:41.756 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-07-06T20:37:41.757 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-07-06T20:37:41.757 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.757 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-07-06T20:37:41.757 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-07-06T20:37:41.757 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-07-06T20:37:41.758 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 3770, 2023-07-06T20:37:41.758 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21, 2023-07-06T20:37:41.758 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2023-07-06T20:37:41.758 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 3770, 2023-07-06T20:37:41.758 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-07-06T20:37:41.758 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3770, 2023-07-06T20:37:41.759 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 47.060323629, 2023-07-06T20:37:41.759 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.012482844 2023-07-06T20:37:41.759 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.759 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-07-06T20:37:41.759 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-07-06T20:37:41.760 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3770, 2023-07-06T20:37:41.760 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3771 2023-07-06T20:37:41.760 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.760 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 3770, 2023-07-06T20:37:41.760 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 3770, 2023-07-06T20:37:41.761 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2023-07-06T20:37:41.761 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-07-06T20:37:41.761 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-07-06T20:37:41.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 2253, 2023-07-06T20:37:41.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2023-07-06T20:37:41.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 1515, 2023-07-06T20:37:41.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2023-07-06T20:37:41.762 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2023-07-06T20:37:41.762 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-07-06T20:37:41.762 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-07-06T20:37:41.762 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-07-06T20:37:41.762 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2023-07-06T20:37:41.762 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2023-07-06T20:37:41.762 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-07-06T20:37:41.763 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-07-06T20:37:41.763 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-07-06T20:37:41.763 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1, 2023-07-06T20:37:41.763 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2023-07-06T20:37:41.763 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-07-06T20:37:41.763 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-07-06T20:37:41.763 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-07-06T20:37:41.763 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-07-06T20:37:41.764 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-07-06T20:37:41.764 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-07-06T20:37:41.764 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-07-06T20:37:41.764 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-07-06T20:37:41.764 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-07-06T20:37:41.764 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-07-06T20:37:41.765 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-07-06T20:37:41.765 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-07-06T20:37:41.765 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-07-06T20:37:41.765 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-07-06T20:37:41.766 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-07-06T20:37:41.766 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-07-06T20:37:41.766 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-07-06T20:37:41.766 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-07-06T20:37:41.766 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-07-06T20:37:41.766 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-07-06T20:37:41.766 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-07-06T20:37:41.767 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-07-06T20:37:41.767 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-07-06T20:37:41.767 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-07-06T20:37:41.767 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-07-06T20:37:41.767 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-07-06T20:37:41.767 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 87, 2023-07-06T20:37:41.767 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-07-06T20:37:41.768 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 4, 2023-07-06T20:37:41.768 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 5, 2023-07-06T20:37:41.768 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 5, 2023-07-06T20:37:41.768 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-07-06T20:37:41.768 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-07-06T20:37:41.768 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2023-07-06T20:37:41.768 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2023-07-06T20:37:41.769 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2023-07-06T20:37:41.769 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.769 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-07-06T20:37:41.769 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2023-07-06T20:37:41.769 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-07-06T20:37:41.769 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-07-06T20:37:41.769 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-07-06T20:37:41.770 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.770 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-07-06T20:37:41.770 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2023-07-06T20:37:41.770 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.770 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-07-06T20:37:41.770 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-07-06T20:37:41.771 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2023-07-06T20:37:41.771 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-07-06T20:37:41.771 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2023-07-06T20:37:41.771 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2023-07-06T20:37:41.771 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-07-06T20:37:41.771 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.772 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T20:37:41.772 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:41.772 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-06T20:37:41.772 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:41.772 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 4121, 2023-07-06T20:37:41.773 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 16369044, 2023-07-06T20:37:41.773 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:41.773 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 4121, 2023-07-06T20:37:41.773 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-06T20:37:41.773 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-06T20:37:41.774 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 4121, 2023-07-06T20:37:41.774 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 16369044, 2023-07-06T20:37:41.774 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:41.774 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.774 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.774 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.774 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:41.774 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.775 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-07-06T20:37:41.775 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:41.775 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-06T20:37:41.775 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:41.775 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-06T20:37:41.776 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-06T20:37:41.776 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:41.776 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:41.776 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 3770, 2023-07-06T20:37:41.777 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 15149403, 2023-07-06T20:37:41.777 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 3770, 2023-07-06T20:37:41.777 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 15149403, 2023-07-06T20:37:41.777 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:41.777 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.777 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.777 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.778 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:41.778 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.778 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-07-06T20:37:41.778 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:41.778 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-07-06T20:37:41.779 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:41.779 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-06T20:37:41.779 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-06T20:37:41.779 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:41.779 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:41.780 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 3770, 2023-07-06T20:37:41.780 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 3770, 2023-07-06T20:37:41.780 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 3770, 2023-07-06T20:37:41.780 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 3770, 2023-07-06T20:37:41.780 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:41.780 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.780 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.781 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.781 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:41.781 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.781 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-07-06T20:37:41.781 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:41.781 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-07-06T20:37:41.782 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:41.782 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-06T20:37:41.782 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-06T20:37:41.782 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:41.782 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:41.783 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-06T20:37:41.783 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-06T20:37:41.783 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-07-06T20:37:41.783 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-07-06T20:37:41.783 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:41.783 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.783 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.783 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.784 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:41.784 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:41.784 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55e9cff6c460": { 2023-07-06T20:37:41.784 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:41.784 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-07-06T20:37:41.784 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:41.784 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-06T20:37:41.785 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-06T20:37:41.785 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:41.785 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:41.785 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-06T20:37:41.785 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-06T20:37:41.785 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-07-06T20:37:41.785 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-07-06T20:37:41.785 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:41.786 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:41.786 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:41.786 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:41.786 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:41.786 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:41.786 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-07-06T20:37:42.089 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-07-06T20:37:42.090 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-07-06T20:37:42.090 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-07-06T20:37:42.090 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-07-06T20:37:42.090 DEBUG:tasks.check_counter:Getting stats from f 2023-07-06T20:37:42.091 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-06T20:37:42.638 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:37:43.098 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:42 smithi156 ceph-mon[129280]: pgmap v879: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2023-07-06T20:37:43.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:42 smithi098 ceph-mon[128968]: pgmap v879: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2023-07-06T20:37:43.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:42 smithi069 ceph-mon[126399]: pgmap v879: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2023-07-06T20:37:44.008 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-07-06T20:37:44.008 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T20:37:44.008 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1518, 2023-07-06T20:37:44.008 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1519, 2023-07-06T20:37:44.008 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 3299417, 2023-07-06T20:37:44.009 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 471851, 2023-07-06T20:37:44.009 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 7, 2023-07-06T20:37:44.009 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2023-07-06T20:37:44.009 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.129433097, 2023-07-06T20:37:44.009 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.031370678, 2023-07-06T20:37:44.009 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.038109445, 2023-07-06T20:37:44.009 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.047826099, 2023-07-06T20:37:44.009 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:44.010 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1517, 2023-07-06T20:37:44.010 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.317184866, 2023-07-06T20:37:44.010 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000209086 2023-07-06T20:37:44.010 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.010 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:44.010 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.010 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.011 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.011 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.011 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-06T20:37:44.011 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-07-06T20:37:44.011 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.011 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T20:37:44.011 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1089, 2023-07-06T20:37:44.011 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1337, 2023-07-06T20:37:44.012 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 269752, 2023-07-06T20:37:44.012 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 926546, 2023-07-06T20:37:44.012 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-07-06T20:37:44.012 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2023-07-06T20:37:44.012 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.110803173, 2023-07-06T20:37:44.012 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.031086923, 2023-07-06T20:37:44.012 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.024696246, 2023-07-06T20:37:44.012 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.045117826, 2023-07-06T20:37:44.013 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:44.013 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1336, 2023-07-06T20:37:44.013 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.277197054, 2023-07-06T20:37:44.013 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000207482 2023-07-06T20:37:44.013 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.013 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:44.013 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.013 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.014 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.014 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.014 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-07-06T20:37:44.014 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 589728 2023-07-06T20:37:44.014 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.014 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T20:37:44.014 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1513, 2023-07-06T20:37:44.015 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1503, 2023-07-06T20:37:44.015 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 938064, 2023-07-06T20:37:44.015 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 552379, 2023-07-06T20:37:44.015 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 3, 2023-07-06T20:37:44.015 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2023-07-06T20:37:44.015 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.171286143, 2023-07-06T20:37:44.015 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.050583612, 2023-07-06T20:37:44.015 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.051063974, 2023-07-06T20:37:44.016 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.057271898, 2023-07-06T20:37:44.016 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:44.016 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1502, 2023-07-06T20:37:44.016 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.507859192, 2023-07-06T20:37:44.016 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000338121 2023-07-06T20:37:44.016 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.016 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:44.016 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.017 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.017 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.017 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.017 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 546624, 2023-07-06T20:37:44.017 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 190064 2023-07-06T20:37:44.017 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.017 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-07-06T20:37:44.017 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-07-06T20:37:44.018 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-07-06T20:37:44.018 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.018 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-07-06T20:37:44.018 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-07-06T20:37:44.018 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-07-06T20:37:44.018 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6377, 2023-07-06T20:37:44.018 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.174044729, 2023-07-06T20:37:44.019 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000027292 2023-07-06T20:37:44.019 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:44.019 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.019 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-07-06T20:37:44.019 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-07-06T20:37:44.019 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-07-06T20:37:44.019 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.019 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.020 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.020 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:44.020 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.020 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-07-06T20:37:44.020 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2023-07-06T20:37:44.020 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2023-07-06T20:37:44.020 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-07-06T20:37:44.020 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.021 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.021 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.021 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.021 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-07-06T20:37:44.021 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2023-07-06T20:37:44.021 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2023-07-06T20:37:44.021 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-07-06T20:37:44.021 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2023-07-06T20:37:44.022 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2023-07-06T20:37:44.022 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-07-06T20:37:44.022 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 591, 2023-07-06T20:37:44.022 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 0, 2023-07-06T20:37:44.022 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 0, 2023-07-06T20:37:44.022 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 591, 2023-07-06T20:37:44.022 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 591, 2023-07-06T20:37:44.022 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 0, 2023-07-06T20:37:44.023 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-07-06T20:37:44.023 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-07-06T20:37:44.023 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 15, 2023-07-06T20:37:44.023 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2023-07-06T20:37:44.023 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2023-07-06T20:37:44.023 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2023-07-06T20:37:44.023 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2023-07-06T20:37:44.023 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2023-07-06T20:37:44.024 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-07-06T20:37:44.024 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2023-07-06T20:37:44.024 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-07-06T20:37:44.024 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-07-06T20:37:44.024 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2023-07-06T20:37:44.024 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2023-07-06T20:37:44.024 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2023-07-06T20:37:44.025 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2023-07-06T20:37:44.025 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-07-06T20:37:44.025 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2023-07-06T20:37:44.025 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-07-06T20:37:44.025 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2023-07-06T20:37:44.025 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2023-07-06T20:37:44.025 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2023-07-06T20:37:44.025 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2023-07-06T20:37:44.026 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-07-06T20:37:44.026 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-07-06T20:37:44.026 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-07-06T20:37:44.026 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2023-07-06T20:37:44.026 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-07-06T20:37:44.026 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2023-07-06T20:37:44.026 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2023-07-06T20:37:44.026 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2023-07-06T20:37:44.027 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2023-07-06T20:37:44.027 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2023-07-06T20:37:44.027 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-07-06T20:37:44.027 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-06T20:37:44.027 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-06T20:37:44.027 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2023-07-06T20:37:44.027 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2023-07-06T20:37:44.027 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2023-07-06T20:37:44.028 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2023-07-06T20:37:44.028 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.028 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-07-06T20:37:44.028 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-07-06T20:37:44.028 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2023-07-06T20:37:44.028 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2023-07-06T20:37:44.028 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2023-07-06T20:37:44.028 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2023-07-06T20:37:44.029 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2023-07-06T20:37:44.029 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-07-06T20:37:44.029 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-07-06T20:37:44.029 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2023-07-06T20:37:44.029 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2023-07-06T20:37:44.029 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-07-06T20:37:44.029 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-07-06T20:37:44.029 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-07-06T20:37:44.030 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-07-06T20:37:44.030 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-07-06T20:37:44.030 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-07-06T20:37:44.030 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-07-06T20:37:44.030 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-07-06T20:37:44.030 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2023-07-06T20:37:44.030 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-07-06T20:37:44.031 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2023-07-06T20:37:44.031 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-07-06T20:37:44.031 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-07-06T20:37:44.031 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.031 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-07-06T20:37:44.031 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2023-07-06T20:37:44.031 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-07-06T20:37:44.031 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-07-06T20:37:44.032 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 0, 2023-07-06T20:37:44.032 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-07-06T20:37:44.032 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-07-06T20:37:44.032 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2023-07-06T20:37:44.032 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-07-06T20:37:44.032 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-07-06T20:37:44.032 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 3, 2023-07-06T20:37:44.032 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-07-06T20:37:44.033 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-07-06T20:37:44.033 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2023-07-06T20:37:44.033 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 0, 2023-07-06T20:37:44.033 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 7099954, 2023-07-06T20:37:44.033 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-07-06T20:37:44.033 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.033 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.033 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.034 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.034 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 2126 2023-07-06T20:37:44.034 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.034 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-07-06T20:37:44.034 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 520, 2023-07-06T20:37:44.034 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 585, 2023-07-06T20:37:44.034 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 65, 2023-07-06T20:37:44.034 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 28, 2023-07-06T20:37:44.035 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 28, 2023-07-06T20:37:44.035 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 0, 2023-07-06T20:37:44.035 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 591, 2023-07-06T20:37:44.035 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 591, 2023-07-06T20:37:44.035 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 0, 2023-07-06T20:37:44.035 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-07-06T20:37:44.035 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2023-07-06T20:37:44.035 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2023-07-06T20:37:44.036 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 50292, 2023-07-06T20:37:44.036 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207156 2023-07-06T20:37:44.036 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.036 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-07-06T20:37:44.036 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2023-07-06T20:37:44.036 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2023-07-06T20:37:44.036 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2023-07-06T20:37:44.036 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2023-07-06T20:37:44.037 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2023-07-06T20:37:44.037 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-07-06T20:37:44.037 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.037 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.037 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.037 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.037 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-07-06T20:37:44.037 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.038 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.038 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.038 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.038 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-07-06T20:37:44.038 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.038 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.038 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.039 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.039 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-07-06T20:37:44.039 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.039 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.039 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.039 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.039 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-07-06T20:37:44.039 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.040 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.040 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.040 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.040 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-07-06T20:37:44.040 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.040 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.040 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.040 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.041 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-07-06T20:37:44.041 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.041 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.041 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.041 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.041 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-07-06T20:37:44.041 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.041 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.042 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.042 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.042 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-07-06T20:37:44.042 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.042 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.042 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.042 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.042 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-07-06T20:37:44.043 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.043 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.043 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.043 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.043 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-07-06T20:37:44.043 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.043 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.043 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.044 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.044 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-07-06T20:37:44.044 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.044 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.044 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.044 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.044 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-07-06T20:37:44.045 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.045 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.045 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.045 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.045 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-07-06T20:37:44.045 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.045 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.045 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.046 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.046 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-07-06T20:37:44.046 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.046 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.046 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.046 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.046 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-07-06T20:37:44.046 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.047 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.047 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.047 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.047 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-07-06T20:37:44.047 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.047 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.047 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.047 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.048 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-07-06T20:37:44.048 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.048 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.048 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.048 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.048 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-07-06T20:37:44.048 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.048 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.049 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.049 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.049 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-07-06T20:37:44.049 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.049 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.049 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.049 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.049 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-07-06T20:37:44.050 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.050 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.050 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.050 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.050 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-07-06T20:37:44.050 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.050 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.050 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.051 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.051 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-07-06T20:37:44.051 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.051 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.051 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.051 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.051 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-07-06T20:37:44.052 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.052 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.052 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.052 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.052 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-07-06T20:37:44.052 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.052 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.052 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.053 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.053 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-07-06T20:37:44.053 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.053 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.053 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.053 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.053 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-07-06T20:37:44.053 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.054 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.054 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.054 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.054 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-07-06T20:37:44.054 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.054 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.054 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.054 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.055 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-07-06T20:37:44.055 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.055 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.055 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.055 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.055 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-07-06T20:37:44.055 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-07-06T20:37:44.055 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-07-06T20:37:44.056 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.056 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.056 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.056 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:44.056 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.056 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-07-06T20:37:44.056 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 1, 2023-07-06T20:37:44.057 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 2, 2023-07-06T20:37:44.057 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 1, 2023-07-06T20:37:44.057 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 1, 2023-07-06T20:37:44.057 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-07-06T20:37:44.057 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2023-07-06T20:37:44.057 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2023-07-06T20:37:44.057 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 3324 2023-07-06T20:37:44.057 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.058 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-07-06T20:37:44.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-07-06T20:37:44.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-07-06T20:37:44.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T20:37:44.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-07-06T20:37:44.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T20:37:44.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-07-06T20:37:44.058 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T20:37:44.059 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T20:37:44.059 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T20:37:44.059 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T20:37:44.059 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T20:37:44.059 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-07-06T20:37:44.059 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T20:37:44.060 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T20:37:44.060 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-07-06T20:37:44.060 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-07-06T20:37:44.060 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-07-06T20:37:44.060 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-07-06T20:37:44.060 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T20:37:44.060 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T20:37:44.061 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T20:37:44.061 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T20:37:44.061 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T20:37:44.061 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-07-06T20:37:44.061 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-07-06T20:37:44.061 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-07-06T20:37:44.061 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T20:37:44.061 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T20:37:44.062 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-07-06T20:37:44.062 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-07-06T20:37:44.062 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-07-06T20:37:44.062 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-07-06T20:37:44.062 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T20:37:44.062 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-07-06T20:37:44.062 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T20:37:44.062 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-07-06T20:37:44.063 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 209094, 2023-07-06T20:37:44.063 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 564, 2023-07-06T20:37:44.063 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-07-06T20:37:44.063 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-07-06T20:37:44.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-07-06T20:37:44.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-07-06T20:37:44.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-07-06T20:37:44.063 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-07-06T20:37:44.064 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-07-06T20:37:44.064 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-07-06T20:37:44.064 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 7728, 2023-07-06T20:37:44.064 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 163, 2023-07-06T20:37:44.064 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T20:37:44.064 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-07-06T20:37:44.064 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-07-06T20:37:44.064 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-07-06T20:37:44.065 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 1609574, 2023-07-06T20:37:44.065 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 14493, 2023-07-06T20:37:44.065 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-07-06T20:37:44.065 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-07-06T20:37:44.065 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-07-06T20:37:44.065 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-07-06T20:37:44.065 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.065 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-07-06T20:37:44.066 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-07-06T20:37:44.066 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-07-06T20:37:44.066 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 3765, 2023-07-06T20:37:44.066 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21, 2023-07-06T20:37:44.066 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2023-07-06T20:37:44.066 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 3765, 2023-07-06T20:37:44.066 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-07-06T20:37:44.066 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3765, 2023-07-06T20:37:44.067 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 51.002257823, 2023-07-06T20:37:44.067 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.013546416 2023-07-06T20:37:44.067 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.067 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-07-06T20:37:44.067 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-07-06T20:37:44.067 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3765, 2023-07-06T20:37:44.067 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3766 2023-07-06T20:37:44.068 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.068 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 3765, 2023-07-06T20:37:44.068 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 3765, 2023-07-06T20:37:44.068 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2023-07-06T20:37:44.068 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-07-06T20:37:44.068 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-07-06T20:37:44.068 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 2246, 2023-07-06T20:37:44.068 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2023-07-06T20:37:44.069 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 1517, 2023-07-06T20:37:44.069 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2023-07-06T20:37:44.069 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2023-07-06T20:37:44.069 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-07-06T20:37:44.069 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-07-06T20:37:44.069 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-07-06T20:37:44.069 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2023-07-06T20:37:44.069 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2023-07-06T20:37:44.070 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-07-06T20:37:44.070 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-07-06T20:37:44.070 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-07-06T20:37:44.070 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1, 2023-07-06T20:37:44.070 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2023-07-06T20:37:44.070 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-07-06T20:37:44.070 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-07-06T20:37:44.070 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-07-06T20:37:44.071 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-07-06T20:37:44.071 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-07-06T20:37:44.071 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-07-06T20:37:44.071 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-07-06T20:37:44.071 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-07-06T20:37:44.071 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-07-06T20:37:44.071 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-07-06T20:37:44.071 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-07-06T20:37:44.072 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-07-06T20:37:44.072 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-07-06T20:37:44.072 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-07-06T20:37:44.072 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-07-06T20:37:44.072 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-07-06T20:37:44.072 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-07-06T20:37:44.072 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-07-06T20:37:44.072 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-07-06T20:37:44.073 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-07-06T20:37:44.073 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-07-06T20:37:44.073 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-07-06T20:37:44.073 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-07-06T20:37:44.073 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-07-06T20:37:44.073 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-07-06T20:37:44.073 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-07-06T20:37:44.073 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 87, 2023-07-06T20:37:44.074 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-07-06T20:37:44.074 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 4, 2023-07-06T20:37:44.074 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 5, 2023-07-06T20:37:44.074 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 6, 2023-07-06T20:37:44.074 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 1, 2023-07-06T20:37:44.074 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-07-06T20:37:44.074 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2023-07-06T20:37:44.075 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2023-07-06T20:37:44.075 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2023-07-06T20:37:44.075 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.075 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-07-06T20:37:44.075 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2023-07-06T20:37:44.075 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-07-06T20:37:44.075 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-07-06T20:37:44.075 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-07-06T20:37:44.076 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.076 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-07-06T20:37:44.076 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2023-07-06T20:37:44.076 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.076 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-07-06T20:37:44.076 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-07-06T20:37:44.076 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2023-07-06T20:37:44.076 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-07-06T20:37:44.077 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2023-07-06T20:37:44.077 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2023-07-06T20:37:44.077 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-07-06T20:37:44.077 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.077 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T20:37:44.077 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:44.077 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-06T20:37:44.077 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:44.078 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 4116, 2023-07-06T20:37:44.078 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 4115850, 2023-07-06T20:37:44.078 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:44.078 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 4116, 2023-07-06T20:37:44.078 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-06T20:37:44.078 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-06T20:37:44.078 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 4116, 2023-07-06T20:37:44.078 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 4115850, 2023-07-06T20:37:44.079 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:44.079 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.079 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.079 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.079 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:44.079 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.079 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-07-06T20:37:44.079 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:44.080 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-06T20:37:44.080 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:44.080 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-06T20:37:44.080 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-06T20:37:44.080 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:44.080 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:44.080 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 3765, 2023-07-06T20:37:44.081 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 2906229, 2023-07-06T20:37:44.081 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 3765, 2023-07-06T20:37:44.081 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 2906229, 2023-07-06T20:37:44.081 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:44.081 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.081 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.081 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.081 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:44.082 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.082 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-07-06T20:37:44.082 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:44.082 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-07-06T20:37:44.082 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:44.082 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-06T20:37:44.082 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-06T20:37:44.082 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:44.083 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:44.083 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 3765, 2023-07-06T20:37:44.083 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 3765, 2023-07-06T20:37:44.083 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 3765, 2023-07-06T20:37:44.083 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 3765, 2023-07-06T20:37:44.083 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:44.083 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.083 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.084 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.084 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:44.084 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.084 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-07-06T20:37:44.084 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:44.084 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-07-06T20:37:44.084 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:44.085 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-06T20:37:44.085 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-06T20:37:44.085 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:44.085 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:44.085 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-06T20:37:44.085 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-06T20:37:44.085 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-07-06T20:37:44.085 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-07-06T20:37:44.086 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:44.086 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.086 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.086 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.086 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:44.086 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:44.086 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55f5c6976460": { 2023-07-06T20:37:44.086 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:44.087 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-07-06T20:37:44.087 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:44.087 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-06T20:37:44.087 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-06T20:37:44.087 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:44.087 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:44.087 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-06T20:37:44.087 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-06T20:37:44.088 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-07-06T20:37:44.088 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-07-06T20:37:44.088 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:44.088 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:44.088 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:44.088 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:44.088 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:44.088 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:44.089 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-07-06T20:37:44.394 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-07-06T20:37:44.395 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-07-06T20:37:44.395 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-07-06T20:37:44.395 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-07-06T20:37:44.396 DEBUG:tasks.check_counter:Getting stats from i 2023-07-06T20:37:44.396 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-06T20:37:44.945 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:37:45.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:44 smithi156 ceph-mon[129280]: pgmap v880: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-07-06T20:37:45.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:44 smithi098 ceph-mon[128968]: pgmap v880: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-07-06T20:37:45.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:44 smithi069 ceph-mon[126399]: pgmap v880: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-07-06T20:37:46.120 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-07-06T20:37:46.120 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T20:37:46.120 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1738, 2023-07-06T20:37:46.121 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1740, 2023-07-06T20:37:46.121 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 4730651, 2023-07-06T20:37:46.121 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 540528, 2023-07-06T20:37:46.121 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-07-06T20:37:46.121 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2023-07-06T20:37:46.121 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.173705588, 2023-07-06T20:37:46.121 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.042786459, 2023-07-06T20:37:46.121 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.046601229, 2023-07-06T20:37:46.122 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.068851625, 2023-07-06T20:37:46.122 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:46.122 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1738, 2023-07-06T20:37:46.122 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.352163270, 2023-07-06T20:37:46.122 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000202625 2023-07-06T20:37:46.122 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.122 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:46.123 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.123 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.123 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.123 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.123 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-06T20:37:46.123 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-07-06T20:37:46.123 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.123 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T20:37:46.124 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1973, 2023-07-06T20:37:46.124 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2220, 2023-07-06T20:37:46.124 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 8298825, 2023-07-06T20:37:46.124 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1118194, 2023-07-06T20:37:46.124 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 5, 2023-07-06T20:37:46.124 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2023-07-06T20:37:46.124 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.177402667, 2023-07-06T20:37:46.124 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.050518304, 2023-07-06T20:37:46.125 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.049471566, 2023-07-06T20:37:46.125 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.063934467, 2023-07-06T20:37:46.125 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:46.125 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2219, 2023-07-06T20:37:46.125 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.616326803, 2023-07-06T20:37:46.125 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000277749 2023-07-06T20:37:46.125 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.125 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:46.126 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.126 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.126 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.126 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.126 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-07-06T20:37:46.126 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 506080 2023-07-06T20:37:46.126 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.127 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T20:37:46.127 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 856, 2023-07-06T20:37:46.127 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 843, 2023-07-06T20:37:46.127 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 4878701, 2023-07-06T20:37:46.127 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 346688, 2023-07-06T20:37:46.127 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 3, 2023-07-06T20:37:46.127 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2023-07-06T20:37:46.127 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.087333099, 2023-07-06T20:37:46.128 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.031038954, 2023-07-06T20:37:46.128 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.029823829, 2023-07-06T20:37:46.128 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.021809990, 2023-07-06T20:37:46.128 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:46.128 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 842, 2023-07-06T20:37:46.128 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.132332511, 2023-07-06T20:37:46.128 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000157164 2023-07-06T20:37:46.128 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.129 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:46.129 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.129 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.129 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.129 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.129 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 556048, 2023-07-06T20:37:46.129 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 189632 2023-07-06T20:37:46.129 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.130 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-07-06T20:37:46.130 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-07-06T20:37:46.130 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-07-06T20:37:46.130 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.130 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-07-06T20:37:46.130 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-07-06T20:37:46.130 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-07-06T20:37:46.131 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7177, 2023-07-06T20:37:46.131 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.180441627, 2023-07-06T20:37:46.131 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000025141 2023-07-06T20:37:46.131 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:46.131 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.131 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-07-06T20:37:46.131 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-07-06T20:37:46.131 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-07-06T20:37:46.132 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.132 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.132 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.132 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:46.132 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.132 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-07-06T20:37:46.132 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2023-07-06T20:37:46.132 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2023-07-06T20:37:46.133 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-07-06T20:37:46.133 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.133 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.133 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.133 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.133 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-07-06T20:37:46.133 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2023-07-06T20:37:46.133 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2023-07-06T20:37:46.134 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-07-06T20:37:46.134 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2023-07-06T20:37:46.134 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2023-07-06T20:37:46.134 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-07-06T20:37:46.134 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 854, 2023-07-06T20:37:46.134 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 0, 2023-07-06T20:37:46.134 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 0, 2023-07-06T20:37:46.134 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 854, 2023-07-06T20:37:46.135 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 854, 2023-07-06T20:37:46.135 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 0, 2023-07-06T20:37:46.135 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-07-06T20:37:46.135 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-07-06T20:37:46.135 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 16, 2023-07-06T20:37:46.135 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2023-07-06T20:37:46.135 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2023-07-06T20:37:46.136 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2023-07-06T20:37:46.136 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2023-07-06T20:37:46.136 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2023-07-06T20:37:46.136 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-07-06T20:37:46.136 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2023-07-06T20:37:46.136 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-07-06T20:37:46.136 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-07-06T20:37:46.136 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2023-07-06T20:37:46.137 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2023-07-06T20:37:46.137 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2023-07-06T20:37:46.137 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2023-07-06T20:37:46.137 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-07-06T20:37:46.137 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2023-07-06T20:37:46.137 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-07-06T20:37:46.137 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 1046, 2023-07-06T20:37:46.137 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 9135549903, 2023-07-06T20:37:46.138 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2023-07-06T20:37:46.138 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2023-07-06T20:37:46.138 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-07-06T20:37:46.138 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-07-06T20:37:46.138 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-07-06T20:37:46.138 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2023-07-06T20:37:46.139 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-07-06T20:37:46.139 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2023-07-06T20:37:46.139 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2023-07-06T20:37:46.139 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2023-07-06T20:37:46.139 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2023-07-06T20:37:46.139 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2023-07-06T20:37:46.139 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-07-06T20:37:46.140 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-06T20:37:46.140 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-06T20:37:46.140 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2023-07-06T20:37:46.140 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2023-07-06T20:37:46.140 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2023-07-06T20:37:46.140 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2023-07-06T20:37:46.140 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.140 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-07-06T20:37:46.141 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-07-06T20:37:46.141 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2023-07-06T20:37:46.141 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2023-07-06T20:37:46.141 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2023-07-06T20:37:46.141 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2023-07-06T20:37:46.141 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2023-07-06T20:37:46.141 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-07-06T20:37:46.141 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-07-06T20:37:46.142 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2023-07-06T20:37:46.142 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2023-07-06T20:37:46.142 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-07-06T20:37:46.142 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-07-06T20:37:46.142 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-07-06T20:37:46.142 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-07-06T20:37:46.142 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-07-06T20:37:46.142 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-07-06T20:37:46.143 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-07-06T20:37:46.143 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-07-06T20:37:46.143 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2023-07-06T20:37:46.143 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-07-06T20:37:46.143 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2023-07-06T20:37:46.143 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-07-06T20:37:46.143 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-07-06T20:37:46.143 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.144 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-07-06T20:37:46.144 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2023-07-06T20:37:46.144 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-07-06T20:37:46.144 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-07-06T20:37:46.144 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 0, 2023-07-06T20:37:46.144 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-07-06T20:37:46.145 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-07-06T20:37:46.145 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2023-07-06T20:37:46.145 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-07-06T20:37:46.145 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-07-06T20:37:46.145 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 8, 2023-07-06T20:37:46.145 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-07-06T20:37:46.145 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-07-06T20:37:46.146 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2023-07-06T20:37:46.146 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 0, 2023-07-06T20:37:46.146 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 20377249, 2023-07-06T20:37:46.146 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-07-06T20:37:46.146 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.146 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.146 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.147 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.147 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 5968 2023-07-06T20:37:46.147 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.147 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-07-06T20:37:46.147 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 527, 2023-07-06T20:37:46.147 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 1313, 2023-07-06T20:37:46.147 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 786, 2023-07-06T20:37:46.147 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 63, 2023-07-06T20:37:46.148 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 71, 2023-07-06T20:37:46.148 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 8, 2023-07-06T20:37:46.148 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 854, 2023-07-06T20:37:46.148 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 1380, 2023-07-06T20:37:46.148 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 526, 2023-07-06T20:37:46.148 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-07-06T20:37:46.148 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2023-07-06T20:37:46.148 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2023-07-06T20:37:46.149 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 54804, 2023-07-06T20:37:46.149 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207156 2023-07-06T20:37:46.149 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.149 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-07-06T20:37:46.149 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2023-07-06T20:37:46.149 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2023-07-06T20:37:46.149 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2023-07-06T20:37:46.149 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2023-07-06T20:37:46.150 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2023-07-06T20:37:46.150 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-07-06T20:37:46.150 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.150 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.150 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.150 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.150 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-07-06T20:37:46.150 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.151 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.151 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.151 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.151 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-07-06T20:37:46.151 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.151 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.151 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.151 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.152 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-07-06T20:37:46.152 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.152 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.152 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.152 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.152 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-07-06T20:37:46.152 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.153 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.153 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.153 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.153 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-07-06T20:37:46.153 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.153 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.153 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.153 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.154 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-07-06T20:37:46.154 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.154 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.154 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.154 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.154 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-07-06T20:37:46.154 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.154 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.155 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.155 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.155 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-07-06T20:37:46.155 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.155 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.155 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.155 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.155 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-07-06T20:37:46.156 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.156 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.156 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.156 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.156 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-07-06T20:37:46.156 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.156 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.157 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.157 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.157 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-07-06T20:37:46.157 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.157 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.157 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.157 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.157 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-07-06T20:37:46.158 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.158 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.158 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.158 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.158 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-07-06T20:37:46.158 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.158 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.159 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.159 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.159 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-07-06T20:37:46.159 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.159 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.159 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.159 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.159 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-07-06T20:37:46.160 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.160 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.160 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.160 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.160 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-07-06T20:37:46.160 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.160 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.160 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.161 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.161 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-07-06T20:37:46.161 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.161 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.161 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.161 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.161 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-07-06T20:37:46.162 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.162 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.162 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.162 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.162 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-07-06T20:37:46.162 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.162 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.162 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.163 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.163 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-07-06T20:37:46.163 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.163 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.163 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.163 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.163 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-07-06T20:37:46.163 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.164 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.164 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.164 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.164 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-07-06T20:37:46.164 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.164 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.164 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.164 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.165 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-07-06T20:37:46.165 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.165 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.165 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.165 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.165 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-07-06T20:37:46.165 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.165 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.166 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.166 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.166 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-07-06T20:37:46.166 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.166 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.166 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.166 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.166 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-07-06T20:37:46.167 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.167 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.167 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.167 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.167 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-07-06T20:37:46.167 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.167 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.167 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.168 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.168 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-07-06T20:37:46.168 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.168 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.168 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.168 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.168 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-07-06T20:37:46.169 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-07-06T20:37:46.169 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-07-06T20:37:46.169 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.169 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.169 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.169 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:46.169 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.169 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-07-06T20:37:46.170 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 1, 2023-07-06T20:37:46.170 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 2, 2023-07-06T20:37:46.170 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 1, 2023-07-06T20:37:46.170 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 1, 2023-07-06T20:37:46.170 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-07-06T20:37:46.170 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2023-07-06T20:37:46.170 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2023-07-06T20:37:46.170 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 3326 2023-07-06T20:37:46.171 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.171 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-07-06T20:37:46.171 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-07-06T20:37:46.171 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-07-06T20:37:46.171 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T20:37:46.171 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-07-06T20:37:46.171 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T20:37:46.171 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-07-06T20:37:46.172 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T20:37:46.172 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T20:37:46.172 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T20:37:46.172 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T20:37:46.172 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T20:37:46.172 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-07-06T20:37:46.172 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T20:37:46.172 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T20:37:46.173 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-07-06T20:37:46.173 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-07-06T20:37:46.173 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-07-06T20:37:46.173 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-07-06T20:37:46.173 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T20:37:46.173 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T20:37:46.173 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T20:37:46.174 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T20:37:46.174 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T20:37:46.174 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-07-06T20:37:46.174 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-07-06T20:37:46.174 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-07-06T20:37:46.174 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T20:37:46.174 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T20:37:46.174 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-07-06T20:37:46.175 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-07-06T20:37:46.175 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-07-06T20:37:46.175 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-07-06T20:37:46.175 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T20:37:46.175 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-07-06T20:37:46.175 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T20:37:46.175 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-07-06T20:37:46.175 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 208105, 2023-07-06T20:37:46.176 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 556, 2023-07-06T20:37:46.176 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-07-06T20:37:46.176 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-07-06T20:37:46.176 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-07-06T20:37:46.176 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-07-06T20:37:46.176 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-07-06T20:37:46.176 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-07-06T20:37:46.176 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-07-06T20:37:46.177 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-07-06T20:37:46.177 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 7728, 2023-07-06T20:37:46.177 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 163, 2023-07-06T20:37:46.177 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T20:37:46.177 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-07-06T20:37:46.177 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-07-06T20:37:46.177 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-07-06T20:37:46.177 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 1857844, 2023-07-06T20:37:46.178 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 15744, 2023-07-06T20:37:46.178 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-07-06T20:37:46.178 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-07-06T20:37:46.178 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-07-06T20:37:46.178 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-07-06T20:37:46.178 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.178 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-07-06T20:37:46.178 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-07-06T20:37:46.179 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-07-06T20:37:46.179 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 4211, 2023-07-06T20:37:46.179 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 26, 2023-07-06T20:37:46.179 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 4, 2023-07-06T20:37:46.179 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 4211, 2023-07-06T20:37:46.179 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-07-06T20:37:46.179 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 4211, 2023-07-06T20:37:46.179 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 33.125520575, 2023-07-06T20:37:46.180 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.007866426 2023-07-06T20:37:46.180 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.180 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-07-06T20:37:46.180 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-07-06T20:37:46.180 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 4211, 2023-07-06T20:37:46.180 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 4212 2023-07-06T20:37:46.180 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.180 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 4211, 2023-07-06T20:37:46.181 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 4211, 2023-07-06T20:37:46.181 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2023-07-06T20:37:46.181 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-07-06T20:37:46.181 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-07-06T20:37:46.181 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 2342, 2023-07-06T20:37:46.181 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2023-07-06T20:37:46.181 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 1866, 2023-07-06T20:37:46.181 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2023-07-06T20:37:46.182 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2023-07-06T20:37:46.182 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-07-06T20:37:46.182 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-07-06T20:37:46.182 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-07-06T20:37:46.182 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2023-07-06T20:37:46.182 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2023-07-06T20:37:46.182 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-07-06T20:37:46.183 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-07-06T20:37:46.183 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-07-06T20:37:46.183 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 2, 2023-07-06T20:37:46.183 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2023-07-06T20:37:46.183 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-07-06T20:37:46.183 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-07-06T20:37:46.183 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-07-06T20:37:46.183 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-07-06T20:37:46.184 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-07-06T20:37:46.184 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-07-06T20:37:46.184 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-07-06T20:37:46.184 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-07-06T20:37:46.184 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-07-06T20:37:46.184 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-07-06T20:37:46.184 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-07-06T20:37:46.184 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-07-06T20:37:46.185 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-07-06T20:37:46.185 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-07-06T20:37:46.185 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-07-06T20:37:46.185 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-07-06T20:37:46.185 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-07-06T20:37:46.185 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-07-06T20:37:46.185 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-07-06T20:37:46.185 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-07-06T20:37:46.186 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-07-06T20:37:46.186 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-07-06T20:37:46.186 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-07-06T20:37:46.186 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-07-06T20:37:46.186 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-07-06T20:37:46.186 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-07-06T20:37:46.186 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 87, 2023-07-06T20:37:46.186 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-07-06T20:37:46.187 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 4, 2023-07-06T20:37:46.187 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 5, 2023-07-06T20:37:46.187 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 5, 2023-07-06T20:37:46.187 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-07-06T20:37:46.187 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-07-06T20:37:46.187 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2023-07-06T20:37:46.187 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2023-07-06T20:37:46.187 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2023-07-06T20:37:46.188 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.188 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-07-06T20:37:46.188 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2023-07-06T20:37:46.188 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-07-06T20:37:46.188 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-07-06T20:37:46.188 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-07-06T20:37:46.188 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.188 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-07-06T20:37:46.189 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2023-07-06T20:37:46.189 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.189 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-07-06T20:37:46.189 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-07-06T20:37:46.189 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2023-07-06T20:37:46.189 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-07-06T20:37:46.189 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2023-07-06T20:37:46.189 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2023-07-06T20:37:46.190 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-07-06T20:37:46.190 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.190 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T20:37:46.190 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:46.190 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-06T20:37:46.190 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:46.190 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 4563, 2023-07-06T20:37:46.191 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 17476498, 2023-07-06T20:37:46.191 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:46.191 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 4563, 2023-07-06T20:37:46.191 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-06T20:37:46.191 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-06T20:37:46.191 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 4563, 2023-07-06T20:37:46.191 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 17476498, 2023-07-06T20:37:46.191 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:46.192 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.192 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.192 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.192 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:46.192 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.192 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-07-06T20:37:46.192 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:46.192 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-06T20:37:46.193 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:46.193 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-06T20:37:46.193 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-06T20:37:46.193 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:46.193 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:46.193 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 4211, 2023-07-06T20:37:46.193 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 16186163, 2023-07-06T20:37:46.193 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 4211, 2023-07-06T20:37:46.194 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 16186163, 2023-07-06T20:37:46.194 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:46.194 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.194 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.194 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.194 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:46.194 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.194 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-07-06T20:37:46.195 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:46.195 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-07-06T20:37:46.195 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:46.195 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-06T20:37:46.195 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-06T20:37:46.195 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:46.195 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:46.196 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 4211, 2023-07-06T20:37:46.196 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 4211, 2023-07-06T20:37:46.196 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 4211, 2023-07-06T20:37:46.196 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 4211, 2023-07-06T20:37:46.196 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:46.196 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.196 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.196 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.197 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:46.197 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.197 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-07-06T20:37:46.197 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:46.197 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-07-06T20:37:46.197 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:46.197 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-06T20:37:46.197 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-06T20:37:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-06T20:37:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-06T20:37:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-07-06T20:37:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-07-06T20:37:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:46.198 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55b5204d8460": { 2023-07-06T20:37:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-07-06T20:37:46.199 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-06T20:37:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-06T20:37:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-06T20:37:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-06T20:37:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-06T20:37:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-07-06T20:37:46.200 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-07-06T20:37:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:46.201 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:46.201 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-07-06T20:37:46.485 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2023-07-06T20:37:46.485 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2023-07-06T20:37:46.486 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2023-07-06T20:37:46.486 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2023-07-06T20:37:46.486 DEBUG:tasks.check_counter:Getting stats from l 2023-07-06T20:37:46.486 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:79365bb3b803780276d69efb2cc4afe5b160de9e shell --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-06T20:37:47.035 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/mon.c/config 2023-07-06T20:37:47.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:46 smithi156 ceph-mon[129280]: pgmap v881: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2023-07-06T20:37:47.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:46 smithi098 ceph-mon[128968]: pgmap v881: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2023-07-06T20:37:47.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:46 smithi069 ceph-mon[126399]: pgmap v881: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2023-07-06T20:37:48.387 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-07-06T20:37:48.387 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T20:37:48.388 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 0, 2023-07-06T20:37:48.388 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2, 2023-07-06T20:37:48.388 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 0, 2023-07-06T20:37:48.388 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 0, 2023-07-06T20:37:48.388 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:48.389 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 0, 2023-07-06T20:37:48.389 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.001186928, 2023-07-06T20:37:48.389 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.000000000, 2023-07-06T20:37:48.389 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.000285427, 2023-07-06T20:37:48.389 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-06T20:37:48.389 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:48.389 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:48.389 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:48.390 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:48.390 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:48.390 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:48.390 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:48.390 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:48.390 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:48.390 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:48.390 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-06T20:37:48.391 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-07-06T20:37:48.391 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:48.391 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T20:37:48.391 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2, 2023-07-06T20:37:48.392 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 255, 2023-07-06T20:37:48.392 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 320, 2023-07-06T20:37:48.392 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 196624, 2023-07-06T20:37:48.392 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2023-07-06T20:37:48.393 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2023-07-06T20:37:48.393 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.013550564, 2023-07-06T20:37:48.393 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.009685138, 2023-07-06T20:37:48.393 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.001319376, 2023-07-06T20:37:48.393 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-06T20:37:48.393 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:48.393 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 253, 2023-07-06T20:37:48.393 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.128030631, 2023-07-06T20:37:48.394 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000506049 2023-07-06T20:37:48.394 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:48.394 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:48.394 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:48.394 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:48.394 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:48.394 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:48.395 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-06T20:37:48.395 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 196624 2023-07-06T20:37:48.395 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:48.395 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T20:37:48.395 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 340, 2023-07-06T20:37:48.395 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 335, 2023-07-06T20:37:48.395 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 508048, 2023-07-06T20:37:48.396 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 191344, 2023-07-06T20:37:48.396 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 2, 2023-07-06T20:37:48.396 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 1, 2023-07-06T20:37:48.396 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.048597950, 2023-07-06T20:37:48.396 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.017774078, 2023-07-06T20:37:48.396 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.018816753, 2023-07-06T20:37:48.396 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.010002918, 2023-07-06T20:37:48.396 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T20:37:48.397 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 335, 2023-07-06T20:37:48.397 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.105263082, 2023-07-06T20:37:48.397 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000314218 2023-07-06T20:37:48.397 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:48.397 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-06T20:37:48.397 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:48.397 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:48.397 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:48.398 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:48.398 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 508048, 2023-07-06T20:37:48.398 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 191344 2023-07-06T20:37:48.398 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:48.398 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-07-06T20:37:48.398 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 0, 2023-07-06T20:37:48.398 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-07-06T20:37:48.399 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:48.399 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-07-06T20:37:48.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-07-06T20:37:48.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-07-06T20:37:48.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T20:37:48.399 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-07-06T20:37:48.400 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T20:37:48.400 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-07-06T20:37:48.400 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T20:37:48.400 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T20:37:48.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T20:37:48.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T20:37:48.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T20:37:48.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-07-06T20:37:48.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T20:37:48.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T20:37:48.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-07-06T20:37:48.401 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-07-06T20:37:48.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-07-06T20:37:48.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-07-06T20:37:48.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T20:37:48.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T20:37:48.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T20:37:48.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T20:37:48.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T20:37:48.402 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-07-06T20:37:48.403 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-07-06T20:37:48.403 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-07-06T20:37:48.403 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T20:37:48.403 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T20:37:48.403 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-07-06T20:37:48.403 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-07-06T20:37:48.403 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-07-06T20:37:48.404 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-07-06T20:37:48.404 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T20:37:48.404 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-07-06T20:37:48.404 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T20:37:48.404 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-07-06T20:37:48.404 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 148704, 2023-07-06T20:37:48.404 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 29, 2023-07-06T20:37:48.405 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-07-06T20:37:48.405 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-07-06T20:37:48.405 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-07-06T20:37:48.405 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-07-06T20:37:48.405 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-07-06T20:37:48.405 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-07-06T20:37:48.405 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-07-06T20:37:48.405 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-07-06T20:37:48.406 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 0, 2023-07-06T20:37:48.406 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 0, 2023-07-06T20:37:48.406 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T20:37:48.406 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-07-06T20:37:48.406 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-07-06T20:37:48.406 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-07-06T20:37:48.406 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 928, 2023-07-06T20:37:48.406 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 2, 2023-07-06T20:37:48.407 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-07-06T20:37:48.407 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-07-06T20:37:48.407 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-07-06T20:37:48.407 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-07-06T20:37:48.407 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-06T20:37:48.407 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T20:37:48.407 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-06T20:37:48.408 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-06T20:37:48.408 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-06T20:37:48.408 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 338, 2023-07-06T20:37:48.408 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 457536, 2023-07-06T20:37:48.408 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-06T20:37:48.408 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 338, 2023-07-06T20:37:48.408 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-06T20:37:48.409 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-06T20:37:48.409 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 338, 2023-07-06T20:37:48.409 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 457536, 2023-07-06T20:37:48.409 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-06T20:37:48.409 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-06T20:37:48.409 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-06T20:37:48.410 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-06T20:37:48.410 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:48.410 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-06T20:37:48.410 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-07-06T20:37:48.877 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-07-06T20:37:48.878 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-07-06T20:37:48.878 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-07-06T20:37:48.878 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-07-06T20:37:48.878 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-06T20:37:48.888 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-06T20:37:48.888 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:37:48.889 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:37:48.938 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:37:48.938 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:37:48.985 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:37:49.087 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:48 smithi069 ceph-mon[126399]: pgmap v882: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-07-06T20:37:49.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:48 smithi156 ceph-mon[129280]: pgmap v882: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-07-06T20:37:49.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:48 smithi098 ceph-mon[128968]: pgmap v882: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-07-06T20:37:49.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.225+0000 7f72155ff700 1 -- 172.21.15.69:0/2208797454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7210100680 msgr2=0x7f7210102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:49.229 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.225+0000 7f72155ff700 1 --2- 172.21.15.69:0/2208797454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7210100680 0x7f7210102a60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7200009990 tx=0x7f7200009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.229 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.226+0000 7f72155ff700 1 -- 172.21.15.69:0/2208797454 shutdown_connections 2023-07-06T20:37:49.229 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.226+0000 7f72155ff700 1 --2- 172.21.15.69:0/2208797454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72101058c0 0x7f7210107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.229 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.226+0000 7f72155ff700 1 --2- 172.21.15.69:0/2208797454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7210100680 0x7f7210102a60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.229 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.226+0000 7f72155ff700 1 --2- 172.21.15.69:0/2208797454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7210102fa0 0x7f7210105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.226+0000 7f72155ff700 1 -- 172.21.15.69:0/2208797454 >> 172.21.15.69:0/2208797454 conn(0x7f72100fa100 msgr2=0x7f72100fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:49.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.226+0000 7f72155ff700 1 -- 172.21.15.69:0/2208797454 shutdown_connections 2023-07-06T20:37:49.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.226+0000 7f72155ff700 1 -- 172.21.15.69:0/2208797454 wait complete. 2023-07-06T20:37:49.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f72155ff700 1 Processor -- start 2023-07-06T20:37:49.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f72155ff700 1 -- start start 2023-07-06T20:37:49.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f72155ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7210100680 0x7f7210197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:49.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f72155ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7210102fa0 0x7f7210198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:49.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f72155ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72101058c0 0x7f721019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:49.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f72155ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721019f680 con 0x7f72101058c0 2023-07-06T20:37:49.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f72155ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721019f7c0 con 0x7f7210100680 2023-07-06T20:37:49.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f72155ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721019f950 con 0x7f7210102fa0 2023-07-06T20:37:49.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f720effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7210100680 0x7f7210197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:49.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f720effd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7210100680 0x7f7210197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:51882/0 (socket says 172.21.15.69:51882) 2023-07-06T20:37:49.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f720effd700 1 -- 172.21.15.69:0/1200785362 learned_addr learned my addr 172.21.15.69:0/1200785362 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:49.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f720f7fe700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72101058c0 0x7f721019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:49.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.227+0000 7f720e7fc700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7210102fa0 0x7f7210198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:49.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.228+0000 7f720effd700 1 -- 172.21.15.69:0/1200785362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7210102fa0 msgr2=0x7f7210198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:49.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.228+0000 7f720effd700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7210102fa0 0x7f7210198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.228+0000 7f720effd700 1 -- 172.21.15.69:0/1200785362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72101058c0 msgr2=0x7f721019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:49.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.228+0000 7f720effd700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72101058c0 0x7f721019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.228+0000 7f720effd700 1 -- 172.21.15.69:0/1200785362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72000095f0 con 0x7f7210100680 2023-07-06T20:37:49.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.228+0000 7f720effd700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7210100680 0x7f7210197b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7200004740 tx=0x7f7200009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:49.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.228+0000 7f71f7fff700 1 -- 172.21.15.69:0/1200785362 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720001d070 con 0x7f7210100680 2023-07-06T20:37:49.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.228+0000 7f72155ff700 1 -- 172.21.15.69:0/1200785362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721019fbd0 con 0x7f7210100680 2023-07-06T20:37:49.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.228+0000 7f71f7fff700 1 -- 172.21.15.69:0/1200785362 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7200004830 con 0x7f7210100680 2023-07-06T20:37:49.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.228+0000 7f71f7fff700 1 -- 172.21.15.69:0/1200785362 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72000053c0 con 0x7f7210100680 2023-07-06T20:37:49.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.228+0000 7f72155ff700 1 -- 172.21.15.69:0/1200785362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721019fff0 con 0x7f7210100680 2023-07-06T20:37:49.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.229+0000 7f72155ff700 1 -- 172.21.15.69:0/1200785362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7210107480 con 0x7f7210100680 2023-07-06T20:37:49.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.231+0000 7f71f7fff700 1 -- 172.21.15.69:0/1200785362 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7200005560 con 0x7f7210100680 2023-07-06T20:37:49.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.233+0000 7f71f7fff700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71f80a11a0 0x7f71f80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:49.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.233+0000 7f71f7fff700 1 -- 172.21.15.69:0/1200785362 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f72000cdf50 con 0x7f7210100680 2023-07-06T20:37:49.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.233+0000 7f720e7fc700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71f80a11a0 0x7f71f80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:49.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.233+0000 7f720e7fc700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71f80a11a0 0x7f71f80a3650 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7204009c60 tx=0x7f7204009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:49.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.235+0000 7f71f7fff700 1 -- 172.21.15.69:0/1200785362 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7200097700 con 0x7f7210100680 2023-07-06T20:37:49.438 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.436+0000 7f72155ff700 1 -- 172.21.15.69:0/1200785362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f721002cab0 con 0x7f7210100680 2023-07-06T20:37:49.440 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.438+0000 7f71f7fff700 1 -- 172.21.15.69:0/1200785362 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7f720009ad20 con 0x7f7210100680 2023-07-06T20:37:49.441 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:49.441 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":87,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:18:59.638711+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:37:49.443 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.441+0000 7f72155ff700 1 -- 172.21.15.69:0/1200785362 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71f80a11a0 msgr2=0x7f71f80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:49.443 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.441+0000 7f72155ff700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71f80a11a0 0x7f71f80a3650 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7204009c60 tx=0x7f7204009260 comp rx=0 tx=0).stop 2023-07-06T20:37:49.443 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.441+0000 7f72155ff700 1 -- 172.21.15.69:0/1200785362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7210100680 msgr2=0x7f7210197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:49.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.441+0000 7f72155ff700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7210100680 0x7f7210197b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7200004740 tx=0x7f7200009580 comp rx=0 tx=0).stop 2023-07-06T20:37:49.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.441+0000 7f72155ff700 1 -- 172.21.15.69:0/1200785362 shutdown_connections 2023-07-06T20:37:49.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.441+0000 7f72155ff700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f71f80a11a0 0x7f71f80a3650 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.441+0000 7f72155ff700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72101058c0 0x7f721019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.441+0000 7f72155ff700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7210100680 0x7f7210197b00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.441+0000 7f72155ff700 1 --2- 172.21.15.69:0/1200785362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7210102fa0 0x7f7210198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.444 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.441+0000 7f72155ff700 1 -- 172.21.15.69:0/1200785362 >> 172.21.15.69:0/1200785362 conn(0x7f72100fa100 msgr2=0x7f72100fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:49.445 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.441+0000 7f72155ff700 1 -- 172.21.15.69:0/1200785362 shutdown_connections 2023-07-06T20:37:49.445 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.441+0000 7f72155ff700 1 -- 172.21.15.69:0/1200785362 wait complete. 2023-07-06T20:37:49.462 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T20:13:14.022799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T20:17:26.523183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T20:17:27.543386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T20:37:49.463 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T20:37:49.664 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.662+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/276470534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a740ffc20 msgr2=0x7f1a74100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:49.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.662+0000 7f1a7c5ff700 1 --2- 172.21.15.69:0/276470534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a740ffc20 0x7f1a74100030 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009990 tx=0x7f1a70009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.662+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/276470534 shutdown_connections 2023-07-06T20:37:49.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.662+0000 7f1a7c5ff700 1 --2- 172.21.15.69:0/276470534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a741017a0 0x7f1a74108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.662+0000 7f1a7c5ff700 1 --2- 172.21.15.69:0/276470534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a740ffc20 0x7f1a74100030 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.662+0000 7f1a7c5ff700 1 --2- 172.21.15.69:0/276470534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a74100e10 0x7f1a74101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.665 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.662+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/276470534 >> 172.21.15.69:0/276470534 conn(0x7f1a740fb260 msgr2=0x7f1a740fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:49.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.662+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/276470534 shutdown_connections 2023-07-06T20:37:49.666 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.662+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/276470534 wait complete. 2023-07-06T20:37:49.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.663+0000 7f1a7c5ff700 1 Processor -- start 2023-07-06T20:37:49.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.663+0000 7f1a7c5ff700 1 -- start start 2023-07-06T20:37:49.667 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.663+0000 7f1a7c5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a740ffc20 0x7f1a740743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.663+0000 7f1a7c5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a74100e10 0x7f1a74072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.663+0000 7f1a7c5ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a741017a0 0x7f1a74072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.663+0000 7f1a7c5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a74073710 con 0x7f1a74100e10 2023-07-06T20:37:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.663+0000 7f1a7c5ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a74073880 con 0x7f1a741017a0 2023-07-06T20:37:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.663+0000 7f1a7c5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419fa10 con 0x7f1a740ffc20 2023-07-06T20:37:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.664+0000 7f1a79b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a74100e10 0x7f1a74072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.664+0000 7f1a7ab9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a741017a0 0x7f1a74072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.664+0000 7f1a7a39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a740ffc20 0x7f1a740743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.664+0000 7f1a7ab9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a741017a0 0x7f1a74072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:51906/0 (socket says 172.21.15.69:51906) 2023-07-06T20:37:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.664+0000 7f1a7ab9c700 1 -- 172.21.15.69:0/2786091230 learned_addr learned my addr 172.21.15.69:0/2786091230 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.664+0000 7f1a79b9a700 1 -- 172.21.15.69:0/2786091230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a740ffc20 msgr2=0x7f1a740743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.664+0000 7f1a79b9a700 1 --2- 172.21.15.69:0/2786091230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a740ffc20 0x7f1a740743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.664+0000 7f1a79b9a700 1 -- 172.21.15.69:0/2786091230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a741017a0 msgr2=0x7f1a74072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.664+0000 7f1a79b9a700 1 --2- 172.21.15.69:0/2786091230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a741017a0 0x7f1a74072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.664+0000 7f1a79b9a700 1 -- 172.21.15.69:0/2786091230 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a700095f0 con 0x7f1a74100e10 2023-07-06T20:37:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.664+0000 7f1a79b9a700 1 --2- 172.21.15.69:0/2786091230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a74100e10 0x7f1a74072a30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400e9f0 tx=0x7f1a6400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.664+0000 7f1a6b7fe700 1 -- 172.21.15.69:0/2786091230 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400c9c0 con 0x7f1a74100e10 2023-07-06T20:37:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.665+0000 7f1a6b7fe700 1 -- 172.21.15.69:0/2786091230 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a64010430 con 0x7f1a74100e10 2023-07-06T20:37:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.665+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/2786091230 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7419fcf0 con 0x7f1a74100e10 2023-07-06T20:37:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.665+0000 7f1a6b7fe700 1 -- 172.21.15.69:0/2786091230 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64018680 con 0x7f1a74100e10 2023-07-06T20:37:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.665+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/2786091230 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a741a01b0 con 0x7f1a74100e10 2023-07-06T20:37:49.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.666+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/2786091230 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7410b440 con 0x7f1a74100e10 2023-07-06T20:37:49.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.667+0000 7f1a6b7fe700 1 -- 172.21.15.69:0/2786091230 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1a64010860 con 0x7f1a74100e10 2023-07-06T20:37:49.671 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.669+0000 7f1a6b7fe700 1 --2- 172.21.15.69:0/2786091230 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1a600a1150 0x7f1a600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:49.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.669+0000 7f1a6b7fe700 1 -- 172.21.15.69:0/2786091230 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f1a64014070 con 0x7f1a74100e10 2023-07-06T20:37:49.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.669+0000 7f1a7a39b700 1 --2- 172.21.15.69:0/2786091230 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1a600a1150 0x7f1a600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:49.672 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.670+0000 7f1a7a39b700 1 --2- 172.21.15.69:0/2786091230 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1a600a1150 0x7f1a600a3600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000b2e0 tx=0x7f1a70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:49.675 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.673+0000 7f1a6b7fe700 1 -- 172.21.15.69:0/2786091230 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1a64051770 con 0x7f1a74100e10 2023-07-06T20:37:49.876 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.874+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/2786091230 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a741a0760 con 0x7f1a74100e10 2023-07-06T20:37:49.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.874+0000 7f1a6b7fe700 1 -- 172.21.15.69:0/2786091230 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a64096180 con 0x7f1a74100e10 2023-07-06T20:37:49.877 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2023-07-06T20:37:49.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.876+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/2786091230 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1a600a1150 msgr2=0x7f1a600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:49.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.876+0000 7f1a7c5ff700 1 --2- 172.21.15.69:0/2786091230 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1a600a1150 0x7f1a600a3600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000b2e0 tx=0x7f1a70005fb0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.876+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/2786091230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a74100e10 msgr2=0x7f1a74072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:49.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.876+0000 7f1a7c5ff700 1 --2- 172.21.15.69:0/2786091230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a74100e10 0x7f1a74072a30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400e9f0 tx=0x7f1a6400ed00 comp rx=0 tx=0).stop 2023-07-06T20:37:49.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.877+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/2786091230 shutdown_connections 2023-07-06T20:37:49.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.877+0000 7f1a7c5ff700 1 --2- 172.21.15.69:0/2786091230 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f1a600a1150 0x7f1a600a3600 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.877+0000 7f1a7c5ff700 1 --2- 172.21.15.69:0/2786091230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a741017a0 0x7f1a74072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.877+0000 7f1a7c5ff700 1 --2- 172.21.15.69:0/2786091230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a740ffc20 0x7f1a740743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.877+0000 7f1a7c5ff700 1 --2- 172.21.15.69:0/2786091230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a74100e10 0x7f1a74072a30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:49.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.877+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/2786091230 >> 172.21.15.69:0/2786091230 conn(0x7f1a740fb260 msgr2=0x7f1a740fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:49.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.877+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/2786091230 shutdown_connections 2023-07-06T20:37:49.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:49.877+0000 7f1a7c5ff700 1 -- 172.21.15.69:0/2786091230 wait complete. 2023-07-06T20:37:49.897 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T20:37:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.094+0000 7fcc325ff700 1 -- 172.21.15.69:0/2038677758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc2c101790 msgr2=0x7fcc2c108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.094+0000 7fcc325ff700 1 --2- 172.21.15.69:0/2038677758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc2c101790 0x7fcc2c108050 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400b0a0 tx=0x7fcc2400b3b0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.094+0000 7fcc325ff700 1 -- 172.21.15.69:0/2038677758 shutdown_connections 2023-07-06T20:37:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.094+0000 7fcc325ff700 1 --2- 172.21.15.69:0/2038677758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc2c101790 0x7fcc2c108050 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.094+0000 7fcc325ff700 1 --2- 172.21.15.69:0/2038677758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc2c0ffc10 0x7fcc2c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.094+0000 7fcc325ff700 1 --2- 172.21.15.69:0/2038677758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc2c100e00 0x7fcc2c101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.094+0000 7fcc325ff700 1 -- 172.21.15.69:0/2038677758 >> 172.21.15.69:0/2038677758 conn(0x7fcc2c0fb210 msgr2=0x7fcc2c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.094+0000 7fcc325ff700 1 -- 172.21.15.69:0/2038677758 shutdown_connections 2023-07-06T20:37:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.094+0000 7fcc325ff700 1 -- 172.21.15.69:0/2038677758 wait complete. 2023-07-06T20:37:50.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.095+0000 7fcc325ff700 1 Processor -- start 2023-07-06T20:37:50.099 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.095+0000 7fcc325ff700 1 -- start start 2023-07-06T20:37:50.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc325ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc2c0ffc10 0x7fcc2c197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:50.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc325ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc2c100e00 0x7fcc2c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:50.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc325ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc2c101790 0x7fcc2c19f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:50.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc325ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c19f820 con 0x7fcc2c100e00 2023-07-06T20:37:50.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc325ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c19f990 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.100 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc325ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c19fb50 con 0x7fcc2c101790 2023-07-06T20:37:50.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc2c0ffc10 0x7fcc2c197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:50.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc2b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc2c100e00 0x7fcc2c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:50.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc2c0ffc10 0x7fcc2c197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:51922/0 (socket says 172.21.15.69:51922) 2023-07-06T20:37:50.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc2bfff700 1 -- 172.21.15.69:0/3547226481 learned_addr learned my addr 172.21.15.69:0/3547226481 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:50.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc30b9c700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc2c101790 0x7fcc2c19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:50.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc2bfff700 1 -- 172.21.15.69:0/3547226481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc2c101790 msgr2=0x7fcc2c19f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc2bfff700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc2c101790 0x7fcc2c19f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.101 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc2bfff700 1 -- 172.21.15.69:0/3547226481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc2c100e00 msgr2=0x7fcc2c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc2bfff700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc2c100e00 0x7fcc2c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc2bfff700 1 -- 172.21.15.69:0/3547226481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc24009d00 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.096+0000 7fcc2b7fe700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc2c100e00 0x7fcc2c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:37:50.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.097+0000 7fcc2bfff700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc2c0ffc10 0x7fcc2c197cb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e970 tx=0x7fcc1c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:50.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.097+0000 7fcc297fa700 1 -- 172.21.15.69:0/3547226481 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c009790 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.097+0000 7fcc325ff700 1 -- 172.21.15.69:0/3547226481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2c19fe30 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.097+0000 7fcc297fa700 1 -- 172.21.15.69:0/3547226481 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1c00eec0 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.102 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.097+0000 7fcc297fa700 1 -- 172.21.15.69:0/3547226481 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c003cf0 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.097+0000 7fcc325ff700 1 -- 172.21.15.69:0/3547226481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c1a02b0 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.098+0000 7fcc325ff700 1 -- 172.21.15.69:0/3547226481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2c10b440 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.103 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.100+0000 7fcc297fa700 1 -- 172.21.15.69:0/3547226481 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcc1c00ccd0 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.101+0000 7fcc297fa700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcc140a11d0 0x7fcc140a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:50.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.102+0000 7fcc297fa700 1 -- 172.21.15.69:0/3547226481 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fcc1c014070 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.102+0000 7fcc2b7fe700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcc140a11d0 0x7fcc140a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:50.104 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.102+0000 7fcc2b7fe700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcc140a11d0 0x7fcc140a3680 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c1991b0 tx=0x7fcc20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:50.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:49 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1200785362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:37:50.107 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.105+0000 7fcc297fa700 1 -- 172.21.15.69:0/3547226481 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcc1c096870 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:49 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1200785362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:37:50.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:49 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1200785362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:37:50.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.303+0000 7fcc325ff700 1 -- 172.21.15.69:0/3547226481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc2c1a0b50 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.304+0000 7fcc297fa700 1 -- 172.21.15.69:0/3547226481 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc1c099e90 con 0x7fcc2c0ffc10 2023-07-06T20:37:50.307 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2023-07-06T20:37:50.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.306+0000 7fcc325ff700 1 -- 172.21.15.69:0/3547226481 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcc140a11d0 msgr2=0x7fcc140a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.306+0000 7fcc325ff700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcc140a11d0 0x7fcc140a3680 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c1991b0 tx=0x7fcc20009260 comp rx=0 tx=0).stop 2023-07-06T20:37:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.307+0000 7fcc325ff700 1 -- 172.21.15.69:0/3547226481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc2c0ffc10 msgr2=0x7fcc2c197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.307+0000 7fcc325ff700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc2c0ffc10 0x7fcc2c197cb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e970 tx=0x7fcc1c00ec80 comp rx=0 tx=0).stop 2023-07-06T20:37:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.307+0000 7fcc325ff700 1 -- 172.21.15.69:0/3547226481 shutdown_connections 2023-07-06T20:37:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.307+0000 7fcc325ff700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fcc140a11d0 0x7fcc140a3680 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.307+0000 7fcc325ff700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc2c101790 0x7fcc2c19f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.307+0000 7fcc325ff700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc2c0ffc10 0x7fcc2c197cb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.307+0000 7fcc325ff700 1 --2- 172.21.15.69:0/3547226481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc2c100e00 0x7fcc2c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.307+0000 7fcc325ff700 1 -- 172.21.15.69:0/3547226481 >> 172.21.15.69:0/3547226481 conn(0x7fcc2c0fb210 msgr2=0x7fcc2c106690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.307+0000 7fcc325ff700 1 -- 172.21.15.69:0/3547226481 shutdown_connections 2023-07-06T20:37:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.307+0000 7fcc325ff700 1 -- 172.21.15.69:0/3547226481 wait complete. 2023-07-06T20:37:50.328 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T20:37:50.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.524+0000 7f23123ff700 1 -- 172.21.15.69:0/3982614713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f230c0ffc20 msgr2=0x7f230c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.524+0000 7f23123ff700 1 --2- 172.21.15.69:0/3982614713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f230c0ffc20 0x7f230c100030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009990 tx=0x7f22fc009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.524+0000 7f23123ff700 1 -- 172.21.15.69:0/3982614713 shutdown_connections 2023-07-06T20:37:50.527 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.524+0000 7f23123ff700 1 --2- 172.21.15.69:0/3982614713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f230c1017a0 0x7f230c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.524+0000 7f23123ff700 1 --2- 172.21.15.69:0/3982614713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f230c0ffc20 0x7f230c100030 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.524+0000 7f23123ff700 1 --2- 172.21.15.69:0/3982614713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f230c100e10 0x7f230c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.524+0000 7f23123ff700 1 -- 172.21.15.69:0/3982614713 >> 172.21.15.69:0/3982614713 conn(0x7f230c0fb260 msgr2=0x7f230c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:50.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.524+0000 7f23123ff700 1 -- 172.21.15.69:0/3982614713 shutdown_connections 2023-07-06T20:37:50.528 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.524+0000 7f23123ff700 1 -- 172.21.15.69:0/3982614713 wait complete. 2023-07-06T20:37:50.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.525+0000 7f23123ff700 1 Processor -- start 2023-07-06T20:37:50.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.525+0000 7f23123ff700 1 -- start start 2023-07-06T20:37:50.530 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f23123ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f230c0ffc20 0x7f230c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:50.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f23123ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f230c100e10 0x7f230c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:50.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f23123ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f230c1017a0 0x7f230c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:50.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f23123ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c19f790 con 0x7f230c0ffc20 2023-07-06T20:37:50.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f23123ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c19f900 con 0x7f230c1017a0 2023-07-06T20:37:50.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f23123ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c19fac0 con 0x7f230c100e10 2023-07-06T20:37:50.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f230bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f230c0ffc20 0x7f230c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:50.531 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f231099c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f230c1017a0 0x7f230c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:50.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f230bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f230c0ffc20 0x7f230c197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48176/0 (socket says 172.21.15.69:48176) 2023-07-06T20:37:50.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f231099c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f230c1017a0 0x7f230c19f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:51936/0 (socket says 172.21.15.69:51936) 2023-07-06T20:37:50.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f230bfff700 1 -- 172.21.15.69:0/2947686528 learned_addr learned my addr 172.21.15.69:0/2947686528 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:50.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f230b7fe700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f230c100e10 0x7f230c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:50.532 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f230bfff700 1 -- 172.21.15.69:0/2947686528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f230c100e10 msgr2=0x7f230c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f230bfff700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f230c100e10 0x7f230c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f230bfff700 1 -- 172.21.15.69:0/2947686528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f230c1017a0 msgr2=0x7f230c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f230bfff700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f230c1017a0 0x7f230c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.533 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f230bfff700 1 -- 172.21.15.69:0/2947686528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc0095f0 con 0x7f230c0ffc20 2023-07-06T20:37:50.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.526+0000 7f231099c700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f230c1017a0 0x7f230c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:37:50.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.527+0000 7f230bfff700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f230c0ffc20 0x7f230c197c50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f22fc005740 tx=0x7f22fc005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:50.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.527+0000 7f23097fa700 1 -- 172.21.15.69:0/2947686528 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc01d070 con 0x7f230c0ffc20 2023-07-06T20:37:50.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.527+0000 7f23123ff700 1 -- 172.21.15.69:0/2947686528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230c19fd40 con 0x7f230c0ffc20 2023-07-06T20:37:50.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.527+0000 7f23097fa700 1 -- 172.21.15.69:0/2947686528 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22fc021410 con 0x7f230c0ffc20 2023-07-06T20:37:50.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.527+0000 7f23097fa700 1 -- 172.21.15.69:0/2947686528 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc017650 con 0x7f230c0ffc20 2023-07-06T20:37:50.534 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.527+0000 7f23123ff700 1 -- 172.21.15.69:0/2947686528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c1a0170 con 0x7f230c0ffc20 2023-07-06T20:37:50.535 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.528+0000 7f23123ff700 1 -- 172.21.15.69:0/2947686528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230c192300 con 0x7f230c0ffc20 2023-07-06T20:37:50.535 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.530+0000 7f23097fa700 1 -- 172.21.15.69:0/2947686528 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f22fc021580 con 0x7f230c0ffc20 2023-07-06T20:37:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.531+0000 7f23097fa700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f22f40a11d0 0x7f22f40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.532+0000 7f23097fa700 1 -- 172.21.15.69:0/2947686528 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f22fc0ce380 con 0x7f230c0ffc20 2023-07-06T20:37:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.532+0000 7f230b7fe700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f22f40a11d0 0x7f22f40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.532+0000 7f230b7fe700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f22f40a11d0 0x7f22f40a3680 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2300009bf0 tx=0x7f2300009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.533+0000 7f23097fa700 1 -- 172.21.15.69:0/2947686528 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f22fc097b60 con 0x7f230c0ffc20 2023-07-06T20:37:50.733 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.731+0000 7f23123ff700 1 -- 172.21.15.69:0/2947686528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f230c02cb30 con 0x7f230c0ffc20 2023-07-06T20:37:50.734 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.731+0000 7f23097fa700 1 -- 172.21.15.69:0/2947686528 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f22fc09b180 con 0x7f230c0ffc20 2023-07-06T20:37:50.734 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2023-07-06T20:37:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.734+0000 7f23123ff700 1 -- 172.21.15.69:0/2947686528 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f22f40a11d0 msgr2=0x7f22f40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.734+0000 7f23123ff700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f22f40a11d0 0x7f22f40a3680 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2300009bf0 tx=0x7f2300009260 comp rx=0 tx=0).stop 2023-07-06T20:37:50.736 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.734+0000 7f23123ff700 1 -- 172.21.15.69:0/2947686528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f230c0ffc20 msgr2=0x7f230c197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.734+0000 7f23123ff700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f230c0ffc20 0x7f230c197c50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f22fc005740 tx=0x7f22fc005770 comp rx=0 tx=0).stop 2023-07-06T20:37:50.737 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.734+0000 7f23123ff700 1 -- 172.21.15.69:0/2947686528 shutdown_connections 2023-07-06T20:37:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.734+0000 7f23123ff700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f22f40a11d0 0x7f22f40a3680 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.734+0000 7f23123ff700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f230c1017a0 0x7f230c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.734+0000 7f23123ff700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f230c0ffc20 0x7f230c197c50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.734+0000 7f23123ff700 1 --2- 172.21.15.69:0/2947686528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f230c100e10 0x7f230c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.734+0000 7f23123ff700 1 -- 172.21.15.69:0/2947686528 >> 172.21.15.69:0/2947686528 conn(0x7f230c0fb260 msgr2=0x7f230c106770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.734+0000 7f23123ff700 1 -- 172.21.15.69:0/2947686528 shutdown_connections 2023-07-06T20:37:50.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.734+0000 7f23123ff700 1 -- 172.21.15.69:0/2947686528 wait complete. 2023-07-06T20:37:50.756 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T20:37:50.954 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.951+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3265181391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ae40ffc10 msgr2=0x7f0ae4100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.954 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.951+0000 7f0ae8bff700 1 --2- 172.21.15.69:0/3265181391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ae40ffc10 0x7f0ae4100020 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009990 tx=0x7f0ad4009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.954 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.951+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3265181391 shutdown_connections 2023-07-06T20:37:50.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.951+0000 7f0ae8bff700 1 --2- 172.21.15.69:0/3265181391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae4101790 0x7f0ae4108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.951+0000 7f0ae8bff700 1 --2- 172.21.15.69:0/3265181391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ae40ffc10 0x7f0ae4100020 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.951+0000 7f0ae8bff700 1 --2- 172.21.15.69:0/3265181391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ae4100e00 0x7f0ae4101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.951+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3265181391 >> 172.21.15.69:0/3265181391 conn(0x7f0ae40fb210 msgr2=0x7f0ae40fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:50.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.951+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3265181391 shutdown_connections 2023-07-06T20:37:50.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.951+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3265181391 wait complete. 2023-07-06T20:37:50.959 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.952+0000 7f0ae8bff700 1 Processor -- start 2023-07-06T20:37:50.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.952+0000 7f0ae8bff700 1 -- start start 2023-07-06T20:37:50.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae8bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ae40ffc10 0x7f0ae40743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:50.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae8bff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae4100e00 0x7f0ae4072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:50.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae8bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ae4101790 0x7f0ae4072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:50.961 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae8bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae4073710 con 0x7f0ae4101790 2023-07-06T20:37:50.961 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae8bff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae4073880 con 0x7f0ae4100e00 2023-07-06T20:37:50.961 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae8bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae419f9f0 con 0x7f0ae40ffc10 2023-07-06T20:37:50.961 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae1d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae4100e00 0x7f0ae4072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:50.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae1d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae4100e00 0x7f0ae4072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:51944/0 (socket says 172.21.15.69:51944) 2023-07-06T20:37:50.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae1d9b700 1 -- 172.21.15.69:0/3516938135 learned_addr learned my addr 172.21.15.69:0/3516938135 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:50.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae2d9d700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ae4101790 0x7f0ae4072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:50.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae259c700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ae40ffc10 0x7f0ae40743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:50.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae2d9d700 1 -- 172.21.15.69:0/3516938135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ae40ffc10 msgr2=0x7f0ae40743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae2d9d700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ae40ffc10 0x7f0ae40743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae2d9d700 1 -- 172.21.15.69:0/3516938135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae4100e00 msgr2=0x7f0ae4072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:50.963 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae2d9d700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae4100e00 0x7f0ae4072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:50.964 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae2d9d700 1 -- 172.21.15.69:0/3516938135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad40095f0 con 0x7f0ae4101790 2023-07-06T20:37:50.964 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae1d9b700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae4100e00 0x7f0ae4072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T20:37:50.964 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.953+0000 7f0ae2d9d700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ae4101790 0x7f0ae4072f70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00efc0 tx=0x7f0adc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:50.964 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.954+0000 7f0ad37fe700 1 -- 172.21.15.69:0/3516938135 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc018070 con 0x7f0ae4101790 2023-07-06T20:37:50.964 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.954+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3516938135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae419fc70 con 0x7f0ae4101790 2023-07-06T20:37:50.964 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.954+0000 7f0ad37fe700 1 -- 172.21.15.69:0/3516938135 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0adc00f040 con 0x7f0ae4101790 2023-07-06T20:37:50.965 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.954+0000 7f0ad37fe700 1 -- 172.21.15.69:0/3516938135 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00cd40 con 0x7f0ae4101790 2023-07-06T20:37:50.965 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.954+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3516938135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae41a0130 con 0x7f0ae4101790 2023-07-06T20:37:50.965 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.955+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3516938135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae410b430 con 0x7f0ae4101790 2023-07-06T20:37:50.965 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.957+0000 7f0ad37fe700 1 -- 172.21.15.69:0/3516938135 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0adc003a00 con 0x7f0ae4101790 2023-07-06T20:37:50.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.959+0000 7f0ad37fe700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0acc0a1150 0x7f0acc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:50.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.959+0000 7f0ad37fe700 1 -- 172.21.15.69:0/3516938135 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f0adc0cc6b0 con 0x7f0ae4101790 2023-07-06T20:37:50.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.959+0000 7f0ae259c700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0acc0a1150 0x7f0acc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:50.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.960+0000 7f0ae259c700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0acc0a1150 0x7f0acc0a3600 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400b2e0 tx=0x7f0ad4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:50.966 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:50.962+0000 7f0ad37fe700 1 -- 172.21.15.69:0/3516938135 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0adc095e00 con 0x7f0ae4101790 2023-07-06T20:37:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:50 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2786091230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:37:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:50 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3547226481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:37:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:50 smithi069 ceph-mon[126399]: pgmap v883: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 6 op/s 2023-07-06T20:37:51.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:50 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/2947686528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:37:51.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:50 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2786091230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:37:51.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:50 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3547226481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:37:51.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:50 smithi156 ceph-mon[129280]: pgmap v883: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 6 op/s 2023-07-06T20:37:51.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:50 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/2947686528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:37:51.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:50 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2786091230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-06T20:37:51.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:50 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3547226481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-06T20:37:51.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:50 smithi098 ceph-mon[128968]: pgmap v883: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 6 op/s 2023-07-06T20:37:51.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:50 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/2947686528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-06T20:37:51.180 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.178+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3516938135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0ae4106b40 con 0x7f0ae4101790 2023-07-06T20:37:51.181 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.179+0000 7f0ad37fe700 1 -- 172.21.15.69:0/3516938135 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+584 (secure 0 0 0) 0x7f0adc099420 con 0x7f0ae4101790 2023-07-06T20:37:51.181 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1409752664 2023-07-07T20:10:57.432754+0000 2023-07-06T20:37:51.182 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/9699389 2023-07-07T20:10:57.432754+0000 2023-07-06T20:37:51.182 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/9699389 2023-07-07T20:10:57.432754+0000 2023-07-06T20:37:51.182 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/20539643 2023-07-07T20:10:57.432754+0000 2023-07-06T20:37:51.182 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3832776393 2023-07-07T20:10:31.404980+0000 2023-07-06T20:37:51.182 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1479828612 2023-07-07T20:10:31.404980+0000 2023-07-06T20:37:51.182 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/1829009377 2023-07-07T20:10:31.404980+0000 2023-07-06T20:37:51.182 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/1829009377 2023-07-07T20:10:31.404980+0000 2023-07-06T20:37:51.183 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4048964594 2023-07-07T20:10:31.404980+0000 2023-07-06T20:37:51.183 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2854344202 2023-07-07T20:10:57.432754+0000 2023-07-06T20:37:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.181+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3516938135 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0acc0a1150 msgr2=0x7f0acc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.181+0000 7f0ae8bff700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0acc0a1150 0x7f0acc0a3600 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400b2e0 tx=0x7f0ad4005fb0 comp rx=0 tx=0).stop 2023-07-06T20:37:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.181+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3516938135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ae4101790 msgr2=0x7f0ae4072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.181+0000 7f0ae8bff700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ae4101790 0x7f0ae4072f70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00efc0 tx=0x7f0adc00c3a0 comp rx=0 tx=0).stop 2023-07-06T20:37:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.181+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3516938135 shutdown_connections 2023-07-06T20:37:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.181+0000 7f0ae8bff700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f0acc0a1150 0x7f0acc0a3600 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.181+0000 7f0ae8bff700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ae4101790 0x7f0ae4072f70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.181+0000 7f0ae8bff700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ae40ffc10 0x7f0ae40743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.181+0000 7f0ae8bff700 1 --2- 172.21.15.69:0/3516938135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae4100e00 0x7f0ae4072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.181+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3516938135 >> 172.21.15.69:0/3516938135 conn(0x7f0ae40fb210 msgr2=0x7f0ae40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.181+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3516938135 shutdown_connections 2023-07-06T20:37:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:51.181+0000 7f0ae8bff700 1 -- 172.21.15.69:0/3516938135 wait complete. 2023-07-06T20:37:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:listed 10 entries 2023-07-06T20:37:51.203 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-06T20:37:51.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:37:51.203 DEBUG:teuthology.orchestra.run.smithi069:> sudo umount /tmp/cephfs 2023-07-06T20:37:52.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:51 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/3516938135' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-06T20:37:52.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:51 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/3516938135' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-06T20:37:52.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:51 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/3516938135' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-06T20:37:52.925 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2023-07-06T20:37:52.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:37:52.925 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-07-06T20:37:53.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:52 smithi156 ceph-mon[129280]: pgmap v884: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2023-07-06T20:37:53.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:52 smithi098 ceph-mon[128968]: pgmap v884: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2023-07-06T20:37:53.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:37:53.146 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2023-07-06T20:37:53.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:52 smithi069 ceph-mon[126399]: pgmap v884: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2023-07-06T20:37:53.368 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-07-06T20:37:53.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:37:53.369 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-07-06T20:37:53.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:37:53.715 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2023-07-06T20:37:53.897 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-06T20:37:53.897 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-07-06T20:37:53.897 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T20:37:53.943 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345556k,nr_inodes=4086389,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/53506a19ccb5c48f929361336ca24b8f9a68497207af03b41ad80e24653ab2ae/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/U3UEW53TTXPDWWTKM4DKJQ34HU:/var/lib/containers/storage/overlay/l/L5HTXFYIRDZJPFXCL5WAVZUXYS,upperdir=/var/lib/containers/storage/overlay/53506a19ccb5c48f929361336ca24b8f9a68497207af03b41ad80e24653ab2ae/diff,workdir=/var/lib/containers/storage/overlay/53506a19ccb5c48f929361336ca24b8f9a68497207af03b41ad80e24653ab2ae/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/638f84c5ab8e298dd42fac5518cfdc635dba0c1166fe93e701da8dd54f444248/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/U3UEW53TTXPDWWTKM4DKJQ34HU:/var/lib/containers/storage/overlay/l/L5HTXFYIRDZJPFXCL5WAVZUXYS,upperdir=/var/lib/containers/storage/overlay/638f84c5ab8e298dd42fac5518cfdc635dba0c1166fe93e701da8dd54f444248/diff,workdir=/var/lib/containers/storage/overlay/638f84c5ab8e298dd42fac5518cfdc635dba0c1166fe93e701da8dd54f444248/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/820f7f3b5dc1ed3e1ac52538de194054883c8b945e67216056f6692069352224/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/U3UEW53TTXPDWWTKM4DKJQ34HU:/var/lib/containers/storage/overlay/l/L5HTXFYIRDZJPFXCL5WAVZUXYS,upperdir=/var/lib/containers/storage/overlay/820f7f3b5dc1ed3e1ac52538de194054883c8b945e67216056f6692069352224/diff,workdir=/var/lib/containers/storage/overlay/820f7f3b5dc1ed3e1ac52538de194054883c8b945e67216056f6692069352224/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8a8bbd5c49e56dc858a50a602bf1491812a0782f616efacdc91328596dcf0994/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/U3UEW53TTXPDWWTKM4DKJQ34HU:/var/lib/containers/storage/overlay/l/L5HTXFYIRDZJPFXCL5WAVZUXYS,upperdir=/var/lib/containers/storage/overlay/8a8bbd5c49e56dc858a50a602bf1491812a0782f616efacdc91328596dcf0994/diff,workdir=/var/lib/containers/storage/overlay/8a8bbd5c49e56dc858a50a602bf1491812a0782f616efacdc91328596dcf0994/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/02cac95171115a63529bd794c2d9c66a4676aca72b4c220d29b33164067349a4/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/U3UEW53TTXPDWWTKM4DKJQ34HU:/var/lib/containers/storage/overlay/l/L5HTXFYIRDZJPFXCL5WAVZUXYS,upperdir=/var/lib/containers/storage/overlay/02cac95171115a63529bd794c2d9c66a4676aca72b4c220d29b33164067349a4/diff,workdir=/var/lib/containers/storage/overlay/02cac95171115a63529bd794c2d9c66a4676aca72b4c220d29b33164067349a4/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/29735b706efd8af7c003030c9b517aceee184e0e44802f598509fe91cb6c46dc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/U3UEW53TTXPDWWTKM4DKJQ34HU:/var/lib/containers/storage/overlay/l/L5HTXFYIRDZJPFXCL5WAVZUXYS,upperdir=/var/lib/containers/storage/overlay/29735b706efd8af7c003030c9b517aceee184e0e44802f598509fe91cb6c46dc/diff,workdir=/var/lib/containers/storage/overlay/29735b706efd8af7c003030c9b517aceee184e0e44802f598509fe91cb6c46dc/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/773028e253e9c34ce13a105704f6830e582de13834bb70be4bfb06b6a31f3ecd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/U3UEW53TTXPDWWTKM4DKJQ34HU:/var/lib/containers/storage/overlay/l/L5HTXFYIRDZJPFXCL5WAVZUXYS,upperdir=/var/lib/containers/storage/overlay/773028e253e9c34ce13a105704f6830e582de13834bb70be4bfb06b6a31f3ecd/diff,workdir=/var/lib/containers/storage/overlay/773028e253e9c34ce13a105704f6830e582de13834bb70be4bfb06b6a31f3ecd/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b51d8c36c198e378572788dd59fa8739961a694deda7e7042fd721e3c3366885/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/U3UEW53TTXPDWWTKM4DKJQ34HU:/var/lib/containers/storage/overlay/l/L5HTXFYIRDZJPFXCL5WAVZUXYS,upperdir=/var/lib/containers/storage/overlay/b51d8c36c198e378572788dd59fa8739961a694deda7e7042fd721e3c3366885/diff,workdir=/var/lib/containers/storage/overlay/b51d8c36c198e378572788dd59fa8739961a694deda7e7042fd721e3c3366885/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b3ba1a53e6afdcbccb5d6c73230ce728aefa14c223b43f093165d0a71e169982/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/U3UEW53TTXPDWWTKM4DKJQ34HU:/var/lib/containers/storage/overlay/l/L5HTXFYIRDZJPFXCL5WAVZUXYS,upperdir=/var/lib/containers/storage/overlay/b3ba1a53e6afdcbccb5d6c73230ce728aefa14c223b43f093165d0a71e169982/diff,workdir=/var/lib/containers/storage/overlay/b3ba1a53e6afdcbccb5d6c73230ce728aefa14c223b43f093165d0a71e169982/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d9b5aa93ca68c8c21984ba10971a9ea0d013e1212dfd393939a26260dfc27c20/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/U3UEW53TTXPDWWTKM4DKJQ34HU:/var/lib/containers/storage/overlay/l/L5HTXFYIRDZJPFXCL5WAVZUXYS,upperdir=/var/lib/containers/storage/overlay/d9b5aa93ca68c8c21984ba10971a9ea0d013e1212dfd393939a26260dfc27c20/diff,workdir=/var/lib/containers/storage/overlay/d9b5aa93ca68c8c21984ba10971a9ea0d013e1212dfd393939a26260dfc27c20/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-07-06T20:37:53.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:37:53.944 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2023-07-06T20:37:54.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T20:37:54.027 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2023-07-06T20:37:54.139 INFO:teuthology.orchestra.run.smithi069.stderr:Cannot find device "ceph-brx" 2023-07-06T20:37:54.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:37:54.142 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-06T20:37:54.153 INFO:tasks.cephadm:Teardown begin 2023-07-06T20:37:54.153 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T20:37:54.154 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:37:54.351 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.348+0000 7fad1b9ff700 1 -- 172.21.15.69:0/326692709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad140ffc30 msgr2=0x7fad14100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:54.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.348+0000 7fad1b9ff700 1 --2- 172.21.15.69:0/326692709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad140ffc30 0x7fad14100040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fad10009990 tx=0x7fad10009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.348+0000 7fad1b9ff700 1 -- 172.21.15.69:0/326692709 shutdown_connections 2023-07-06T20:37:54.352 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.348+0000 7fad1b9ff700 1 --2- 172.21.15.69:0/326692709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad141017b0 0x7fad14108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.348+0000 7fad1b9ff700 1 --2- 172.21.15.69:0/326692709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad140ffc30 0x7fad14100040 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.348+0000 7fad1b9ff700 1 --2- 172.21.15.69:0/326692709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad14100e20 0x7fad14101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.348+0000 7fad1b9ff700 1 -- 172.21.15.69:0/326692709 >> 172.21.15.69:0/326692709 conn(0x7fad140fb210 msgr2=0x7fad140fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:54.353 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.348+0000 7fad1b9ff700 1 -- 172.21.15.69:0/326692709 shutdown_connections 2023-07-06T20:37:54.354 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.348+0000 7fad1b9ff700 1 -- 172.21.15.69:0/326692709 wait complete. 2023-07-06T20:37:54.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.349+0000 7fad1b9ff700 1 Processor -- start 2023-07-06T20:37:54.357 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad1b9ff700 1 -- start start 2023-07-06T20:37:54.358 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad1b9ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad140ffc30 0x7fad140743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:54.358 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad1b9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad14100e20 0x7fad14072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:54.358 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad1b9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad141017b0 0x7fad14072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:54.358 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad1b9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad14073710 con 0x7fad14100e20 2023-07-06T20:37:54.359 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad1b9ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad14073880 con 0x7fad140ffc30 2023-07-06T20:37:54.359 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad1b9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1419f940 con 0x7fad141017b0 2023-07-06T20:37:54.359 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad1979b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad140ffc30 0x7fad140743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:54.359 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad1979b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad140ffc30 0x7fad140743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:43392/0 (socket says 172.21.15.69:43392) 2023-07-06T20:37:54.359 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad1979b700 1 -- 172.21.15.69:0/186100959 learned_addr learned my addr 172.21.15.69:0/186100959 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:54.360 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad18f9a700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad14100e20 0x7fad14072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:54.360 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad19f9c700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad141017b0 0x7fad14072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:54.360 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.350+0000 7fad1979b700 1 -- 172.21.15.69:0/186100959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad141017b0 msgr2=0x7fad14072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:54.360 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.351+0000 7fad1979b700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad141017b0 0x7fad14072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.360 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.351+0000 7fad1979b700 1 -- 172.21.15.69:0/186100959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad14100e20 msgr2=0x7fad14072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:54.360 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.351+0000 7fad1979b700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad14100e20 0x7fad14072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.360 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.351+0000 7fad1979b700 1 -- 172.21.15.69:0/186100959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad04009520 con 0x7fad140ffc30 2023-07-06T20:37:54.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.351+0000 7fad19f9c700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad141017b0 0x7fad14072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:37:54.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.351+0000 7fad1979b700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad140ffc30 0x7fad140743f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad10005720 tx=0x7fad10009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:54.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.351+0000 7fad0a7fc700 1 -- 172.21.15.69:0/186100959 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1001d070 con 0x7fad140ffc30 2023-07-06T20:37:54.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.351+0000 7fad1b9ff700 1 -- 172.21.15.69:0/186100959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad100095f0 con 0x7fad140ffc30 2023-07-06T20:37:54.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.351+0000 7fad0a7fc700 1 -- 172.21.15.69:0/186100959 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad10021410 con 0x7fad140ffc30 2023-07-06T20:37:54.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.351+0000 7fad0a7fc700 1 -- 172.21.15.69:0/186100959 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10017650 con 0x7fad140ffc30 2023-07-06T20:37:54.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.351+0000 7fad1b9ff700 1 -- 172.21.15.69:0/186100959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1419ff00 con 0x7fad140ffc30 2023-07-06T20:37:54.361 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.352+0000 7fad1b9ff700 1 -- 172.21.15.69:0/186100959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad14107180 con 0x7fad140ffc30 2023-07-06T20:37:54.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.354+0000 7fad0a7fc700 1 -- 172.21.15.69:0/186100959 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fad10021580 con 0x7fad140ffc30 2023-07-06T20:37:54.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.356+0000 7fad0a7fc700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fad000a11a0 0x7fad000a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:54.362 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.356+0000 7fad0a7fc700 1 -- 172.21.15.69:0/186100959 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fad100ce420 con 0x7fad140ffc30 2023-07-06T20:37:54.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.356+0000 7fad18f9a700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fad000a11a0 0x7fad000a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:54.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.356+0000 7fad18f9a700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fad000a11a0 0x7fad000a3650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fad04009df0 tx=0x7fad04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:54.363 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.359+0000 7fad0a7fc700 1 -- 172.21.15.69:0/186100959 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fad10097b70 con 0x7fad140ffc30 2023-07-06T20:37:54.603 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.601+0000 7fad1b9ff700 1 -- 172.21.15.69:0/186100959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad1419fb00 con 0x7fad140ffc30 2023-07-06T20:37:54.604 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.602+0000 7fad0a7fc700 1 -- 172.21.15.69:0/186100959 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10795 (secure 0 0 0) 0x7fad1009b190 con 0x7fad140ffc30 2023-07-06T20:37:54.604 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:54.605 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-06T20:17:28.144582+0000","modified":"2023-07-06T20:22:22.079611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24539,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/3235525224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3235525224},{"type":"v1","addr":"172.21.15.98:6839","nonce":3235525224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:37:54.606 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.604+0000 7fad1b9ff700 1 -- 172.21.15.69:0/186100959 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fad000a11a0 msgr2=0x7fad000a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:54.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.604+0000 7fad1b9ff700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fad000a11a0 0x7fad000a3650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fad04009df0 tx=0x7fad04009260 comp rx=0 tx=0).stop 2023-07-06T20:37:54.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.604+0000 7fad1b9ff700 1 -- 172.21.15.69:0/186100959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad140ffc30 msgr2=0x7fad140743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:54.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.604+0000 7fad1b9ff700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad140ffc30 0x7fad140743f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad10005720 tx=0x7fad10009580 comp rx=0 tx=0).stop 2023-07-06T20:37:54.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.604+0000 7fad1b9ff700 1 -- 172.21.15.69:0/186100959 shutdown_connections 2023-07-06T20:37:54.607 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.604+0000 7fad1b9ff700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fad000a11a0 0x7fad000a3650 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.604+0000 7fad1b9ff700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad141017b0 0x7fad14072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.604+0000 7fad1b9ff700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad140ffc30 0x7fad140743f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.604+0000 7fad1b9ff700 1 --2- 172.21.15.69:0/186100959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad14100e20 0x7fad14072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.604+0000 7fad1b9ff700 1 -- 172.21.15.69:0/186100959 >> 172.21.15.69:0/186100959 conn(0x7fad140fb210 msgr2=0x7fad141042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:54.608 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.604+0000 7fad1b9ff700 1 -- 172.21.15.69:0/186100959 shutdown_connections 2023-07-06T20:37:54.609 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.604+0000 7fad1b9ff700 1 -- 172.21.15.69:0/186100959 wait complete. 2023-07-06T20:37:54.609 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 30 2023-07-06T20:37:54.625 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T20:37:54.828 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.826+0000 7fc52ddff700 1 -- 172.21.15.69:0/4225973846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc528100e20 msgr2=0x7fc528101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:54.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.826+0000 7fc52ddff700 1 --2- 172.21.15.69:0/4225973846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc528100e20 0x7fc528101270 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0098f0 tx=0x7fc51c009c00 comp rx=0 tx=0).stop 2023-07-06T20:37:54.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.826+0000 7fc52ddff700 1 -- 172.21.15.69:0/4225973846 shutdown_connections 2023-07-06T20:37:54.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.826+0000 7fc52ddff700 1 --2- 172.21.15.69:0/4225973846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5281017b0 0x7fc528108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.826+0000 7fc52ddff700 1 --2- 172.21.15.69:0/4225973846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5280ffc30 0x7fc528100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.829 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.826+0000 7fc52ddff700 1 --2- 172.21.15.69:0/4225973846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc528100e20 0x7fc528101270 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.826+0000 7fc52ddff700 1 -- 172.21.15.69:0/4225973846 >> 172.21.15.69:0/4225973846 conn(0x7fc5280fb210 msgr2=0x7fc5280fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:54.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.826+0000 7fc52ddff700 1 -- 172.21.15.69:0/4225973846 shutdown_connections 2023-07-06T20:37:54.830 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.826+0000 7fc52ddff700 1 -- 172.21.15.69:0/4225973846 wait complete. 2023-07-06T20:37:54.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.827+0000 7fc52ddff700 1 Processor -- start 2023-07-06T20:37:54.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.827+0000 7fc52ddff700 1 -- start start 2023-07-06T20:37:54.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.827+0000 7fc52ddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5280ffc30 0x7fc528197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:54.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.827+0000 7fc52ddff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc528100e20 0x7fc528198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:54.832 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc52ddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5281017b0 0x7fc52819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:54.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc52ddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52819f780 con 0x7fc5281017b0 2023-07-06T20:37:54.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc52ddff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52819f8f0 con 0x7fc528100e20 2023-07-06T20:37:54.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc52ddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52819fab0 con 0x7fc5280ffc30 2023-07-06T20:37:54.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc526ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc528100e20 0x7fc528198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:54.833 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc526ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc528100e20 0x7fc528198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:43408/0 (socket says 172.21.15.69:43408) 2023-07-06T20:37:54.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc526ffd700 1 -- 172.21.15.69:0/917726843 learned_addr learned my addr 172.21.15.69:0/917726843 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:54.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc5277fe700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5280ffc30 0x7fc528197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:54.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc527fff700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5281017b0 0x7fc52819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:54.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc526ffd700 1 -- 172.21.15.69:0/917726843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5280ffc30 msgr2=0x7fc528197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:54.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc526ffd700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5280ffc30 0x7fc528197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.834 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc526ffd700 1 -- 172.21.15.69:0/917726843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5281017b0 msgr2=0x7fc52819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:54.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc526ffd700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5281017b0 0x7fc52819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:54.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc526ffd700 1 -- 172.21.15.69:0/917726843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc520009d00 con 0x7fc528100e20 2023-07-06T20:37:54.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc527fff700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5281017b0 0x7fc52819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T20:37:54.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.828+0000 7fc526ffd700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc528100e20 0x7fc528198180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc51c005e90 tx=0x7fc51c00b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:54.835 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.829+0000 7fc524ff9700 1 -- 172.21.15.69:0/917726843 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c01c070 con 0x7fc528100e20 2023-07-06T20:37:54.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.829+0000 7fc52ddff700 1 -- 172.21.15.69:0/917726843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c009520 con 0x7fc528100e20 2023-07-06T20:37:54.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.829+0000 7fc524ff9700 1 -- 172.21.15.69:0/917726843 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc51c00f7f0 con 0x7fc528100e20 2023-07-06T20:37:54.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.829+0000 7fc524ff9700 1 -- 172.21.15.69:0/917726843 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c004b30 con 0x7fc528100e20 2023-07-06T20:37:54.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.829+0000 7fc52ddff700 1 -- 172.21.15.69:0/917726843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5281a0020 con 0x7fc528100e20 2023-07-06T20:37:54.836 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.830+0000 7fc52ddff700 1 -- 172.21.15.69:0/917726843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc528192310 con 0x7fc528100e20 2023-07-06T20:37:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.832+0000 7fc524ff9700 1 -- 172.21.15.69:0/917726843 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc51c020070 con 0x7fc528100e20 2023-07-06T20:37:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.833+0000 7fc524ff9700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5100a11a0 0x7fc5100a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.833+0000 7fc524ff9700 1 -- 172.21.15.69:0/917726843 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fc51c0cdea0 con 0x7fc528100e20 2023-07-06T20:37:54.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.833+0000 7fc5277fe700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5100a11a0 0x7fc5100a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:54.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.834+0000 7fc5277fe700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5100a11a0 0x7fc5100a3650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc528199110 tx=0x7fc518009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:54.838 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:54.836+0000 7fc524ff9700 1 -- 172.21.15.69:0/917726843 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc51c0975f0 con 0x7fc528100e20 2023-07-06T20:37:55.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:54 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/186100959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:37:55.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:54 smithi069 ceph-mon[126399]: pgmap v885: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 18 KiB/s wr, 8 op/s 2023-07-06T20:37:55.087 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.085+0000 7fc52ddff700 1 -- 172.21.15.69:0/917726843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc52802c8e0 con 0x7fc528100e20 2023-07-06T20:37:55.088 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.086+0000 7fc524ff9700 1 -- 172.21.15.69:0/917726843 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10795 (secure 0 0 0) 0x7fc51c09ac10 con 0x7fc528100e20 2023-07-06T20:37:55.089 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:55.089 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3728004874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3728004874},{"type":"v1","addr":"172.21.15.69:6835","nonce":3728004874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/896595344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":896595344},{"type":"v1","addr":"172.21.15.69:6837","nonce":896595344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1973475728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1973475728},{"type":"v1","addr":"172.21.15.69:6839","nonce":1973475728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/1464129965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":1464129965},{"type":"v1","addr":"172.21.15.69:6841","nonce":1464129965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1091328636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1091328636},{"type":"v1","addr":"172.21.15.156:6833","nonce":1091328636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3115382908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3115382908},{"type":"v1","addr":"172.21.15.98:6833","nonce":3115382908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-06T20:17:28.144582+0000","modified":"2023-07-06T20:22:22.079611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24584,"mds_1":24539,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2525319359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2525319359},{"type":"v1","addr":"172.21.15.156:6835","nonce":2525319359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.98:6835/827333150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":827333150},{"type":"v1","addr":"172.21.15.98:6835","nonce":827333150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/595692705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":595692705},{"type":"v1","addr":"172.21.15.156:6837","nonce":595692705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6839/4200610083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":4200610083},{"type":"v1","addr":"172.21.15.156:6839","nonce":4200610083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/4010267682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":4010267682},{"type":"v1","addr":"172.21.15.98:6837","nonce":4010267682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/3235525224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3235525224},{"type":"v1","addr":"172.21.15.98:6839","nonce":3235525224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T20:37:55.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.089+0000 7fc52ddff700 1 -- 172.21.15.69:0/917726843 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5100a11a0 msgr2=0x7fc5100a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:55.091 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.089+0000 7fc52ddff700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5100a11a0 0x7fc5100a3650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc528199110 tx=0x7fc518009260 comp rx=0 tx=0).stop 2023-07-06T20:37:55.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.089+0000 7fc52ddff700 1 -- 172.21.15.69:0/917726843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc528100e20 msgr2=0x7fc528198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:55.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.089+0000 7fc52ddff700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc528100e20 0x7fc528198180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc51c005e90 tx=0x7fc51c00b730 comp rx=0 tx=0).stop 2023-07-06T20:37:55.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.089+0000 7fc52ddff700 1 -- 172.21.15.69:0/917726843 shutdown_connections 2023-07-06T20:37:55.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.089+0000 7fc52ddff700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fc5100a11a0 0x7fc5100a3650 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.089+0000 7fc52ddff700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5281017b0 0x7fc52819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.089+0000 7fc52ddff700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5280ffc30 0x7fc528197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.092 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.089+0000 7fc52ddff700 1 --2- 172.21.15.69:0/917726843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc528100e20 0x7fc528198180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.093 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.089+0000 7fc52ddff700 1 -- 172.21.15.69:0/917726843 >> 172.21.15.69:0/917726843 conn(0x7fc5280fb210 msgr2=0x7fc528106710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:55.093 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.089+0000 7fc52ddff700 1 -- 172.21.15.69:0/917726843 shutdown_connections 2023-07-06T20:37:55.093 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.089+0000 7fc52ddff700 1 -- 172.21.15.69:0/917726843 wait complete. 2023-07-06T20:37:55.093 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 30 2023-07-06T20:37:55.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:54 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/186100959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:37:55.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:54 smithi156 ceph-mon[129280]: pgmap v885: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 18 KiB/s wr, 8 op/s 2023-07-06T20:37:55.110 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T20:37:55.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:54 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/186100959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:37:55.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:54 smithi098 ceph-mon[128968]: pgmap v885: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 18 KiB/s wr, 8 op/s 2023-07-06T20:37:55.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.304+0000 7f16365ff700 1 -- 172.21.15.69:0/2260509377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16300ffc30 msgr2=0x7f1630100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:55.306 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.304+0000 7f16365ff700 1 --2- 172.21.15.69:0/2260509377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16300ffc30 0x7f1630100040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f16200098f0 tx=0x7f1620009c00 comp rx=0 tx=0).stop 2023-07-06T20:37:55.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.304+0000 7f16365ff700 1 -- 172.21.15.69:0/2260509377 shutdown_connections 2023-07-06T20:37:55.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.304+0000 7f16365ff700 1 --2- 172.21.15.69:0/2260509377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16301017b0 0x7f1630108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.304+0000 7f16365ff700 1 --2- 172.21.15.69:0/2260509377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16300ffc30 0x7f1630100040 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.304+0000 7f16365ff700 1 --2- 172.21.15.69:0/2260509377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1630100e20 0x7f1630101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.304+0000 7f16365ff700 1 -- 172.21.15.69:0/2260509377 >> 172.21.15.69:0/2260509377 conn(0x7f16300fb210 msgr2=0x7f16300fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:55.307 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.304+0000 7f16365ff700 1 -- 172.21.15.69:0/2260509377 shutdown_connections 2023-07-06T20:37:55.308 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.304+0000 7f16365ff700 1 -- 172.21.15.69:0/2260509377 wait complete. 2023-07-06T20:37:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.305+0000 7f16365ff700 1 Processor -- start 2023-07-06T20:37:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.305+0000 7f16365ff700 1 -- start start 2023-07-06T20:37:55.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.305+0000 7f16365ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16300ffc30 0x7f1630197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:55.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.305+0000 7f16365ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1630100e20 0x7f16301980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:55.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.305+0000 7f16365ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16301017b0 0x7f163019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:55.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.305+0000 7f16365ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163019f6f0 con 0x7f16301017b0 2023-07-06T20:37:55.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.305+0000 7f16365ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163019f830 con 0x7f1630100e20 2023-07-06T20:37:55.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.305+0000 7f16365ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163019f9c0 con 0x7f16300ffc30 2023-07-06T20:37:55.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.305+0000 7f162f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1630100e20 0x7f16301980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:55.310 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f162f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1630100e20 0x7f16301980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:43420/0 (socket says 172.21.15.69:43420) 2023-07-06T20:37:55.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f162f7fe700 1 -- 172.21.15.69:0/1287087669 learned_addr learned my addr 172.21.15.69:0/1287087669 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:55.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f1634b9c700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16301017b0 0x7f163019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:55.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f162ffff700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16300ffc30 0x7f1630197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:55.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f162f7fe700 1 -- 172.21.15.69:0/1287087669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16300ffc30 msgr2=0x7f1630197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:55.311 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f162f7fe700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16300ffc30 0x7f1630197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f162f7fe700 1 -- 172.21.15.69:0/1287087669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16301017b0 msgr2=0x7f163019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:55.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f162f7fe700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16301017b0 0x7f163019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f162f7fe700 1 -- 172.21.15.69:0/1287087669 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1620009520 con 0x7f1630100e20 2023-07-06T20:37:55.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f162f7fe700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1630100e20 0x7f16301980b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f162400e9c0 tx=0x7f162400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:55.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f162d7fa700 1 -- 172.21.15.69:0/1287087669 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624004a30 con 0x7f1630100e20 2023-07-06T20:37:55.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f16365ff700 1 -- 172.21.15.69:0/1287087669 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163019fca0 con 0x7f1630100e20 2023-07-06T20:37:55.312 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.306+0000 7f162d7fa700 1 -- 172.21.15.69:0/1287087669 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f162400cc80 con 0x7f1630100e20 2023-07-06T20:37:55.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.307+0000 7f162d7fa700 1 -- 172.21.15.69:0/1287087669 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624005440 con 0x7f1630100e20 2023-07-06T20:37:55.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.307+0000 7f16365ff700 1 -- 172.21.15.69:0/1287087669 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16301a01b0 con 0x7f1630100e20 2023-07-06T20:37:55.313 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.307+0000 7f16365ff700 1 -- 172.21.15.69:0/1287087669 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16301922c0 con 0x7f1630100e20 2023-07-06T20:37:55.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.310+0000 7f162d7fa700 1 -- 172.21.15.69:0/1287087669 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1624004b90 con 0x7f1630100e20 2023-07-06T20:37:55.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.311+0000 7f162d7fa700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f16180a11a0 0x7f16180a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:55.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.311+0000 7f162d7fa700 1 -- 172.21.15.69:0/1287087669 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f1624014070 con 0x7f1630100e20 2023-07-06T20:37:55.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.311+0000 7f162ffff700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f16180a11a0 0x7f16180a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:55.314 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.312+0000 7f162ffff700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f16180a11a0 0x7f16180a3650 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f16200098c0 tx=0x7f1620019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:55.317 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.315+0000 7f162d7fa700 1 -- 172.21.15.69:0/1287087669 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f16240962e0 con 0x7f1630100e20 2023-07-06T20:37:55.517 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.515+0000 7f16365ff700 1 -- 172.21.15.69:0/1287087669 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f163002cae0 con 0x7f1630100e20 2023-07-06T20:37:55.519 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.517+0000 7f162d7fa700 1 -- 172.21.15.69:0/1287087669 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22612 (secure 0 0 0) 0x7f1624099900 con 0x7f1630100e20 2023-07-06T20:37:55.520 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-07-06T20:37:55.520 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":87,"fsid":"e3e8ec0c-1c38-11ee-9b2f-001a4aab830c","created":"2023-07-06T20:09:56.506880+0000","modified":"2023-07-06T20:18:59.638711+0000","last_up_change":"2023-07-06T20:16:41.340151+0000","last_in_change":"2023-07-06T20:16:23.533667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T20:13:14.022799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-06T20:17:26.523183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-06T20:17:27.543386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02b7a479-153d-48fb-bac6-606262f65f1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6803","nonce":2593880786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6805","nonce":2593880786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6809","nonce":2593880786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2593880786},{"type":"v1","addr":"172.21.15.69:6807","nonce":2593880786}]},"public_addr":"172.21.15.69:6803/2593880786","cluster_addr":"172.21.15.69:6805/2593880786","heartbeat_back_addr":"172.21.15.69:6809/2593880786","heartbeat_front_addr":"172.21.15.69:6807/2593880786","state":["exists","up"]},{"osd":1,"uuid":"85f52335-a4a7-42f5-8fbc-6414205fe241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6801","nonce":3741678125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6803","nonce":3741678125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6807","nonce":3741678125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3741678125},{"type":"v1","addr":"172.21.15.98:6805","nonce":3741678125}]},"public_addr":"172.21.15.98:6801/3741678125","cluster_addr":"172.21.15.98:6803/3741678125","heartbeat_back_addr":"172.21.15.98:6807/3741678125","heartbeat_front_addr":"172.21.15.98:6805/3741678125","state":["exists","up"]},{"osd":2,"uuid":"dea6f798-761a-460a-a87b-f7b2bc6e28a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6801","nonce":1470225946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6803","nonce":1470225946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6807","nonce":1470225946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1470225946},{"type":"v1","addr":"172.21.15.156:6805","nonce":1470225946}]},"public_addr":"172.21.15.156:6801/1470225946","cluster_addr":"172.21.15.156:6803/1470225946","heartbeat_back_addr":"172.21.15.156:6807/1470225946","heartbeat_front_addr":"172.21.15.156:6805/1470225946","state":["exists","up"]},{"osd":3,"uuid":"8cca1e09-50d7-4a90-bc9e-a62e092c5d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6811","nonce":3049408200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6813","nonce":3049408200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6817","nonce":3049408200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3049408200},{"type":"v1","addr":"172.21.15.69:6815","nonce":3049408200}]},"public_addr":"172.21.15.69:6811/3049408200","cluster_addr":"172.21.15.69:6813/3049408200","heartbeat_back_addr":"172.21.15.69:6817/3049408200","heartbeat_front_addr":"172.21.15.69:6815/3049408200","state":["exists","up"]},{"osd":4,"uuid":"66a50ae1-cca5-474e-a2f0-ace23bbc0ed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6809","nonce":3859428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6811","nonce":3859428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6815","nonce":3859428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3859428162},{"type":"v1","addr":"172.21.15.98:6813","nonce":3859428162}]},"public_addr":"172.21.15.98:6809/3859428162","cluster_addr":"172.21.15.98:6811/3859428162","heartbeat_back_addr":"172.21.15.98:6815/3859428162","heartbeat_front_addr":"172.21.15.98:6813/3859428162","state":["exists","up"]},{"osd":5,"uuid":"f62add7f-8f7d-46a4-b28b-e26110346d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6809","nonce":428236634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6811","nonce":428236634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6815","nonce":428236634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":428236634},{"type":"v1","addr":"172.21.15.156:6813","nonce":428236634}]},"public_addr":"172.21.15.156:6809/428236634","cluster_addr":"172.21.15.156:6811/428236634","heartbeat_back_addr":"172.21.15.156:6815/428236634","heartbeat_front_addr":"172.21.15.156:6813/428236634","state":["exists","up"]},{"osd":6,"uuid":"3fdcdab4-b7d2-4e02-908a-f7ce47c8c735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6819","nonce":2001926544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6821","nonce":2001926544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6825","nonce":2001926544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2001926544},{"type":"v1","addr":"172.21.15.69:6823","nonce":2001926544}]},"public_addr":"172.21.15.69:6819/2001926544","cluster_addr":"172.21.15.69:6821/2001926544","heartbeat_back_addr":"172.21.15.69:6825/2001926544","heartbeat_front_addr":"172.21.15.69:6823/2001926544","state":["exists","up"]},{"osd":7,"uuid":"ce45b288-d0c7-4ce3-9b99-0b48a168cd74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6817","nonce":238502140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6819","nonce":238502140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6823","nonce":238502140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":238502140},{"type":"v1","addr":"172.21.15.98:6821","nonce":238502140}]},"public_addr":"172.21.15.98:6817/238502140","cluster_addr":"172.21.15.98:6819/238502140","heartbeat_back_addr":"172.21.15.98:6823/238502140","heartbeat_front_addr":"172.21.15.98:6821/238502140","state":["exists","up"]},{"osd":8,"uuid":"8288c36a-d97e-4279-95ac-a215817b4ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6817","nonce":3522956813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6819","nonce":3522956813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6823","nonce":3522956813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3522956813},{"type":"v1","addr":"172.21.15.156:6821","nonce":3522956813}]},"public_addr":"172.21.15.156:6817/3522956813","cluster_addr":"172.21.15.156:6819/3522956813","heartbeat_back_addr":"172.21.15.156:6823/3522956813","heartbeat_front_addr":"172.21.15.156:6821/3522956813","state":["exists","up"]},{"osd":9,"uuid":"3fc7fc17-09f7-4d5c-b267-46605c603c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6827","nonce":2424610097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6829","nonce":2424610097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6833","nonce":2424610097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2424610097},{"type":"v1","addr":"172.21.15.69:6831","nonce":2424610097}]},"public_addr":"172.21.15.69:6827/2424610097","cluster_addr":"172.21.15.69:6829/2424610097","heartbeat_back_addr":"172.21.15.69:6833/2424610097","heartbeat_front_addr":"172.21.15.69:6831/2424610097","state":["exists","up"]},{"osd":10,"uuid":"e2f95a02-3a23-48bb-8b7e-3ecc462b3192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6825","nonce":2499526049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6827","nonce":2499526049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6831","nonce":2499526049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2499526049},{"type":"v1","addr":"172.21.15.98:6829","nonce":2499526049}]},"public_addr":"172.21.15.98:6825/2499526049","cluster_addr":"172.21.15.98:6827/2499526049","heartbeat_back_addr":"172.21.15.98:6831/2499526049","heartbeat_front_addr":"172.21.15.98:6829/2499526049","state":["exists","up"]},{"osd":11,"uuid":"5d0deb6c-92a0-4ae9-897e-51ddfc190bf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6825","nonce":266940539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6827","nonce":266940539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6831","nonce":266940539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":266940539},{"type":"v1","addr":"172.21.15.156:6829","nonce":266940539}]},"public_addr":"172.21.15.156:6825/266940539","cluster_addr":"172.21.15.156:6827/266940539","heartbeat_back_addr":"172.21.15.156:6831/266940539","heartbeat_front_addr":"172.21.15.156:6829/266940539","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:28.320823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:12:50.653743+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:11.782987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:13:37.881773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:00.228539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:21.078534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:14:46.603695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:09.360713+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:30.233382+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:15:55.909636+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:18.268726+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T20:16:39.524039+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1409752664":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6801/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:6800/9699389":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/20539643":"2023-07-07T20:10:57.432754+0000","172.21.15.69:0/3832776393":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/1479828612":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6800/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:6801/1829009377":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/4048964594":"2023-07-07T20:10:31.404980+0000","172.21.15.69:0/2854344202":"2023-07-07T20:10:57.432754+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T20:37:55.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.519+0000 7f16365ff700 1 -- 172.21.15.69:0/1287087669 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f16180a11a0 msgr2=0x7f16180a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:55.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.519+0000 7f16365ff700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f16180a11a0 0x7f16180a3650 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f16200098c0 tx=0x7f1620019040 comp rx=0 tx=0).stop 2023-07-06T20:37:55.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.519+0000 7f16365ff700 1 -- 172.21.15.69:0/1287087669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1630100e20 msgr2=0x7f16301980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:55.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.519+0000 7f16365ff700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1630100e20 0x7f16301980b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f162400e9c0 tx=0x7f162400ecd0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.519+0000 7f16365ff700 1 -- 172.21.15.69:0/1287087669 shutdown_connections 2023-07-06T20:37:55.522 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.519+0000 7f16365ff700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f16180a11a0 0x7f16180a3650 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.519+0000 7f16365ff700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16301017b0 0x7f163019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.519+0000 7f16365ff700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16300ffc30 0x7f1630197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.519+0000 7f16365ff700 1 --2- 172.21.15.69:0/1287087669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1630100e20 0x7f16301980b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.519+0000 7f16365ff700 1 -- 172.21.15.69:0/1287087669 >> 172.21.15.69:0/1287087669 conn(0x7f16300fb210 msgr2=0x7f16301066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:55.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.519+0000 7f16365ff700 1 -- 172.21.15.69:0/1287087669 shutdown_connections 2023-07-06T20:37:55.523 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.520+0000 7f16365ff700 1 -- 172.21.15.69:0/1287087669 wait complete. 2023-07-06T20:37:55.541 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T20:37:55.738 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.736+0000 7f46681ff700 1 -- 172.21.15.69:0/718223573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4660100e10 msgr2=0x7f4660101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.736+0000 7f46681ff700 1 --2- 172.21.15.69:0/718223573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4660100e10 0x7f4660101260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4650009990 tx=0x7f4650009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.736+0000 7f46681ff700 1 -- 172.21.15.69:0/718223573 shutdown_connections 2023-07-06T20:37:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.736+0000 7f46681ff700 1 --2- 172.21.15.69:0/718223573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46601017a0 0x7f4660108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.736+0000 7f46681ff700 1 --2- 172.21.15.69:0/718223573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46600ffc20 0x7f4660100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.736+0000 7f46681ff700 1 --2- 172.21.15.69:0/718223573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4660100e10 0x7f4660101260 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.736+0000 7f46681ff700 1 -- 172.21.15.69:0/718223573 >> 172.21.15.69:0/718223573 conn(0x7f46600fb260 msgr2=0x7f46600fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:55.739 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.736+0000 7f46681ff700 1 -- 172.21.15.69:0/718223573 shutdown_connections 2023-07-06T20:37:55.740 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.736+0000 7f46681ff700 1 -- 172.21.15.69:0/718223573 wait complete. 2023-07-06T20:37:55.741 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.737+0000 7f46681ff700 1 Processor -- start 2023-07-06T20:37:55.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.737+0000 7f46681ff700 1 -- start start 2023-07-06T20:37:55.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.737+0000 7f46681ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46600ffc20 0x7f4660197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:55.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.737+0000 7f46681ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4660100e10 0x7f4660198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:55.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.737+0000 7f46681ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46601017a0 0x7f466019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:55.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.737+0000 7f46681ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466019f770 con 0x7f46601017a0 2023-07-06T20:37:55.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.737+0000 7f46681ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466019f8b0 con 0x7f4660100e10 2023-07-06T20:37:55.742 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.737+0000 7f46681ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466019fa40 con 0x7f46600ffc20 2023-07-06T20:37:55.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.738+0000 7f466579a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4660100e10 0x7f4660198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:55.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.738+0000 7f466679c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46601017a0 0x7f466019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:55.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.738+0000 7f466579a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4660100e10 0x7f4660198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.69:43438/0 (socket says 172.21.15.69:43438) 2023-07-06T20:37:55.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.738+0000 7f466579a700 1 -- 172.21.15.69:0/1661126550 learned_addr learned my addr 172.21.15.69:0/1661126550 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:55.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.738+0000 7f4665f9b700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46600ffc20 0x7f4660197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:55.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.738+0000 7f466579a700 1 -- 172.21.15.69:0/1661126550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46600ffc20 msgr2=0x7f4660197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:55.743 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.738+0000 7f466579a700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46600ffc20 0x7f4660197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.738+0000 7f466579a700 1 -- 172.21.15.69:0/1661126550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46601017a0 msgr2=0x7f466019f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:55.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.738+0000 7f466579a700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46601017a0 0x7f466019f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:55.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.738+0000 7f466579a700 1 -- 172.21.15.69:0/1661126550 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46500095f0 con 0x7f4660100e10 2023-07-06T20:37:55.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.738+0000 7f466579a700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4660100e10 0x7f4660198130 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4650004760 tx=0x7f4650004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:55.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.739+0000 7f4656ffd700 1 -- 172.21.15.69:0/1661126550 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465001d070 con 0x7f4660100e10 2023-07-06T20:37:55.744 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.739+0000 7f46681ff700 1 -- 172.21.15.69:0/1661126550 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466019fcc0 con 0x7f4660100e10 2023-07-06T20:37:55.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.739+0000 7f4656ffd700 1 -- 172.21.15.69:0/1661126550 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4650004830 con 0x7f4660100e10 2023-07-06T20:37:55.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.739+0000 7f4656ffd700 1 -- 172.21.15.69:0/1661126550 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650005350 con 0x7f4660100e10 2023-07-06T20:37:55.745 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.739+0000 7f46681ff700 1 -- 172.21.15.69:0/1661126550 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46601a0080 con 0x7f4660100e10 2023-07-06T20:37:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.740+0000 7f46681ff700 1 -- 172.21.15.69:0/1661126550 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466010b4b0 con 0x7f4660100e10 2023-07-06T20:37:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.742+0000 7f4656ffd700 1 -- 172.21.15.69:0/1661126550 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4650005510 con 0x7f4660100e10 2023-07-06T20:37:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.743+0000 7f4656ffd700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f464c0a1150 0x7f464c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.743+0000 7f4656ffd700 1 -- 172.21.15.69:0/1661126550 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f46500cdb10 con 0x7f4660100e10 2023-07-06T20:37:55.746 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.743+0000 7f4665f9b700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f464c0a1150 0x7f464c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:55.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.744+0000 7f4665f9b700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f464c0a1150 0x7f464c0a3600 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4660100c70 tx=0x7f465c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:55.748 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.746+0000 7f4656ffd700 1 -- 172.21.15.69:0/1661126550 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4650097260 con 0x7f4660100e10 2023-07-06T20:37:55.992 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:55.989+0000 7f46681ff700 1 -- 172.21.15.69:0/1661126550 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f46601a0690 con 0x7f4660100e10 2023-07-06T20:37:56.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.001+0000 7f4656ffd700 1 -- 172.21.15.69:0/1661126550 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7f465009a880 con 0x7f4660100e10 2023-07-06T20:37:56.006 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.004+0000 7f46681ff700 1 -- 172.21.15.69:0/1661126550 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f464c0a1150 msgr2=0x7f464c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.004+0000 7f46681ff700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f464c0a1150 0x7f464c0a3600 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4660100c70 tx=0x7f465c009260 comp rx=0 tx=0).stop 2023-07-06T20:37:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.004+0000 7f46681ff700 1 -- 172.21.15.69:0/1661126550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4660100e10 msgr2=0x7f4660198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.004+0000 7f46681ff700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4660100e10 0x7f4660198130 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4650004760 tx=0x7f4650004790 comp rx=0 tx=0).stop 2023-07-06T20:37:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.004+0000 7f46681ff700 1 -- 172.21.15.69:0/1661126550 shutdown_connections 2023-07-06T20:37:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.004+0000 7f46681ff700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f464c0a1150 0x7f464c0a3600 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:56.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.004+0000 7f46681ff700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46601017a0 0x7f466019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:56.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.004+0000 7f46681ff700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46600ffc20 0x7f4660197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:56.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.004+0000 7f46681ff700 1 --2- 172.21.15.69:0/1661126550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4660100e10 0x7f4660198130 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:56.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.004+0000 7f46681ff700 1 -- 172.21.15.69:0/1661126550 >> 172.21.15.69:0/1661126550 conn(0x7f46600fb260 msgr2=0x7f46601066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:56.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.004+0000 7f46681ff700 1 -- 172.21.15.69:0/1661126550 shutdown_connections 2023-07-06T20:37:56.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.004+0000 7f46681ff700 1 -- 172.21.15.69:0/1661126550 wait complete. 2023-07-06T20:37:56.009 INFO:teuthology.orchestra.run.smithi069.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T20:37:56.026 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T20:37:56.042 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-i[155344]: ignoring --setuser ceph since I am not root 2023-07-06T20:37:56.042 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-i[155344]: ignoring --setgroup ceph since I am not root 2023-07-06T20:37:56.042 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:55 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/917726843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:37:56.042 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:55 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1287087669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:37:56.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:55 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/917726843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:37:56.083 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:55 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1287087669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:37:56.087 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:55 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/917726843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-06T20:37:56.087 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:55 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1287087669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-06T20:37:56.087 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:55 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-a[126375]: 2023-07-06T20:37:55.991+0000 7f20521d7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T20:37:56.227 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.224+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/2090186397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa240ffc80 msgr2=0x7ffa24100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:56.227 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.224+0000 7ffa2a1ff700 1 --2- 172.21.15.69:0/2090186397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa240ffc80 0x7ffa24100090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009990 tx=0x7ffa14009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:56.227 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.224+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/2090186397 shutdown_connections 2023-07-06T20:37:56.227 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.224+0000 7ffa2a1ff700 1 --2- 172.21.15.69:0/2090186397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa24101800 0x7ffa241080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.224+0000 7ffa2a1ff700 1 --2- 172.21.15.69:0/2090186397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa240ffc80 0x7ffa24100090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.224+0000 7ffa2a1ff700 1 --2- 172.21.15.69:0/2090186397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa24100e70 0x7ffa241012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.224+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/2090186397 >> 172.21.15.69:0/2090186397 conn(0x7ffa240fb2a0 msgr2=0x7ffa240fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.224+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/2090186397 shutdown_connections 2023-07-06T20:37:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.224+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/2090186397 wait complete. 2023-07-06T20:37:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.225+0000 7ffa2a1ff700 1 Processor -- start 2023-07-06T20:37:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa2a1ff700 1 -- start start 2023-07-06T20:37:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa2a1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa240ffc80 0x7ffa24197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa2a1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa24100e70 0x7ffa24198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa2a1ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa24101800 0x7ffa2419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa2a1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2419f690 con 0x7ffa240ffc80 2023-07-06T20:37:56.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa2a1ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2419f7d0 con 0x7ffa24101800 2023-07-06T20:37:56.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa2a1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2419f960 con 0x7ffa24100e70 2023-07-06T20:37:56.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa237fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa240ffc80 0x7ffa24197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:56.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa237fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa240ffc80 0x7ffa24197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58184/0 (socket says 172.21.15.69:58184) 2023-07-06T20:37:56.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa237fe700 1 -- 172.21.15.69:0/948436129 learned_addr learned my addr 172.21.15.69:0/948436129 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:56.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa23fff700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa24101800 0x7ffa2419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:56.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa22ffd700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa24100e70 0x7ffa24198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:56.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa237fe700 1 -- 172.21.15.69:0/948436129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa24100e70 msgr2=0x7ffa24198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:56.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.226+0000 7ffa237fe700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa24100e70 0x7ffa24198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:56.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.227+0000 7ffa237fe700 1 -- 172.21.15.69:0/948436129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa24101800 msgr2=0x7ffa2419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:56.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.227+0000 7ffa237fe700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa24101800 0x7ffa2419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:56.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.227+0000 7ffa237fe700 1 -- 172.21.15.69:0/948436129 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa140095f0 con 0x7ffa240ffc80 2023-07-06T20:37:56.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.227+0000 7ffa237fe700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa240ffc80 0x7ffa24197b10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400b680 tx=0x7ffa14005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:56.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.227+0000 7ffa20ff9700 1 -- 172.21.15.69:0/948436129 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1401d070 con 0x7ffa240ffc80 2023-07-06T20:37:56.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.227+0000 7ffa20ff9700 1 -- 172.21.15.69:0/948436129 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa14021410 con 0x7ffa240ffc80 2023-07-06T20:37:56.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.227+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/948436129 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2419fbe0 con 0x7ffa240ffc80 2023-07-06T20:37:56.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.227+0000 7ffa20ff9700 1 -- 172.21.15.69:0/948436129 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa14017650 con 0x7ffa240ffc80 2023-07-06T20:37:56.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.227+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/948436129 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa241a00b0 con 0x7ffa240ffc80 2023-07-06T20:37:56.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.228+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/948436129 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa24192290 con 0x7ffa240ffc80 2023-07-06T20:37:56.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.230+0000 7ffa20ff9700 1 -- 172.21.15.69:0/948436129 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ffa14021580 con 0x7ffa240ffc80 2023-07-06T20:37:56.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.231+0000 7ffa20ff9700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffa0c0a11d0 0x7ffa0c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:56.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.232+0000 7ffa20ff9700 1 -- 172.21.15.69:0/948436129 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8535+0+0 (secure 0 0 0) 0x7ffa140ce540 con 0x7ffa240ffc80 2023-07-06T20:37:56.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.232+0000 7ffa22ffd700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffa0c0a11d0 0x7ffa0c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:56.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.232+0000 7ffa22ffd700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffa0c0a11d0 0x7ffa0c0a3680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffa18005950 tx=0x7ffa1800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:56.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.233+0000 7ffa20ff9700 1 -- 172.21.15.69:0/948436129 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ffa14097b90 con 0x7ffa240ffc80 2023-07-06T20:37:56.357 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-c[155006]: ignoring --setuser ceph since I am not root 2023-07-06T20:37:56.357 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-c[155006]: ignoring --setgroup ceph since I am not root 2023-07-06T20:37:56.357 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-c[155006]: starting mds.c at 2023-07-06T20:37:56.358 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-f[154665]: ignoring --setuser ceph since I am not root 2023-07-06T20:37:56.358 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-f[154665]: ignoring --setgroup ceph since I am not root 2023-07-06T20:37:56.358 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-mds[154689]: -- 172.21.15.156:0/2003514774 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56347846b1e0 con 0x563478440800 2023-07-06T20:37:56.358 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-f[154665]: starting mds.f at 2023-07-06T20:37:56.358 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-mds[155369]: -- 172.21.15.156:0/664140570 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5607230cd1e0 con 0x5607230a2800 2023-07-06T20:37:56.359 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-i[155344]: starting mds.i at 2023-07-06T20:37:56.376 INFO:journalctl@ceph.mds.e.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-e[153716]: ignoring --setuser ceph since I am not root 2023-07-06T20:37:56.376 INFO:journalctl@ceph.mds.e.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-e[153716]: ignoring --setgroup ceph since I am not root 2023-07-06T20:37:56.376 INFO:journalctl@ceph.mds.e.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-mds[153740]: -- 172.21.15.98:0/3399043752 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558afafdb1e0 con 0x558afafb1000 2023-07-06T20:37:56.376 INFO:journalctl@ceph.mds.e.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-e[153716]: starting mds.e at 2023-07-06T20:37:56.377 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-h[154057]: ignoring --setuser ceph since I am not root 2023-07-06T20:37:56.377 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-h[154057]: ignoring --setgroup ceph since I am not root 2023-07-06T20:37:56.377 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-mds[154081]: -- 172.21.15.98:0/3039840551 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5635f82bb1e0 con 0x5635f8290c00 2023-07-06T20:37:56.377 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-h[154057]: starting mds.h at 2023-07-06T20:37:56.377 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-b[154398]: ignoring --setuser ceph since I am not root 2023-07-06T20:37:56.377 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-b[154398]: ignoring --setgroup ceph since I am not root 2023-07-06T20:37:56.378 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-b[154398]: starting mds.b at 2023-07-06T20:37:56.481 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:56.478+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/948436129 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffa2402ca70 con 0x7ffa240ffc80 2023-07-06T20:37:57.004 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.001+0000 7ffa20ff9700 1 -- 172.21.15.69:0/948436129 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffa1409b1b0 con 0x7ffa240ffc80 2023-07-06T20:37:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.006+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/948436129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffa0c0a11d0 msgr2=0x7ffa0c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.006+0000 7ffa2a1ff700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffa0c0a11d0 0x7ffa0c0a3680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffa18005950 tx=0x7ffa1800b410 comp rx=0 tx=0).stop 2023-07-06T20:37:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.006+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/948436129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa240ffc80 msgr2=0x7ffa24197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.006+0000 7ffa2a1ff700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa240ffc80 0x7ffa24197b10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400b680 tx=0x7ffa14005740 comp rx=0 tx=0).stop 2023-07-06T20:37:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.006+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/948436129 shutdown_connections 2023-07-06T20:37:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.006+0000 7ffa2a1ff700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7ffa0c0a11d0 0x7ffa0c0a3680 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.006+0000 7ffa2a1ff700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa24101800 0x7ffa2419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.006+0000 7ffa2a1ff700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa240ffc80 0x7ffa24197b10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.006+0000 7ffa2a1ff700 1 --2- 172.21.15.69:0/948436129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa24100e70 0x7ffa24198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:57.010 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.006+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/948436129 >> 172.21.15.69:0/948436129 conn(0x7ffa240fb2a0 msgr2=0x7ffa24106770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:57.010 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.006+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/948436129 shutdown_connections 2023-07-06T20:37:57.010 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.006+0000 7ffa2a1ff700 1 -- 172.21.15.69:0/948436129 wait complete. 2023-07-06T20:37:57.029 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T20:37:57.090 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:56 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1661126550' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-06T20:37:57.090 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:56 smithi069 ceph-mon[126399]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-06T20:37:57.090 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:56 smithi069 ceph-mon[126399]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-06T20:37:57.090 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:56 smithi069 ceph-mon[126399]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T20:37:57.090 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:56 smithi069 ceph-mon[126399]: osdmap e88: 12 total, 12 up, 12 in 2023-07-06T20:37:57.090 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:56 smithi069 ceph-mon[126399]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-06T20:37:57.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:56 smithi069 ceph-mon[126399]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-06T20:37:57.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:56 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/948436129' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-06T20:37:57.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:56 smithi069 ceph-mon[126399]: pgmap v887: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-07-06T20:37:57.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1661126550' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-06T20:37:57.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-mon[129280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-06T20:37:57.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-mon[129280]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-06T20:37:57.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-mon[129280]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T20:37:57.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-mon[129280]: osdmap e88: 12 total, 12 up, 12 in 2023-07-06T20:37:57.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-mon[129280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-06T20:37:57.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-mon[129280]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-06T20:37:57.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/948436129' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-06T20:37:57.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:56 smithi156 ceph-mon[129280]: pgmap v887: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-07-06T20:37:57.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1661126550' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-06T20:37:57.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-mon[128968]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-06T20:37:57.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-mon[128968]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-06T20:37:57.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-mon[128968]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T20:37:57.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-mon[128968]: osdmap e88: 12 total, 12 up, 12 in 2023-07-06T20:37:57.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-mon[128968]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-06T20:37:57.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-mon[128968]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-06T20:37:57.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/948436129' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-06T20:37:57.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:56 smithi098 ceph-mon[128968]: pgmap v887: 97 pgs: 97 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-07-06T20:37:57.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.229+0000 7fba643ff700 1 -- 172.21.15.69:0/1742711086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba5c0ffc20 msgr2=0x7fba5c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:57.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.229+0000 7fba643ff700 1 --2- 172.21.15.69:0/1742711086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba5c0ffc20 0x7fba5c100030 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fba580098f0 tx=0x7fba58009c00 comp rx=0 tx=0).stop 2023-07-06T20:37:57.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.229+0000 7fba643ff700 1 -- 172.21.15.69:0/1742711086 shutdown_connections 2023-07-06T20:37:57.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.229+0000 7fba643ff700 1 --2- 172.21.15.69:0/1742711086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5c1017a0 0x7fba5c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:57.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.229+0000 7fba643ff700 1 --2- 172.21.15.69:0/1742711086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba5c0ffc20 0x7fba5c100030 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:57.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.229+0000 7fba643ff700 1 --2- 172.21.15.69:0/1742711086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5c100e10 0x7fba5c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:57.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.229+0000 7fba643ff700 1 -- 172.21.15.69:0/1742711086 >> 172.21.15.69:0/1742711086 conn(0x7fba5c0fb260 msgr2=0x7fba5c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:57.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.229+0000 7fba643ff700 1 -- 172.21.15.69:0/1742711086 shutdown_connections 2023-07-06T20:37:57.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.229+0000 7fba643ff700 1 -- 172.21.15.69:0/1742711086 wait complete. 2023-07-06T20:37:57.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.230+0000 7fba643ff700 1 Processor -- start 2023-07-06T20:37:57.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.230+0000 7fba643ff700 1 -- start start 2023-07-06T20:37:57.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.230+0000 7fba643ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5c0ffc20 0x7fba5c0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:57.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.230+0000 7fba643ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5c100e10 0x7fba5c072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:57.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.230+0000 7fba643ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba5c1017a0 0x7fba5c072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:57.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.230+0000 7fba643ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c073710 con 0x7fba5c1017a0 2023-07-06T20:37:57.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.230+0000 7fba643ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c073880 con 0x7fba5c100e10 2023-07-06T20:37:57.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.230+0000 7fba643ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c19fa40 con 0x7fba5c0ffc20 2023-07-06T20:37:57.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.231+0000 7fba6299c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba5c1017a0 0x7fba5c072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:57.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.231+0000 7fba6299c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba5c1017a0 0x7fba5c072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58198/0 (socket says 172.21.15.69:58198) 2023-07-06T20:37:57.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.231+0000 7fba6299c700 1 -- 172.21.15.69:0/4024987676 learned_addr learned my addr 172.21.15.69:0/4024987676 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:57.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.231+0000 7fba6219b700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5c0ffc20 0x7fba5c0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:57.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.231+0000 7fba6199a700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5c100e10 0x7fba5c072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:57.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.231+0000 7fba6299c700 1 -- 172.21.15.69:0/4024987676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5c0ffc20 msgr2=0x7fba5c0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:57.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.231+0000 7fba6299c700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5c0ffc20 0x7fba5c0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:57.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.231+0000 7fba6299c700 1 -- 172.21.15.69:0/4024987676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5c100e10 msgr2=0x7fba5c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:57.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.231+0000 7fba6299c700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5c100e10 0x7fba5c072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:57.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.231+0000 7fba6299c700 1 -- 172.21.15.69:0/4024987676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba58009520 con 0x7fba5c1017a0 2023-07-06T20:37:57.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.231+0000 7fba6299c700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba5c1017a0 0x7fba5c072f70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fba5400ef30 tx=0x7fba5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:57.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.231+0000 7fba537fe700 1 -- 172.21.15.69:0/4024987676 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54018070 con 0x7fba5c1017a0 2023-07-06T20:37:57.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.232+0000 7fba643ff700 1 -- 172.21.15.69:0/4024987676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5c19fcc0 con 0x7fba5c1017a0 2023-07-06T20:37:57.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.232+0000 7fba537fe700 1 -- 172.21.15.69:0/4024987676 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba5400f040 con 0x7fba5c1017a0 2023-07-06T20:37:57.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.232+0000 7fba537fe700 1 -- 172.21.15.69:0/4024987676 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5400ccf0 con 0x7fba5c1017a0 2023-07-06T20:37:57.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.232+0000 7fba643ff700 1 -- 172.21.15.69:0/4024987676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5c1a01d0 con 0x7fba5c1017a0 2023-07-06T20:37:57.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.233+0000 7fba643ff700 1 -- 172.21.15.69:0/4024987676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5c107210 con 0x7fba5c1017a0 2023-07-06T20:37:57.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.235+0000 7fba537fe700 1 -- 172.21.15.69:0/4024987676 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fba54003a00 con 0x7fba5c1017a0 2023-07-06T20:37:57.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.236+0000 7fba537fe700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fba480a1150 0x7fba480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:57.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.236+0000 7fba537fe700 1 -- 172.21.15.69:0/4024987676 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8535+0+0 (secure 0 0 0) 0x7fba540cc580 con 0x7fba5c1017a0 2023-07-06T20:37:57.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.236+0000 7fba6219b700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fba480a1150 0x7fba480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:57.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.237+0000 7fba6219b700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fba480a1150 0x7fba480a3600 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fba58009fd0 tx=0x7fba58019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:57.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.240+0000 7fba537fe700 1 -- 172.21.15.69:0/4024987676 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fba54095bd0 con 0x7fba5c1017a0 2023-07-06T20:37:57.438 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:57.436+0000 7fba643ff700 1 -- 172.21.15.69:0/4024987676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fba5c061250 con 0x7fba5c1017a0 2023-07-06T20:37:58.007 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.004+0000 7fba537fe700 1 -- 172.21.15.69:0/4024987676 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) v1 ==== 181+0+0 (secure 0 0 0) 0x7fba540991f0 con 0x7fba5c1017a0 2023-07-06T20:37:58.010 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.007+0000 7fba643ff700 1 -- 172.21.15.69:0/4024987676 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fba480a1150 msgr2=0x7fba480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:58.010 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.007+0000 7fba643ff700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fba480a1150 0x7fba480a3600 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fba58009fd0 tx=0x7fba58019040 comp rx=0 tx=0).stop 2023-07-06T20:37:58.010 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.007+0000 7fba643ff700 1 -- 172.21.15.69:0/4024987676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba5c1017a0 msgr2=0x7fba5c072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:58.010 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.007+0000 7fba643ff700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba5c1017a0 0x7fba5c072f70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fba5400ef30 tx=0x7fba5400c3a0 comp rx=0 tx=0).stop 2023-07-06T20:37:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.008+0000 7fba643ff700 1 -- 172.21.15.69:0/4024987676 shutdown_connections 2023-07-06T20:37:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.008+0000 7fba643ff700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7fba480a1150 0x7fba480a3600 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.008+0000 7fba643ff700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba5c1017a0 0x7fba5c072f70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.008+0000 7fba643ff700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba5c0ffc20 0x7fba5c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.008+0000 7fba643ff700 1 --2- 172.21.15.69:0/4024987676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba5c100e10 0x7fba5c072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.008+0000 7fba643ff700 1 -- 172.21.15.69:0/4024987676 >> 172.21.15.69:0/4024987676 conn(0x7fba5c0fb260 msgr2=0x7fba5c104350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:58.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.008+0000 7fba643ff700 1 -- 172.21.15.69:0/4024987676 shutdown_connections 2023-07-06T20:37:58.012 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.008+0000 7fba643ff700 1 -- 172.21.15.69:0/4024987676 wait complete. 2023-07-06T20:37:58.012 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' removed 2023-07-06T20:37:58.030 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T20:37:58.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-06T20:37:58.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-06T20:37:58.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: Cluster is now healthy 2023-07-06T20:37:58.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.156:6837/2284843613,v1:172.21.15.156:6839/2284843613] up:boot 2023-07-06T20:37:58.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.156:6834/1860649369,v1:172.21.15.156:6835/1860649369] up:boot 2023-07-06T20:37:58.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.156:6836/1811197667,v1:172.21.15.156:6838/1811197667] up:boot 2023-07-06T20:37:58.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.98:6837/1072061004,v1:172.21.15.98:6839/1072061004] up:boot 2023-07-06T20:37:58.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.98:6834/3286938406,v1:172.21.15.98:6836/3286938406] up:boot 2023-07-06T20:37:58.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: mds.? [v2:172.21.15.98:6835/4147767148,v1:172.21.15.98:6838/4147767148] up:boot 2023-07-06T20:37:58.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/948436129' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-06T20:37:58.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: fsmap 12 up:standby 2023-07-06T20:37:58.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-06T20:37:58.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-06T20:37:58.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-06T20:37:58.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-06T20:37:58.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-06T20:37:58.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-06T20:37:58.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:57 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/4024987676' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-06T20:37:58.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-06T20:37:58.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-06T20:37:58.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: Cluster is now healthy 2023-07-06T20:37:58.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.156:6837/2284843613,v1:172.21.15.156:6839/2284843613] up:boot 2023-07-06T20:37:58.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.156:6834/1860649369,v1:172.21.15.156:6835/1860649369] up:boot 2023-07-06T20:37:58.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.156:6836/1811197667,v1:172.21.15.156:6838/1811197667] up:boot 2023-07-06T20:37:58.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.98:6837/1072061004,v1:172.21.15.98:6839/1072061004] up:boot 2023-07-06T20:37:58.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.98:6834/3286938406,v1:172.21.15.98:6836/3286938406] up:boot 2023-07-06T20:37:58.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: mds.? [v2:172.21.15.98:6835/4147767148,v1:172.21.15.98:6838/4147767148] up:boot 2023-07-06T20:37:58.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/948436129' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-06T20:37:58.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: fsmap 12 up:standby 2023-07-06T20:37:58.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-06T20:37:58.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-06T20:37:58.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-06T20:37:58.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-06T20:37:58.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-06T20:37:58.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-06T20:37:58.109 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:57 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/4024987676' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-06T20:37:58.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-06T20:37:58.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-06T20:37:58.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: Cluster is now healthy 2023-07-06T20:37:58.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.156:6837/2284843613,v1:172.21.15.156:6839/2284843613] up:boot 2023-07-06T20:37:58.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.156:6834/1860649369,v1:172.21.15.156:6835/1860649369] up:boot 2023-07-06T20:37:58.126 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.156:6836/1811197667,v1:172.21.15.156:6838/1811197667] up:boot 2023-07-06T20:37:58.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.98:6837/1072061004,v1:172.21.15.98:6839/1072061004] up:boot 2023-07-06T20:37:58.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.98:6834/3286938406,v1:172.21.15.98:6836/3286938406] up:boot 2023-07-06T20:37:58.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: mds.? [v2:172.21.15.98:6835/4147767148,v1:172.21.15.98:6838/4147767148] up:boot 2023-07-06T20:37:58.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/948436129' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-06T20:37:58.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: fsmap 12 up:standby 2023-07-06T20:37:58.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-06T20:37:58.127 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-06T20:37:58.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-06T20:37:58.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-06T20:37:58.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-06T20:37:58.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: from='mgr.14152 172.21.15.69:0/590473337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-06T20:37:58.128 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:57 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/4024987676' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-06T20:37:58.231 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.229+0000 7f38bafff700 1 -- 172.21.15.69:0/883084439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b40ffc20 msgr2=0x7f38b4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.229+0000 7f38bafff700 1 --2- 172.21.15.69:0/883084439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b40ffc20 0x7f38b4100030 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009990 tx=0x7f38a8009ca0 comp rx=0 tx=0).stop 2023-07-06T20:37:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.229+0000 7f38bafff700 1 -- 172.21.15.69:0/883084439 shutdown_connections 2023-07-06T20:37:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.229+0000 7f38bafff700 1 --2- 172.21.15.69:0/883084439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38b41017a0 0x7f38b4108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.229+0000 7f38bafff700 1 --2- 172.21.15.69:0/883084439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b40ffc20 0x7f38b4100030 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.229+0000 7f38bafff700 1 --2- 172.21.15.69:0/883084439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b4100e10 0x7f38b4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.229+0000 7f38bafff700 1 -- 172.21.15.69:0/883084439 >> 172.21.15.69:0/883084439 conn(0x7f38b40fb260 msgr2=0x7f38b40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:58.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.229+0000 7f38bafff700 1 -- 172.21.15.69:0/883084439 shutdown_connections 2023-07-06T20:37:58.233 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.229+0000 7f38bafff700 1 -- 172.21.15.69:0/883084439 wait complete. 2023-07-06T20:37:58.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.230+0000 7f38bafff700 1 Processor -- start 2023-07-06T20:37:58.234 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.230+0000 7f38bafff700 1 -- start start 2023-07-06T20:37:58.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.230+0000 7f38bafff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38b40ffc20 0x7f38b40743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:58.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.230+0000 7f38bafff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b4100e10 0x7f38b4072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:58.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38bafff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b41017a0 0x7f38b4072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:58.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38bafff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b4073710 con 0x7f38b41017a0 2023-07-06T20:37:58.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38bafff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b4073880 con 0x7f38b4100e10 2023-07-06T20:37:58.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38bafff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b419f9e0 con 0x7f38b40ffc20 2023-07-06T20:37:58.235 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38b959c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b41017a0 0x7f38b4072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:58.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38b959c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b41017a0 0x7f38b4072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58220/0 (socket says 172.21.15.69:58220) 2023-07-06T20:37:58.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38b959c700 1 -- 172.21.15.69:0/1073660109 learned_addr learned my addr 172.21.15.69:0/1073660109 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-07-06T20:37:58.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38b3fff700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b4100e10 0x7f38b4072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:58.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38b8d9b700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38b40ffc20 0x7f38b40743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:58.236 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38b959c700 1 -- 172.21.15.69:0/1073660109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38b40ffc20 msgr2=0x7f38b40743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:58.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38b959c700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38b40ffc20 0x7f38b40743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:58.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38b959c700 1 -- 172.21.15.69:0/1073660109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b4100e10 msgr2=0x7f38b4072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:58.237 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38b959c700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b4100e10 0x7f38b4072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:58.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38b959c700 1 -- 172.21.15.69:0/1073660109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0009520 con 0x7f38b41017a0 2023-07-06T20:37:58.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.231+0000 7f38b959c700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b41017a0 0x7f38b4072f70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00ef30 tx=0x7f38ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:58.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.232+0000 7f38b1ffb700 1 -- 172.21.15.69:0/1073660109 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac018070 con 0x7f38b41017a0 2023-07-06T20:37:58.238 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.232+0000 7f38b1ffb700 1 -- 172.21.15.69:0/1073660109 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38ac00f040 con 0x7f38b41017a0 2023-07-06T20:37:58.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.232+0000 7f38bafff700 1 -- 172.21.15.69:0/1073660109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a80095f0 con 0x7f38b41017a0 2023-07-06T20:37:58.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.232+0000 7f38b1ffb700 1 -- 172.21.15.69:0/1073660109 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00cd00 con 0x7f38b41017a0 2023-07-06T20:37:58.239 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.232+0000 7f38bafff700 1 -- 172.21.15.69:0/1073660109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b41a0030 con 0x7f38b41017a0 2023-07-06T20:37:58.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.233+0000 7f38bafff700 1 -- 172.21.15.69:0/1073660109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b41071c0 con 0x7f38b41017a0 2023-07-06T20:37:58.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.235+0000 7f38b1ffb700 1 -- 172.21.15.69:0/1073660109 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f38ac003a00 con 0x7f38b41017a0 2023-07-06T20:37:58.241 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.236+0000 7f38b1ffb700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f38a40a11d0 0x7f38a40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T20:37:58.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.236+0000 7f38b1ffb700 1 -- 172.21.15.69:0/1073660109 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 7973+0+0 (secure 0 0 0) 0x7f38ac0cc2c0 con 0x7f38b41017a0 2023-07-06T20:37:58.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.236+0000 7f38b8d9b700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f38a40a11d0 0x7f38a40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T20:37:58.242 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.237+0000 7f38b8d9b700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f38a40a11d0 0x7f38a40a3680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f38a800b2e0 tx=0x7f38a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T20:37:58.243 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.240+0000 7f38b1ffb700 1 -- 172.21.15.69:0/1073660109 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f38ac095b50 con 0x7f38b41017a0 2023-07-06T20:37:58.440 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:58.438+0000 7f38bafff700 1 -- 172.21.15.69:0/1073660109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38b402c8e0 con 0x7f38b41017a0 2023-07-06T20:37:59.014 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.011+0000 7f38b1ffb700 1 -- 172.21.15.69:0/1073660109 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7f38ac099170 con 0x7f38b41017a0 2023-07-06T20:37:59.024 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.021+0000 7f38bafff700 1 -- 172.21.15.69:0/1073660109 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f38a40a11d0 msgr2=0x7f38a40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:59.024 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.021+0000 7f38bafff700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f38a40a11d0 0x7f38a40a3680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f38a800b2e0 tx=0x7f38a8005fb0 comp rx=0 tx=0).stop 2023-07-06T20:37:59.024 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.021+0000 7f38bafff700 1 -- 172.21.15.69:0/1073660109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b41017a0 msgr2=0x7f38b4072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T20:37:59.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.021+0000 7f38bafff700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b41017a0 0x7f38b4072f70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00ef30 tx=0x7f38ac00c3a0 comp rx=0 tx=0).stop 2023-07-06T20:37:59.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.022+0000 7f38bafff700 1 -- 172.21.15.69:0/1073660109 shutdown_connections 2023-07-06T20:37:59.025 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.022+0000 7f38bafff700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.69:6800/4215258217,v1:172.21.15.69:6801/4215258217] conn(0x7f38a40a11d0 0x7f38a40a3680 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:59.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.022+0000 7f38bafff700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b41017a0 0x7f38b4072f70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:59.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.022+0000 7f38bafff700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38b40ffc20 0x7f38b40743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:59.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.022+0000 7f38bafff700 1 --2- 172.21.15.69:0/1073660109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38b4100e10 0x7f38b4072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T20:37:59.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.022+0000 7f38bafff700 1 -- 172.21.15.69:0/1073660109 >> 172.21.15.69:0/1073660109 conn(0x7f38b40fb260 msgr2=0x7f38b4104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T20:37:59.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.023+0000 7f38bafff700 1 -- 172.21.15.69:0/1073660109 shutdown_connections 2023-07-06T20:37:59.026 INFO:teuthology.orchestra.run.smithi069.stderr:2023-07-06T20:37:59.023+0000 7f38bafff700 1 -- 172.21.15.69:0/1073660109 wait complete. 2023-07-06T20:37:59.027 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' removed 2023-07-06T20:37:59.055 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-06T20:37:59.168 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-06T20:37:59.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:59 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/4024987676' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-06T20:37:59.266 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:59 smithi098 ceph-mon[128968]: osdmap e89: 12 total, 12 up, 12 in 2023-07-06T20:37:59.266 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:59 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1073660109' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-06T20:37:59.266 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:37:59 smithi098 ceph-mon[128968]: pgmap v889: 65 pgs: 65 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:37:59.300 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-06T20:37:59.359 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:59 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/4024987676' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-06T20:37:59.360 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:59 smithi156 ceph-mon[129280]: osdmap e89: 12 total, 12 up, 12 in 2023-07-06T20:37:59.360 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:59 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1073660109' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-06T20:37:59.360 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:37:59 smithi156 ceph-mon[129280]: pgmap v889: 65 pgs: 65 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:37:59.400 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-06T20:37:59.400 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-06T20:37:59.481 DEBUG:teuthology.orchestra.run.smithi098:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-06T20:37:59.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:59 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/4024987676' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-06T20:37:59.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:59 smithi069 ceph-mon[126399]: osdmap e89: 12 total, 12 up, 12 in 2023-07-06T20:37:59.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:59 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1073660109' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-06T20:37:59.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:59 smithi069 ceph-mon[126399]: pgmap v889: 65 pgs: 65 active+clean; 8.5 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-06T20:37:59.551 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-06T20:37:59.613 INFO:tasks.cephadm:Stopping all daemons... 2023-07-06T20:37:59.613 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-06T20:37:59.613 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.a 2023-07-06T20:38:00.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:37:59 smithi069 systemd[1]: Stopping Ceph mon.a for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:38:00.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:38:00 smithi156 ceph-mon[129280]: from='client.? 172.21.15.69:0/1073660109' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-06T20:38:00.357 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:38:00 smithi156 ceph-mon[129280]: osdmap e90: 12 total, 12 up, 12 in 2023-07-06T20:38:00.375 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:38:00 smithi098 ceph-mon[128968]: from='client.? 172.21.15.69:0/1073660109' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-06T20:38:00.376 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:38:00 smithi098 ceph-mon[128968]: osdmap e90: 12 total, 12 up, 12 in 2023-07-06T20:38:00.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:38:00 smithi069 ceph-mon[126399]: from='client.? 172.21.15.69:0/1073660109' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-06T20:38:00.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:38:00 smithi069 ceph-mon[126399]: osdmap e90: 12 total, 12 up, 12 in 2023-07-06T20:38:00.509 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:38:00 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-a[126375]: 2023-07-06T20:38:00.233+0000 7f20579e2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:38:00.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:38:00 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-a[126375]: 2023-07-06T20:38:00.233+0000 7f20579e2700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-06T20:38:01.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Jul 06 20:38:00 smithi069 bash[183576]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-a 2023-07-06T20:38:01.249 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.a.service' 2023-07-06T20:38:01.357 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:38:01.358 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-06T20:38:01.358 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-06T20:38:01.358 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.b 2023-07-06T20:38:01.860 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:38:01 smithi098 systemd[1]: Stopping Ceph mon.b for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:38:01.861 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:38:01 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-b[128944]: 2023-07-06T20:38:01.851+0000 7fa2c8aba700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:38:01.861 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:38:01 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-b[128944]: 2023-07-06T20:38:01.852+0000 7fa2c8aba700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-06T20:38:02.541 INFO:journalctl@ceph.mon.b.smithi098.stdout:Jul 06 20:38:02 smithi098 bash[160037]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-b 2023-07-06T20:38:02.548 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.b.service' 2023-07-06T20:38:02.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:38:02.621 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-06T20:38:02.621 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-06T20:38:02.621 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.c 2023-07-06T20:38:03.003 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:38:02 smithi156 systemd[1]: Stopping Ceph mon.c for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:38:03.345 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:38:03 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-c[129256]: 2023-07-06T20:38:03.000+0000 7f69aea07700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:38:03.345 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:38:03 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-c[129256]: 2023-07-06T20:38:03.000+0000 7f69aea07700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-06T20:38:03.345 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 06 20:38:03 smithi156 bash[162478]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mon-c 2023-07-06T20:38:03.646 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mon.c.service' 2023-07-06T20:38:03.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:38:03.716 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-06T20:38:03.717 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-06T20:38:03.717 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.x 2023-07-06T20:38:04.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:38:03 smithi069 systemd[1]: Stopping Ceph mgr.x for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:38:04.567 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:38:04 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:38:04.282+0000 7f3d6dcae700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:38:04.567 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:38:04 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x[126649]: 2023-07-06T20:38:04.282+0000 7f3d6dcae700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-07-06T20:38:05.009 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Jul 06 20:38:04 smithi069 bash[183719]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-x 2023-07-06T20:38:05.058 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.x.service' 2023-07-06T20:38:05.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:38:05.146 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-06T20:38:05.147 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-06T20:38:05.147 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.y 2023-07-06T20:38:05.625 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:38:05 smithi098 systemd[1]: Stopping Ceph mgr.y for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:38:08.239 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:38:07 smithi098 bash[160187]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-y 2023-07-06T20:38:08.239 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Jul 06 20:38:08 smithi098 systemd[1]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-06T20:38:08.246 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.y.service' 2023-07-06T20:38:08.283 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:38:08.284 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-06T20:38:08.284 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-06T20:38:08.285 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.z 2023-07-06T20:38:08.607 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:38:08 smithi156 systemd[1]: Stopping Ceph mgr.z for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:38:08.992 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:38:08 smithi156 bash[162631]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mgr-z 2023-07-06T20:38:09.177 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 06 20:38:09 smithi156 systemd[1]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-06T20:38:09.178 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mgr.z.service' 2023-07-06T20:38:09.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:38:09.216 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-06T20:38:09.217 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-06T20:38:09.217 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.0 2023-07-06T20:38:09.759 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:09 smithi069 systemd[1]: Stopping Ceph osd.0 for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:38:10.259 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:09 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:09.846+0000 7f87182a5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:38:10.260 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:09 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:09.846+0000 7f87182a5700 -1 osd.0 90 *** Got signal Terminated *** 2023-07-06T20:38:10.260 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:09 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:09.846+0000 7f87182a5700 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T20:38:34.510 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/osd/OSD.cc: In function 'int OSD::shutdown()' thread 7f87182a5700 time 2023-07-06T20:38:34.207192+0000 2023-07-06T20:38:34.511 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/osd/OSD.cc: 4501: FAILED ceph_assert(end_time - start_time_func < cct->_conf->osd_fast_shutdown_timeout) 2023-07-06T20:38:34.511 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T20:38:34.511 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x5567f3bdb695] 2023-07-06T20:38:34.511 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2: /usr/bin/ceph-osd(+0x63485b) [0x5567f3bdb85b] 2023-07-06T20:38:34.511 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 3: (OSD::shutdown()+0xdb2) [0x5567f3d087f2] 2023-07-06T20:38:34.511 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 4: (SignalHandler::entry()+0x657) [0x5567f4360e67] 2023-07-06T20:38:34.512 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 5: /lib64/libpthread.so.0(+0x81ca) [0x7f871fee01ca] 2023-07-06T20:38:34.512 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 6: clone() 2023-07-06T20:38:34.512 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.207+0000 7f87182a5700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/osd/OSD.cc: In function 'int OSD::shutdown()' thread 7f87182a5700 time 2023-07-06T20:38:34.207192+0000 2023-07-06T20:38:34.512 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/osd/OSD.cc: 4501: FAILED ceph_assert(end_time - start_time_func < cct->_conf->osd_fast_shutdown_timeout) 2023-07-06T20:38:34.512 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.512 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T20:38:34.512 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x5567f3bdb695] 2023-07-06T20:38:34.513 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2: /usr/bin/ceph-osd(+0x63485b) [0x5567f3bdb85b] 2023-07-06T20:38:34.513 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 3: (OSD::shutdown()+0xdb2) [0x5567f3d087f2] 2023-07-06T20:38:34.513 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 4: (SignalHandler::entry()+0x657) [0x5567f4360e67] 2023-07-06T20:38:34.513 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 5: /lib64/libpthread.so.0(+0x81ca) [0x7f871fee01ca] 2023-07-06T20:38:34.513 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 6: clone() 2023-07-06T20:38:34.513 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.513 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: *** Caught signal (Aborted) ** 2023-07-06T20:38:34.513 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: in thread 7f87182a5700 thread_name:signal_handler 2023-07-06T20:38:34.514 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T20:38:34.514 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f871feeacf0] 2023-07-06T20:38:34.514 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2: gsignal() 2023-07-06T20:38:34.514 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 3: abort() 2023-07-06T20:38:34.514 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x5567f3bdb6ef] 2023-07-06T20:38:34.514 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 5: /usr/bin/ceph-osd(+0x63485b) [0x5567f3bdb85b] 2023-07-06T20:38:34.514 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 6: (OSD::shutdown()+0xdb2) [0x5567f3d087f2] 2023-07-06T20:38:34.515 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 7: (SignalHandler::entry()+0x657) [0x5567f4360e67] 2023-07-06T20:38:34.515 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7f871fee01ca] 2023-07-06T20:38:34.515 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 9: clone() 2023-07-06T20:38:34.515 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.210+0000 7f87182a5700 -1 *** Caught signal (Aborted) ** 2023-07-06T20:38:34.515 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: in thread 7f87182a5700 thread_name:signal_handler 2023-07-06T20:38:34.515 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.515 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T20:38:34.515 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f871feeacf0] 2023-07-06T20:38:34.516 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2: gsignal() 2023-07-06T20:38:34.516 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 3: abort() 2023-07-06T20:38:34.516 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x5567f3bdb6ef] 2023-07-06T20:38:34.516 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 5: /usr/bin/ceph-osd(+0x63485b) [0x5567f3bdb85b] 2023-07-06T20:38:34.516 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 6: (OSD::shutdown()+0xdb2) [0x5567f3d087f2] 2023-07-06T20:38:34.516 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 7: (SignalHandler::entry()+0x657) [0x5567f4360e67] 2023-07-06T20:38:34.517 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7f871fee01ca] 2023-07-06T20:38:34.517 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 9: clone() 2023-07-06T20:38:34.517 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-07-06T20:38:34.517 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.517 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: -1> 2023-07-06T20:38:34.207+0000 7f87182a5700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/osd/OSD.cc: In function 'int OSD::shutdown()' thread 7f87182a5700 time 2023-07-06T20:38:34.207192+0000 2023-07-06T20:38:34.517 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/osd/OSD.cc: 4501: FAILED ceph_assert(end_time - start_time_func < cct->_conf->osd_fast_shutdown_timeout) 2023-07-06T20:38:34.517 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.517 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T20:38:34.518 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x5567f3bdb695] 2023-07-06T20:38:34.518 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2: /usr/bin/ceph-osd(+0x63485b) [0x5567f3bdb85b] 2023-07-06T20:38:34.518 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 3: (OSD::shutdown()+0xdb2) [0x5567f3d087f2] 2023-07-06T20:38:34.518 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 4: (SignalHandler::entry()+0x657) [0x5567f4360e67] 2023-07-06T20:38:34.518 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 5: /lib64/libpthread.so.0(+0x81ca) [0x7f871fee01ca] 2023-07-06T20:38:34.518 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 6: clone() 2023-07-06T20:38:34.518 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.519 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 0> 2023-07-06T20:38:34.210+0000 7f87182a5700 -1 *** Caught signal (Aborted) ** 2023-07-06T20:38:34.519 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: in thread 7f87182a5700 thread_name:signal_handler 2023-07-06T20:38:34.519 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.519 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T20:38:34.519 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f871feeacf0] 2023-07-06T20:38:34.519 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2: gsignal() 2023-07-06T20:38:34.519 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 3: abort() 2023-07-06T20:38:34.520 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x5567f3bdb6ef] 2023-07-06T20:38:34.520 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 5: /usr/bin/ceph-osd(+0x63485b) [0x5567f3bdb85b] 2023-07-06T20:38:34.520 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 6: (OSD::shutdown()+0xdb2) [0x5567f3d087f2] 2023-07-06T20:38:34.520 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 7: (SignalHandler::entry()+0x657) [0x5567f4360e67] 2023-07-06T20:38:34.520 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7f871fee01ca] 2023-07-06T20:38:34.520 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 9: clone() 2023-07-06T20:38:34.520 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-07-06T20:38:34.520 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.521 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: -1> 2023-07-06T20:38:34.207+0000 7f87182a5700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/osd/OSD.cc: In function 'int OSD::shutdown()' thread 7f87182a5700 time 2023-07-06T20:38:34.207192+0000 2023-07-06T20:38:34.521 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4742-g79365bb3/rpm/el8/BUILD/ceph-18.0.0-4742-g79365bb3/src/osd/OSD.cc: 4501: FAILED ceph_assert(end_time - start_time_func < cct->_conf->osd_fast_shutdown_timeout) 2023-07-06T20:38:34.521 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.521 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T20:38:34.522 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x5567f3bdb695] 2023-07-06T20:38:34.522 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2: /usr/bin/ceph-osd(+0x63485b) [0x5567f3bdb85b] 2023-07-06T20:38:34.522 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 3: (OSD::shutdown()+0xdb2) [0x5567f3d087f2] 2023-07-06T20:38:34.522 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 4: (SignalHandler::entry()+0x657) [0x5567f4360e67] 2023-07-06T20:38:34.522 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 5: /lib64/libpthread.so.0(+0x81ca) [0x7f871fee01ca] 2023-07-06T20:38:34.522 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 6: clone() 2023-07-06T20:38:34.522 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.523 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 0> 2023-07-06T20:38:34.210+0000 7f87182a5700 -1 *** Caught signal (Aborted) ** 2023-07-06T20:38:34.523 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: in thread 7f87182a5700 thread_name:signal_handler 2023-07-06T20:38:34.523 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:34.523 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev) 2023-07-06T20:38:34.523 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f871feeacf0] 2023-07-06T20:38:34.523 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2: gsignal() 2023-07-06T20:38:34.523 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 3: abort() 2023-07-06T20:38:34.523 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x5567f3bdb6ef] 2023-07-06T20:38:34.524 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 5: /usr/bin/ceph-osd(+0x63485b) [0x5567f3bdb85b] 2023-07-06T20:38:34.524 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 6: (OSD::shutdown()+0xdb2) [0x5567f3d087f2] 2023-07-06T20:38:34.524 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 7: (SignalHandler::entry()+0x657) [0x5567f4360e67] 2023-07-06T20:38:34.524 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7f871fee01ca] 2023-07-06T20:38:34.524 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 9: clone() 2023-07-06T20:38:34.524 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-07-06T20:38:34.524 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0[139527]: 2023-07-06T20:38:35.004 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:34 smithi069 bash[183861]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-0 2023-07-06T20:38:35.259 INFO:journalctl@ceph.osd.0.smithi069.stdout:Jul 06 20:38:35 smithi069 systemd[1]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.0.service: Main process exited, code=exited, status=134/n/a 2023-07-06T20:38:36.587 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.0.service' 2023-07-06T20:38:36.675 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:38:36.675 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-06T20:38:36.676 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-06T20:38:36.676 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.1 2023-07-06T20:38:37.025 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:38:36 smithi098 systemd[1]: Stopping Ceph osd.1 for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:38:37.375 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:38:37 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-1[134064]: 2023-07-06T20:38:37.023+0000 7f7d90dbd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:38:37.376 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:38:37 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-1[134064]: 2023-07-06T20:38:37.023+0000 7f7d90dbd700 -1 osd.1 90 *** Got signal Terminated *** 2023-07-06T20:38:37.376 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:38:37 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-1[134064]: 2023-07-06T20:38:37.023+0000 7f7d90dbd700 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T20:38:42.875 INFO:journalctl@ceph.osd.1.smithi098.stdout:Jul 06 20:38:42 smithi098 bash[160342]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-1 2023-07-06T20:38:44.267 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.1.service' 2023-07-06T20:38:44.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:38:44.308 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-06T20:38:44.308 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-06T20:38:44.308 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.2 2023-07-06T20:38:44.607 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:38:44 smithi156 systemd[1]: Stopping Ceph osd.2 for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:38:45.107 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:38:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-2[135046]: 2023-07-06T20:38:44.666+0000 7fd04a77c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:38:45.107 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:38:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-2[135046]: 2023-07-06T20:38:44.666+0000 7fd04a77c700 -1 osd.2 90 *** Got signal Terminated *** 2023-07-06T20:38:45.107 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:38:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-2[135046]: 2023-07-06T20:38:44.666+0000 7fd04a77c700 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T20:38:50.652 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 06 20:38:50 smithi156 bash[162789]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-2 2023-07-06T20:38:52.015 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.2.service' 2023-07-06T20:38:52.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:38:52.056 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-06T20:38:52.056 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-06T20:38:52.056 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.3 2023-07-06T20:38:52.509 INFO:journalctl@ceph.osd.3.smithi069.stdout:Jul 06 20:38:52 smithi069 systemd[1]: Stopping Ceph osd.3 for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:38:53.009 INFO:journalctl@ceph.osd.3.smithi069.stdout:Jul 06 20:38:52 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-3[145011]: 2023-07-06T20:38:52.567+0000 7f96fadda700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:38:53.009 INFO:journalctl@ceph.osd.3.smithi069.stdout:Jul 06 20:38:52 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-3[145011]: 2023-07-06T20:38:52.567+0000 7f96fadda700 -1 osd.3 90 *** Got signal Terminated *** 2023-07-06T20:38:53.009 INFO:journalctl@ceph.osd.3.smithi069.stdout:Jul 06 20:38:52 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-3[145011]: 2023-07-06T20:38:52.567+0000 7f96fadda700 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T20:38:56.259 INFO:journalctl@ceph.osd.3.smithi069.stdout:Jul 06 20:38:55 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-3[145011]: 2023-07-06T20:38:55.990+0000 7f96f5bcd700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:11.549981+0000 front 2023-07-06T20:38:11.550008+0000 (oldest deadline 2023-07-06T20:38:55.552849+0000) 2023-07-06T20:38:56.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:38:55 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:38:55.974+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:38:56.857 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:38:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:38:56.570+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:38:57.009 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:38:56 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:38:56.702+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:38:57.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:38:56 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:38:56.966+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:38:57.509 INFO:journalctl@ceph.osd.3.smithi069.stdout:Jul 06 20:38:57 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-3[145011]: 2023-07-06T20:38:57.029+0000 7f96f5bcd700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:11.549981+0000 front 2023-07-06T20:38:11.550008+0000 (oldest deadline 2023-07-06T20:38:55.552849+0000) 2023-07-06T20:38:57.844 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:38:57 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:38:57.532+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:38:58.009 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:38:57 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:38:57.662+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:38:58.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:38:57 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:38:57.868+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:38:58.107 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:38:57 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:38:57.842+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:38:58.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:38:57 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:38:57.977+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:38:58.561 INFO:journalctl@ceph.osd.3.smithi069.stdout:Jul 06 20:38:58 smithi069 bash[184146]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-3 2023-07-06T20:38:58.808 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:38:58 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:38:58.490+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:38:58.895 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:38:58 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:38:58.615+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:38:59.107 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:38:58 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:38:58.806+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:38:59.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:38:58 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:38:58.892+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:38:59.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:38:58 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:38:58.943+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:38:59.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:38:59 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:38:59.577+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:38:59.854 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:38:59 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:38:59.510+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:00.106 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:38:59 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:38:59.852+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:00.171 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:38:59 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:38:59.913+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:00.195 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.3.service' 2023-07-06T20:39:00.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:39:00.285 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-06T20:39:00.285 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-06T20:39:00.285 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.4 2023-07-06T20:39:00.314 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:38:59 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:38:59.983+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:00.625 INFO:journalctl@ceph.osd.4.smithi098.stdout:Jul 06 20:39:00 smithi098 systemd[1]: Stopping Ceph osd.4 for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:39:00.852 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:00 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:00.520+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:00.920 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:00 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:00.556+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:00.973 INFO:journalctl@ceph.osd.4.smithi098.stdout:Jul 06 20:39:00 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-4[139599]: 2023-07-06T20:39:00.645+0000 7fd076430700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:39:00.973 INFO:journalctl@ceph.osd.4.smithi098.stdout:Jul 06 20:39:00 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-4[139599]: 2023-07-06T20:39:00.645+0000 7fd076430700 -1 osd.4 90 *** Got signal Terminated *** 2023-07-06T20:39:00.973 INFO:journalctl@ceph.osd.4.smithi098.stdout:Jul 06 20:39:00 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-4[139599]: 2023-07-06T20:39:00.645+0000 7fd076430700 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T20:39:01.107 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:00 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:00.849+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:01.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:00 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:00.919+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:01.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:00 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:00.970+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:01.857 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:01 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:01.535+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:01.919 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:01 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:01.516+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:02.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:01 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:01.918+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:02.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:01 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:01.862+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:02.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:02 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:02.012+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:02.845 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:02 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:02.576+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:02.920 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:02 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:02.515+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:03.107 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:02 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:02.843+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:03.107 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:02 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:02.843+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:03.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:02 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:02.918+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:03.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:02 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:02.974+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:03.594 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:03 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:03.319+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:03.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:03 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:03.473+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:03.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:03 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:03.473+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:03.856 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:03 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:03.592+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:03.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:03 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:03.609+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:04.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:03 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:03.888+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:04.347 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:03 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:03.890+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:04.347 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:03 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:03.890+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:04.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:04 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:04.018+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:04.605 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:04 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:04.345+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:04.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:04 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:04.436+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:04.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:04 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:04.436+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:04.856 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:04 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:04.603+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:05.035 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:04 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:04.629+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:05.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:04 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:04.873+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:05.319 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:04 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:04.924+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:05.320 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:04 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:04.924+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:05.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:05 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:05.033+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:05.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:05 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:05.033+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:05.606 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:05 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:05.317+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:05.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:05 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:05.406+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:05.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:05 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:05.406+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:05.964 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:05 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:05.625+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:06.018 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:05 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:05.675+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:06.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:05 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:05.873+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:06.348 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:05 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:05.962+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:06.349 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:05 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:05.962+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:06.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:06 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:06.016+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:06.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:06 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:06.016+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:06.607 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:06 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:06.590+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:06.607 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:06 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:06.346+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:06.723 INFO:journalctl@ceph.osd.4.smithi098.stdout:Jul 06 20:39:06 smithi098 bash[160612]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-4 2023-07-06T20:39:06.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:06 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:06.379+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:06.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:06 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:06.379+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:06.998 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:06 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:06.721+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:07.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:06 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:06.918+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:07.355 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:06 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:06.976+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:07.355 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:06 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:06.976+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:07.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:06 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:06.996+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:07.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:06 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:06.996+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:07.606 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:07 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:07.353+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:07.744 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.4.service' 2023-07-06T20:39:07.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:07 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:07.376+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:07.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:07 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:07.376+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:07.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:39:07.824 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-06T20:39:07.824 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-06T20:39:07.824 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.5 2023-07-06T20:39:07.988 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:07 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:07.617+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:07.988 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:07 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:07.617+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.391607+0000 front 2023-07-06T20:38:39.391572+0000 (oldest deadline 2023-07-06T20:39:06.992199+0000) 2023-07-06T20:39:07.988 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:07 smithi156 systemd[1]: Stopping Ceph osd.5 for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:39:08.039 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:07 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:07.702+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:08.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:07 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:07.893+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:08.354 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:08 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:08.230+0000 7f0aaa956700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:39:08.355 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:08 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:08.230+0000 7f0aaa956700 -1 osd.5 90 *** Got signal Terminated *** 2023-07-06T20:39:08.355 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:08 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:08.230+0000 7f0aaa956700 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T20:39:08.355 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:07 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:07.986+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:08.355 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:07 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:07.986+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:08.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:08 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:08.037+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:08.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:08 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:08.037+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:08.606 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:08 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:08.600+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:08.607 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:08 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:08.600+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.391607+0000 front 2023-07-06T20:38:39.391572+0000 (oldest deadline 2023-07-06T20:39:06.992199+0000) 2023-07-06T20:39:08.607 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:08 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:08.353+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:08.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:08 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:08.365+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:08.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:08 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:08.365+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:09.054 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:08 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:08.678+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:09.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:08 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:08.845+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:09.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:08 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:08.845+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:09.312 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:08 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:08.965+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:09.312 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:08 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:08.965+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:09.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:09 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:09.052+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:09.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:09 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:09.052+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:09.606 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:09 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:09.310+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:09.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:09 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:09.408+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:09.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:09 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:09.408+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:10.001 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:09 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:09.626+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:10.001 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:09 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:09.626+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.391607+0000 front 2023-07-06T20:38:39.391572+0000 (oldest deadline 2023-07-06T20:39:06.992199+0000) 2023-07-06T20:39:10.030 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:09 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:09.679+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:10.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:09 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:09.824+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:10.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:09 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:09.824+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:10.299 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:09 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:09.999+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:10.300 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:09 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:09.999+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:10.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:10 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:10.028+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:10.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:10 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:10.028+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:10.588 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:10 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:10.297+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:10.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:10 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:10.370+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:10.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:10 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:10.370+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:10.857 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:10 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:10.587+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:10.857 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:10 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:10.587+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.391607+0000 front 2023-07-06T20:38:39.391572+0000 (oldest deadline 2023-07-06T20:39:06.992199+0000) 2023-07-06T20:39:11.000 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:10 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:10.650+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:11.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:10 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:10.816+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:11.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:10 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:10.816+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:11.356 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:11.259+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:11.357 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:11.259+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:11.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:11.025+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:11.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:11.025+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:11.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:11.025+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:11.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:10 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:10.999+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:11.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:10 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:10.999+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:11.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:11 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:11.409+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:11.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:11 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:11.409+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:11.857 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:11.600+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:11.857 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:11.600+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.391607+0000 front 2023-07-06T20:38:39.391572+0000 (oldest deadline 2023-07-06T20:39:06.992199+0000) 2023-07-06T20:39:11.857 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:11.600+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:46.992596+0000 front 2023-07-06T20:38:46.992553+0000 (oldest deadline 2023-07-06T20:39:11.092825+0000) 2023-07-06T20:39:12.049 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:11 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:11.650+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:12.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:11 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:11.825+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:12.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:11 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:11.825+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:12.306 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:11.988+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:12.306 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:11.988+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:12.306 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:11 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:11.988+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:12.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:12 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:12.047+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:12.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:12 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:12.047+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:12.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:12 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:12.047+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:12.594 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:12 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:12.304+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:12.594 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:12 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:12.304+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:12.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:12 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:12.437+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:12.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:12 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:12.437+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:12.856 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:12 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:12.592+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:12.486989+0000 front 2023-07-06T20:38:12.487117+0000 (oldest deadline 2023-07-06T20:38:56.490480+0000) 2023-07-06T20:39:12.857 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:12 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:12.592+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.391607+0000 front 2023-07-06T20:38:39.391572+0000 (oldest deadline 2023-07-06T20:39:06.992199+0000) 2023-07-06T20:39:12.857 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:12 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5[140545]: 2023-07-06T20:39:12.592+0000 7f0aa774d700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:46.992596+0000 front 2023-07-06T20:38:46.992553+0000 (oldest deadline 2023-07-06T20:39:11.092825+0000) 2023-07-06T20:39:12.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:12 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:12.602+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:13.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:12 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:12.781+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:13.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:12 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:12.781+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:13.346 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:13 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:13.035+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:13.346 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:13 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:13.035+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:13.347 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:13 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:13.035+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:13.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:13 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:13.087+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:13.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:13 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:13.087+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:13.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:13 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:13.087+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:13.607 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:13 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:13.344+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:13.607 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:13 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:13.344+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:13.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:13 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:13.480+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:13.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:13 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:13.480+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:13.760 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:13 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:13.480+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:47.745928+0000 front 2023-07-06T20:38:47.746153+0000 (oldest deadline 2023-07-06T20:39:13.045944+0000) 2023-07-06T20:39:14.060 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 06 20:39:14 smithi156 bash[163053]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-5 2023-07-06T20:39:14.060 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:13 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:13.987+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:14.060 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:13 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:13.987+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:14.061 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:13 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:13.987+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:14.096 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:13 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:13.628+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:14.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:13 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:13.782+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:14.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:13 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:13.782+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:14.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:13 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:13.782+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:14.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:14 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:14.094+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:14.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:14 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:14.094+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:14.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:14 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:14.094+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:14.595 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:14 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:14.309+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:14.596 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:14 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:14.309+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:14.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:14 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:14.478+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:14.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:14 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:14.478+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:14.759 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:14 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:14.478+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:47.745928+0000 front 2023-07-06T20:38:47.746153+0000 (oldest deadline 2023-07-06T20:39:13.045944+0000) 2023-07-06T20:39:14.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:14 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:14.590+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:14.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:14 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:14.590+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:15.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:14 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:14.758+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:15.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:14 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:14.758+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:15.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:14 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:14.758+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:15.307 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:14 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:14.949+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:15.308 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:14 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:14.949+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:15.308 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:14 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:14.949+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:15.319 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.5.service' 2023-07-06T20:39:15.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:15 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:15.070+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:15.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:15 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:15.070+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:15.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:15 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:15.070+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:15.397 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:39:15.398 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-06T20:39:15.398 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-06T20:39:15.398 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.6 2023-07-06T20:39:15.606 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:15 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:15.305+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:15.607 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:15 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:15.305+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:15.759 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:15 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:15.727+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:15.760 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:15 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:15.727+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:15.760 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:15 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:15.727+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:15.760 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:15 smithi069 systemd[1]: Stopping Ceph osd.6 for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:39:15.761 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:15 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:15.512+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:15.761 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:15 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:15.512+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:15.761 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:15 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:15.512+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:47.745928+0000 front 2023-07-06T20:38:47.746153+0000 (oldest deadline 2023-07-06T20:39:13.045944+0000) 2023-07-06T20:39:15.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:15 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:15.589+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:15.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:15 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:15.589+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:16.259 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:15 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:15.912+0000 7f7880ee2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:39:16.259 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:15 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:15.912+0000 7f7880ee2700 -1 osd.6 90 *** Got signal Terminated *** 2023-07-06T20:39:16.259 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:15 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:15.912+0000 7f7880ee2700 -1 osd.6 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T20:39:16.354 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:15 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:15.914+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:16.354 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:15 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:15.914+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:16.354 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:15 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:15.914+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:16.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:16 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:16.082+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:16.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:16 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:16.082+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:16.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:16 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:16.082+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:16.607 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:16 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:16.352+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:16.607 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:16 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:16.352+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:16.748 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:16 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:16.462+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:16.749 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:16 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:16.462+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:16.749 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:16 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:16.462+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:47.745928+0000 front 2023-07-06T20:38:47.746153+0000 (oldest deadline 2023-07-06T20:39:13.045944+0000) 2023-07-06T20:39:16.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:16 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:16.565+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:16.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:16 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:16.565+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:17.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:16 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:16.746+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:17.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:16 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:16.746+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:17.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:16 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:16.746+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:17.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:16 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:16.948+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:17.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:16 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:16.948+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:17.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:16 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:16.948+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:17.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:17 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:17.042+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:17.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:17 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:17.042+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:17.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:17 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:17.042+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:17.724 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:17 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:17.414+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:17.725 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:17 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:17.414+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:17.725 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:17 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:17.414+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:47.745928+0000 front 2023-07-06T20:38:47.746153+0000 (oldest deadline 2023-07-06T20:39:13.045944+0000) 2023-07-06T20:39:17.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:17 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:17.386+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:17.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:17 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:17.386+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:17.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:17 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:17.540+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:17.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:17 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:17.540+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:18.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:17 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:17.722+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:18.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:17 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:17.722+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:18.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:17 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:17.722+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:18.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:17 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:17.924+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:18.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:17 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:17.924+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:18.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:17 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:17.924+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:18.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:18 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:18.083+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:18.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:18 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:18.083+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:18.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:18 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:18.083+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:18.706 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:18 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:18.367+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:18.706 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:18 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:18.367+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:18.707 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:18 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:18.367+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:47.745928+0000 front 2023-07-06T20:38:47.746153+0000 (oldest deadline 2023-07-06T20:39:13.045944+0000) 2023-07-06T20:39:18.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:18 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:18.384+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:18.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:18 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:18.384+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:18.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:18 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:18.499+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:18.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:18 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:18.499+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:19.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:18 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:18.704+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:19.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:18 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:18.704+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:19.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:18 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:18.704+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:19.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:18 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:18.916+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:19.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:18 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:18.916+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:19.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:18 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:18.916+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:19.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:19 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:19.114+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:19.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:19 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:19.114+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:19.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:19 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:19.114+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:19.717 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:19 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:19.357+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:19.717 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:19 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:19.357+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:19.717 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:19 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:19.357+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:47.745928+0000 front 2023-07-06T20:38:47.746153+0000 (oldest deadline 2023-07-06T20:39:13.045944+0000) 2023-07-06T20:39:19.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:19 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:19.375+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:19.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:19 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:19.375+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:19.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:19 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:19.519+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:19.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:19 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:19.519+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:20.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:19 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:19.714+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:20.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:19 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:19.714+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:20.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:19 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:19.714+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:20.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:19 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:19.714+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:20.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:19 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:19.946+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:20.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:19 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:19.946+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:20.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:19 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:19.946+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:20.547 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:20 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:20.158+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:20.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:20 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:20.158+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:20.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:20 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:20.158+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:20.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:20 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:20.158+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:20.720 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:20 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:20.382+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.238449+0000 front 2023-07-06T20:38:14.238384+0000 (oldest deadline 2023-07-06T20:38:55.741502+0000) 2023-07-06T20:39:20.720 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:20 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:20.382+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.243824+0000 front 2023-07-06T20:38:40.243675+0000 (oldest deadline 2023-07-06T20:39:03.143813+0000) 2023-07-06T20:39:20.720 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:20 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6[150584]: 2023-07-06T20:39:20.382+0000 7f787ccd7700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:47.745928+0000 front 2023-07-06T20:38:47.746153+0000 (oldest deadline 2023-07-06T20:39:13.045944+0000) 2023-07-06T20:39:20.856 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:20 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:20.413+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:20.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:20 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:20.413+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:20.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:20 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:20.545+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:20.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:20 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:20.545+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:20.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:20 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:20.545+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:57.223993+0000 front 2023-07-06T20:38:57.224029+0000 (oldest deadline 2023-07-06T20:39:19.524102+0000) 2023-07-06T20:39:21.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:20 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:20.718+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:21.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:20 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:20.718+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:21.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:20 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:20.718+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:21.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:20 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:20.718+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:21.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:20 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:20.950+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:21.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:20 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:20.950+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:21.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:20 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:20.950+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:21.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:21 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:21.207+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:21.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:21 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:21.207+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:21.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:21 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:21.207+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:21.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:21 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:21.207+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:21.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:21 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:21.452+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:21.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:21 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:21.452+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:21.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:21 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:21.452+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:21.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:21 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:21.501+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:21.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:21 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:21.501+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:21.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:21 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:21.501+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:57.223993+0000 front 2023-07-06T20:38:57.224029+0000 (oldest deadline 2023-07-06T20:39:19.524102+0000) 2023-07-06T20:39:21.914 INFO:journalctl@ceph.osd.6.smithi069.stdout:Jul 06 20:39:21 smithi069 bash[184422]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-6 2023-07-06T20:39:21.914 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:21 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:21.733+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:21.915 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:21 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:21.733+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:21.915 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:21 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:21.733+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:21.915 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:21 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:21.733+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:22.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:21 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:21.995+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:22.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:21 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:21.995+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:22.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:21 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:21.995+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:22.513 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:22 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:22.166+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:22.513 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:22 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:22.166+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:22.513 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:22 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:22.166+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:22.513 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:22 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:22.166+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:22.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:22 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:22.417+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:22.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:22 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:22.417+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:22.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:22 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:22.417+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:22.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:22 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:22.510+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:22.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:22 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:22.510+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:22.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:22 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:22.510+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:57.223993+0000 front 2023-07-06T20:38:57.224029+0000 (oldest deadline 2023-07-06T20:39:19.524102+0000) 2023-07-06T20:39:23.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:22 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:22.716+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:23.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:22 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:22.716+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:23.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:22 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:22.716+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:23.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:22 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:22.716+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:23.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:22 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:22.954+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:23.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:22 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:22.954+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:23.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:22 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:22.954+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:23.481 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:23 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:23.201+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:23.481 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:23 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:23.201+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:23.481 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:23 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:23.201+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:23.481 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:23 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:23.201+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:23.645 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.6.service' 2023-07-06T20:39:23.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:39:23.734 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-06T20:39:23.734 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-06T20:39:23.735 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.7 2023-07-06T20:39:23.764 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:23 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:23.478+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:23.765 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:23 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:23.478+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:23.765 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:23 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:23.478+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:57.223993+0000 front 2023-07-06T20:38:57.224029+0000 (oldest deadline 2023-07-06T20:39:19.524102+0000) 2023-07-06T20:39:23.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:23 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:23.438+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:23.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:23 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:23.438+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:23.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:23 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:23.438+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:24.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:23 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:23.682+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:24.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:23 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:23.682+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:24.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:23 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:23.682+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:24.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:23 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:23.682+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:24.125 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:23 smithi098 systemd[1]: Stopping Ceph osd.7 for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:39:24.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:23 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:23.945+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:24.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:23 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:23.945+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:24.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:23 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:23.945+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:24.506 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:24 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:24.174+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:24.507 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:24 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:24.174+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:24.507 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:24 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:24.174+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:24.507 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:24 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:24.174+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:24.507 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:24 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:24.141+0000 7ff7c0c89700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:39:24.508 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:24 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:24.141+0000 7ff7c0c89700 -1 osd.7 90 *** Got signal Terminated *** 2023-07-06T20:39:24.508 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:24 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:24.141+0000 7ff7c0c89700 -1 osd.7 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T20:39:24.508 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:24 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:24.504+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:24.508 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:24 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:24.504+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:24.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:24 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:24.438+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:24.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:24 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:24.438+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:24.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:24 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:24.438+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:24.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:24 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:24.504+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:57.223993+0000 front 2023-07-06T20:38:57.224029+0000 (oldest deadline 2023-07-06T20:39:19.524102+0000) 2023-07-06T20:39:25.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:24 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:24.687+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:25.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:24 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:24.687+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:25.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:24 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:24.687+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:25.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:24 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:24.687+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:25.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:24 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:24.897+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:25.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:24 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:24.897+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:25.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:24 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:24.897+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:25.469 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:25 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:25.160+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:25.469 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:25 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:25.160+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:25.470 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:25 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:25.160+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:25.470 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:25 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:25.160+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:25.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:25 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:25.480+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:25.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:25 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:25.480+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:25.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:25 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:25.480+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:25.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:25 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:25.467+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:25.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:25 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:25.467+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:25.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:25 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:25.467+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:57.223993+0000 front 2023-07-06T20:38:57.224029+0000 (oldest deadline 2023-07-06T20:39:19.524102+0000) 2023-07-06T20:39:26.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:25 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:25.715+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:26.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:25 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:25.715+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:26.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:25 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:25.715+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:26.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:25 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:25.715+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:26.356 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:25 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:25.907+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:26.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:25 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:25.907+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:26.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:25 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:25.907+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:26.496 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:26 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:26.195+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:26.497 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:26 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:26.195+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:26.497 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:26 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:26.195+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:26.497 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:26 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:26.195+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:26.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:26 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:26.505+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:26.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:26 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:26.505+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:26.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:26 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:26.505+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:26.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:26 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:26.494+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:26.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:26 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:26.494+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:26.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:26 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:26.494+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:57.223993+0000 front 2023-07-06T20:38:57.224029+0000 (oldest deadline 2023-07-06T20:39:19.524102+0000) 2023-07-06T20:39:27.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:26 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:26.754+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:27.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:26 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:26.754+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:27.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:26 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:26.754+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:27.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:26 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:26.754+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:27.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:26 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:26.944+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:27.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:26 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:26.944+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:27.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:26 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:26.944+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:27.626 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:27 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:27.448+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:27.626 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:27 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:27.448+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:27.626 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:27 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:27.448+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:57.223993+0000 front 2023-07-06T20:38:57.224029+0000 (oldest deadline 2023-07-06T20:39:19.524102+0000) 2023-07-06T20:39:27.627 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:27 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:27.448+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.825234+0000 front 2023-07-06T20:39:01.825210+0000 (oldest deadline 2023-07-06T20:39:27.125395+0000) 2023-07-06T20:39:27.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:27 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:27.233+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:27.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:27 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:27.233+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:27.628 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:27 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:27.233+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:27.628 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:27 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:27.233+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:27.856 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:27 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:27.471+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:27.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:27 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:27.471+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:27.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:27 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:27.471+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:28.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:27 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:27.775+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:28.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:27 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:27.775+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:28.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:27 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:27.775+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:28.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:27 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:27.775+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:28.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:27 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:27.775+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:28.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:27 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:27.924+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:28.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:27 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:27.924+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:28.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:27 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:27.924+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:28.466 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:28 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:28.193+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:28.466 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:28 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:28.193+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:28.466 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:28 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:28.193+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:28.466 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:28 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:28.193+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:28.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:28 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:28.456+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:28.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:28 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:28.456+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:28.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:28 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:28.456+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:28.875 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:28 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:28.463+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:36.920613+0000 front 2023-07-06T20:38:36.920632+0000 (oldest deadline 2023-07-06T20:39:02.721147+0000) 2023-07-06T20:39:28.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:28 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:28.463+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.422777+0000 front 2023-07-06T20:38:48.422927+0000 (oldest deadline 2023-07-06T20:39:14.322723+0000) 2023-07-06T20:39:28.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:28 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:28.463+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:57.223993+0000 front 2023-07-06T20:38:57.224029+0000 (oldest deadline 2023-07-06T20:39:19.524102+0000) 2023-07-06T20:39:28.876 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:28 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7[145049]: 2023-07-06T20:39:28.463+0000 7ff7ba279700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.825234+0000 front 2023-07-06T20:39:01.825210+0000 (oldest deadline 2023-07-06T20:39:27.125395+0000) 2023-07-06T20:39:29.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:28 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:28.765+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:29.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:28 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:28.765+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:29.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:28 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:28.765+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:29.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:28 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:28.765+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:29.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:28 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:28.765+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:29.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:28 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:28.918+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:29.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:28 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:28.918+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:29.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:28 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:28.918+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:29.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:29 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:29.212+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:29.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:29 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:29.212+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:29.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:29 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:29.212+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:29.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:29 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:29.212+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:29.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:29 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:29.424+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:29.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:29 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:29.424+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:29.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:29 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:29.424+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:29.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:29 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:29.424+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:05.048586+0000 front 2023-07-06T20:39:05.048486+0000 (oldest deadline 2023-07-06T20:39:28.548518+0000) 2023-07-06T20:39:30.202 INFO:journalctl@ceph.osd.7.smithi098.stdout:Jul 06 20:39:29 smithi098 bash[160877]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-7 2023-07-06T20:39:30.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:29 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:29.777+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:30.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:29 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:29.777+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:30.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:29 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:29.777+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:30.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:29 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:29.777+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:30.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:29 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:29.777+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:30.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:29 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:29.951+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:30.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:29 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:29.951+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:30.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:29 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:29.951+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:30.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:30 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:30.199+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:30.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:30 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:30.199+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:30.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:30 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:30.199+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:30.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:30 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:30.199+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:30.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:30 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:30.458+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:30.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:30 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:30.458+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:30.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:30 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:30.458+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:30.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:30 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:30.458+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:05.048586+0000 front 2023-07-06T20:39:05.048486+0000 (oldest deadline 2023-07-06T20:39:28.548518+0000) 2023-07-06T20:39:31.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:30 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:30.740+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:31.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:30 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:30.740+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:31.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:30 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:30.740+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:31.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:30 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:30.740+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:31.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:30 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:30.740+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:31.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:30 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:30.982+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:31.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:30 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:30.982+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:31.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:30 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:30.982+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:31.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:30 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:30.982+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:31.375 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:31 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:31.222+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:31.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:31 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:31.222+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:31.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:31 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:31.222+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:31.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:31 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:31.222+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:31.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:31 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:31.222+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:31.394 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.7.service' 2023-07-06T20:39:31.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:39:31.474 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-06T20:39:31.474 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-06T20:39:31.475 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.8 2023-07-06T20:39:31.845 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:31 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:31.441+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:31.846 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:31 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:31.441+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:31.846 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:31 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:31.441+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:31.846 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:31 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:31.441+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:05.048586+0000 front 2023-07-06T20:39:05.048486+0000 (oldest deadline 2023-07-06T20:39:28.548518+0000) 2023-07-06T20:39:31.846 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:31 smithi156 systemd[1]: Stopping Ceph osd.8 for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:39:32.106 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:31 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:31.843+0000 7efd39ea8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:39:32.107 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:31 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:31.843+0000 7efd39ea8700 -1 osd.8 90 *** Got signal Terminated *** 2023-07-06T20:39:32.107 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:31 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:31.843+0000 7efd39ea8700 -1 osd.8 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T20:39:32.107 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:31 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:31.934+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:32.107 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:31 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:31.934+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:32.108 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:31 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:31.934+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:32.108 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:31 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:31.934+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:32.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:31 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:31.768+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:32.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:31 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:31.768+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:32.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:31 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:31.768+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:32.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:31 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:31.768+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:32.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:31 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:31.768+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:32.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:32 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:32.189+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:32.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:32 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:32.189+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:32.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:32 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:32.189+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:32.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:32 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:32.189+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:32.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:32 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:32.189+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:32.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:32 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:32.458+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:32.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:32 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:32.458+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:32.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:32 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:32.458+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:32.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:32 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:32.458+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:05.048586+0000 front 2023-07-06T20:39:05.048486+0000 (oldest deadline 2023-07-06T20:39:28.548518+0000) 2023-07-06T20:39:33.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:32 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:32.762+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:33.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:32 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:32.762+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:33.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:32 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:32.762+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:33.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:32 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:32.762+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:33.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:32 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:32.762+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:33.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:32 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:32.902+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:33.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:32 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:32.902+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:33.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:32 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:32.902+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:33.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:32 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:32.902+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:33.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:33 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:33.209+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:33.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:33 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:33.209+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:33.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:33 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:33.209+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:33.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:33 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:33.209+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:33.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:33 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:33.209+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:33.856 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:33 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:33.461+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:33.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:33 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:33.461+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:33.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:33 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:33.461+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:33.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:33 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:33.461+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:05.048586+0000 front 2023-07-06T20:39:05.048486+0000 (oldest deadline 2023-07-06T20:39:28.548518+0000) 2023-07-06T20:39:34.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:33 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:33.783+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:34.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:33 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:33.783+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:34.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:33 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:33.783+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:34.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:33 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:33.783+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:34.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:33 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:33.783+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:34.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:33 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:33.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:34.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:33 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:33.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:34.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:33 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:33.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:34.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:33 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:33.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:34.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:34 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:34.239+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:34.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:34 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:34.239+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:34.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:34 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:34.239+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:34.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:34 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:34.239+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:34.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:34 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:34.239+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:34.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:34 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:34.480+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:34.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:34 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:34.480+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:34.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:34 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:34.480+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:34.858 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:34 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:34.480+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:05.048586+0000 front 2023-07-06T20:39:05.048486+0000 (oldest deadline 2023-07-06T20:39:28.548518+0000) 2023-07-06T20:39:34.858 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:34 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:34.480+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:08.548926+0000 front 2023-07-06T20:39:08.548942+0000 (oldest deadline 2023-07-06T20:39:33.849062+0000) 2023-07-06T20:39:35.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:34.770+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:35.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:34.770+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:35.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:34.770+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:35.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:34.770+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:35.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:34.770+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:35.260 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:34 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:34.770+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:11.517970+0000 front 2023-07-06T20:39:11.517855+0000 (oldest deadline 2023-07-06T20:39:34.417964+0000) 2023-07-06T20:39:35.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:34 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:34.943+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:35.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:34 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:34.943+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:35.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:34 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:34.943+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:35.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:34 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:34.943+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:35.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:35 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:35.233+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:35.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:35 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:35.233+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:35.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:35 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:35.233+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:35.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:35 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:35.233+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:35.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:35 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:35.233+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:35.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:35 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:35.233+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:35.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:35 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:35.501+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:35.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:35 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:35.501+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:35.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:35 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:35.501+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:35.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:35 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:35.501+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:05.048586+0000 front 2023-07-06T20:39:05.048486+0000 (oldest deadline 2023-07-06T20:39:28.548518+0000) 2023-07-06T20:39:35.858 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:35 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:35.501+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:08.548926+0000 front 2023-07-06T20:39:08.548942+0000 (oldest deadline 2023-07-06T20:39:33.849062+0000) 2023-07-06T20:39:36.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:35 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:35.722+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:36.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:35 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:35.722+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:36.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:35 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:35.722+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:36.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:35 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:35.722+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:36.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:35 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:35.722+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:36.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:35 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:35.722+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:11.517970+0000 front 2023-07-06T20:39:11.517855+0000 (oldest deadline 2023-07-06T20:39:34.417964+0000) 2023-07-06T20:39:36.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:35 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:35.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:36.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:35 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:35.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:36.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:35 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:35.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:36.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:35 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:35.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:36.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:35 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:35.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:36.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:36 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:36.211+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:36.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:36 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:36.211+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:36.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:36 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:36.211+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:36.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:36 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:36.211+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:36.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:36 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:36.211+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:36.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:36 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:36.211+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:36.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:36 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:36.551+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:39.741541+0000 front 2023-07-06T20:38:39.741377+0000 (oldest deadline 2023-07-06T20:39:03.241606+0000) 2023-07-06T20:39:36.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:36 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:36.551+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.344073+0000 front 2023-07-06T20:38:48.344052+0000 (oldest deadline 2023-07-06T20:39:11.244337+0000) 2023-07-06T20:39:36.857 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:36 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:36.551+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:55.846207+0000 front 2023-07-06T20:38:55.846420+0000 (oldest deadline 2023-07-06T20:39:20.546132+0000) 2023-07-06T20:39:36.858 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:36 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:36.551+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:05.048586+0000 front 2023-07-06T20:39:05.048486+0000 (oldest deadline 2023-07-06T20:39:28.548518+0000) 2023-07-06T20:39:36.858 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:36 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8[145987]: 2023-07-06T20:39:36.551+0000 7efd34c9b700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:08.548926+0000 front 2023-07-06T20:39:08.548942+0000 (oldest deadline 2023-07-06T20:39:33.849062+0000) 2023-07-06T20:39:37.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:36.676+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:37.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:36.676+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:37.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:36.676+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:37.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:36.676+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:37.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:36.676+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:37.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:36 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:36.676+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:11.517970+0000 front 2023-07-06T20:39:11.517855+0000 (oldest deadline 2023-07-06T20:39:34.417964+0000) 2023-07-06T20:39:37.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:36 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:36.954+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:37.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:36 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:36.954+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:37.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:36 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:36.954+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:37.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:36 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:36.954+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:37.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:36 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:36.954+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:37.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:37 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:37.234+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:37.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:37 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:37.234+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:37.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:37 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:37.234+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:37.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:37 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:37.234+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:37.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:37 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:37.234+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:37.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:37 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:37.234+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:38.004 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 06 20:39:37 smithi156 bash[163327]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-8 2023-07-06T20:39:38.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:37 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:37.673+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:38.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:37 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:37.673+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:38.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:37 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:37.673+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:38.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:37 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:37.673+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:38.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:37 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:37.673+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:38.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:37 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:37.673+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:11.517970+0000 front 2023-07-06T20:39:11.517855+0000 (oldest deadline 2023-07-06T20:39:34.417964+0000) 2023-07-06T20:39:38.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:38 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:38.002+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:38.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:38 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:38.002+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:38.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:38 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:38.002+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:38.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:38 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:38.002+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:38.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:38 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:38.002+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:38.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:38 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:38.210+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:38.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:38 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:38.210+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:38.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:38 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:38.210+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:38.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:38 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:38.210+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:38.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:38 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:38.210+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:38.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:38 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:38.210+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:39.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:38 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:38.705+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:39.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:38 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:38.705+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:39.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:38 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:38.705+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:39.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:38 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:38.705+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:39.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:38 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:38.705+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:39.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:38 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:38.705+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:11.517970+0000 front 2023-07-06T20:39:11.517855+0000 (oldest deadline 2023-07-06T20:39:34.417964+0000) 2023-07-06T20:39:39.107 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:39 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:39.016+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:39.107 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:39 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:39.016+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:39.107 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:39 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:39.016+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:39.107 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:39 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:39.016+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:39.108 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:39 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:39.016+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:39.216 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.8.service' 2023-07-06T20:39:39.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:39:39.257 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-06T20:39:39.257 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-06T20:39:39.257 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.9 2023-07-06T20:39:39.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:39 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:39.175+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:39.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:39 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:39.175+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:39.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:39 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:39.175+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:39.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:39 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:39.175+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:39.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:39 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:39.175+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:39.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:39 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:39.175+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:39.699 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:39 smithi069 systemd[1]: Stopping Ceph osd.9 for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:39:40.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:39 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:39.697+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:40.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:39 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:39.697+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:40.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:39 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:39.697+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:40.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:39 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:39.697+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:40.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:39 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:39.697+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:40.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:39 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:39.697+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:11.517970+0000 front 2023-07-06T20:39:11.517855+0000 (oldest deadline 2023-07-06T20:39:34.417964+0000) 2023-07-06T20:39:40.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:39 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:39.759+0000 7faaf0cc8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:39:40.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:39 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:39.759+0000 7faaf0cc8700 -1 osd.9 90 *** Got signal Terminated *** 2023-07-06T20:39:40.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:39 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:39.759+0000 7faaf0cc8700 -1 osd.9 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T20:39:40.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:39 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:39.970+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:40.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:39 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:39.970+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:40.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:39 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:39.970+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:40.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:39 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:39.970+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:40.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:39 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:39.970+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:40.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:40 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:40.166+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:40.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:40 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:40.166+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:40.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:40 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:40.166+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:40.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:40 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:40.166+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:40.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:40 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:40.166+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:40.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:40 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:40.166+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:41.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:40 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:40.676+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:41.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:40 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:40.676+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:41.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:40 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:40.676+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:41.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:40 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:40.676+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:41.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:40 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:40.676+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:41.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:40 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:40.676+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:11.517970+0000 front 2023-07-06T20:39:11.517855+0000 (oldest deadline 2023-07-06T20:39:34.417964+0000) 2023-07-06T20:39:41.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:40 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:40.945+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:41.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:40 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:40.945+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:41.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:40 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:40.945+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:41.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:40 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:40.945+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:41.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:40 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:40.945+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:41.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:41 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:41.209+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:41.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:41 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:41.209+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:41.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:41 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:41.209+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:41.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:41 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:41.209+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:41.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:41 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:41.209+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:41.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:41 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:41.209+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:42.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:41 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:41.644+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:42.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:41 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:41.644+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:42.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:41 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:41.644+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:42.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:41 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:41.644+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:42.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:41 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:41.644+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:42.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:41 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:41.644+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:11.517970+0000 front 2023-07-06T20:39:11.517855+0000 (oldest deadline 2023-07-06T20:39:34.417964+0000) 2023-07-06T20:39:42.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:41 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:41.963+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:42.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:41 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:41.963+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:42.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:41 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:41.963+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:42.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:41 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:41.963+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:42.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:41 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:41.963+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:42.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:42 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:42.255+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:42.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:42 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:42.255+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:42.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:42 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:42.255+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:42.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:42 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:42.255+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:42.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:42 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:42.255+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:42.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:42 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:42.255+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:43.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:42 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:42.608+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:43.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:42 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:42.608+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:43.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:42 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:42.608+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:43.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:42 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:42.608+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:43.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:42 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:42.608+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:43.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:42 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:42.608+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:11.517970+0000 front 2023-07-06T20:39:11.517855+0000 (oldest deadline 2023-07-06T20:39:34.417964+0000) 2023-07-06T20:39:43.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:42 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:42.987+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:43.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:42 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:42.987+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:43.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:42 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:42.987+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:43.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:42 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:42.987+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:43.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:42 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:42.987+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:43.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:43 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:43.212+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:43.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:43 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:43.212+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:43.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:43 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:43.212+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:43.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:43 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:43.212+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:43.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:43 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:43.212+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:43.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:43 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:43.212+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:44.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:43 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:43.656+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:44.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:43 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:43.656+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:44.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:43 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:43.656+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:44.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:43 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:43.656+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:44.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:43 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:43.657+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:44.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:43 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:43.657+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:11.517970+0000 front 2023-07-06T20:39:11.517855+0000 (oldest deadline 2023-07-06T20:39:34.417964+0000) 2023-07-06T20:39:44.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:44.020+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:44.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:44.020+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:44.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:44.020+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:44.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:44.020+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:44.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:44.020+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:44.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:44 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:44.219+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:44.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:44 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:44.219+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:44.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:44 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:44.219+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:44.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:44 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:44.219+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:44.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:44 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:44.219+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:44.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:44 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:44.219+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:45.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:44 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:44.669+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:13.108429+0000 front 2023-07-06T20:38:13.108547+0000 (oldest deadline 2023-07-06T20:38:57.611370+0000) 2023-07-06T20:39:45.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:44 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:44.669+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:41.712588+0000 front 2023-07-06T20:38:41.712503+0000 (oldest deadline 2023-07-06T20:39:08.113212+0000) 2023-07-06T20:39:45.009 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:44 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:44.669+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.114163+0000 front 2023-07-06T20:38:48.113979+0000 (oldest deadline 2023-07-06T20:39:12.813972+0000) 2023-07-06T20:39:45.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:44 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:44.669+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:52.814440+0000 front 2023-07-06T20:38:52.814419+0000 (oldest deadline 2023-07-06T20:39:18.714755+0000) 2023-07-06T20:39:45.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:44 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:44.669+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:01.016207+0000 front 2023-07-06T20:39:01.016278+0000 (oldest deadline 2023-07-06T20:39:26.916051+0000) 2023-07-06T20:39:45.010 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:44 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9[156056]: 2023-07-06T20:39:44.669+0000 7faaebabb700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:11.517970+0000 front 2023-07-06T20:39:11.517855+0000 (oldest deadline 2023-07-06T20:39:34.417964+0000) 2023-07-06T20:39:45.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:44.985+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:45.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:44.985+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:45.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:44.985+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:45.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:44.985+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:45.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:44 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:44.985+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:45.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:45 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:45.200+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:45.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:45 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:45.200+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:45.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:45 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:45.200+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:45.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:45 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:45.200+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:45.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:45 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:45.200+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:45.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:45 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:45.200+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:45.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:45 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:45.200+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.317427+0000 front 2023-07-06T20:39:20.317398+0000 (oldest deadline 2023-07-06T20:39:45.017338+0000) 2023-07-06T20:39:46.259 INFO:journalctl@ceph.osd.9.smithi069.stdout:Jul 06 20:39:45 smithi069 bash[184689]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-9 2023-07-06T20:39:46.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:45 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:45.955+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:46.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:45 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:45.955+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:46.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:45 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:45.955+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:46.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:45 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:45.955+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:46.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:45 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:45.955+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:46.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:46 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:46.218+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:46.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:46 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:46.218+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:46.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:46 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:46.218+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:46.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:46 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:46.218+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:46.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:46 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:46.218+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:46.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:46 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:46.218+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:46.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:46 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:46.218+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.317427+0000 front 2023-07-06T20:39:20.317398+0000 (oldest deadline 2023-07-06T20:39:45.017338+0000) 2023-07-06T20:39:47.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:46 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:46.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:47.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:46 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:46.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:47.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:46 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:46.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:47.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:46 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:46.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:47.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:46 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:46.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:47.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:46 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:46.930+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:47.578 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.9.service' 2023-07-06T20:39:47.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:47 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:47.262+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:47.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:47 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:47.262+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:47.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:47 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:47.262+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:47.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:47 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:47.262+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:47.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:47 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:47.262+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:47.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:47 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:47.262+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:47.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:47 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:47.262+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.317427+0000 front 2023-07-06T20:39:20.317398+0000 (oldest deadline 2023-07-06T20:39:45.017338+0000) 2023-07-06T20:39:47.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:39:47.667 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-06T20:39:47.667 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-06T20:39:47.668 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.10 2023-07-06T20:39:48.034 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:47 smithi098 systemd[1]: Stopping Ceph osd.10 for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:39:48.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:47 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:47.970+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:48.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:47 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:47.970+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:48.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:47 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:47.970+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:48.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:47 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:47.970+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:48.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:47 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:47.970+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:48.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:47 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:47.970+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:48.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:48 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:48.030+0000 7fd7d8a49700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:39:48.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:48 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:48.030+0000 7fd7d8a49700 -1 osd.10 90 *** Got signal Terminated *** 2023-07-06T20:39:48.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:48 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:48.030+0000 7fd7d8a49700 -1 osd.10 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T20:39:48.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:48 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:48.258+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:48.376 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:48 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:48.258+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:48.377 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:48 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:48.258+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:48.377 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:48 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:48.258+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:48.377 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:48 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:48.258+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:48.377 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:48 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:48.258+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:48.377 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:48 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:48.258+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.317427+0000 front 2023-07-06T20:39:20.317398+0000 (oldest deadline 2023-07-06T20:39:45.017338+0000) 2023-07-06T20:39:49.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:48 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:48.976+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:49.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:48 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:48.976+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:49.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:48 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:48.976+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:49.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:48 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:48.976+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:49.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:48 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:48.976+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:49.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:48 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:48.976+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:49.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:49 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:49.290+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:49.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:49 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:49.290+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:49.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:49 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:49.290+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:49.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:49 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:49.290+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:49.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:49 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:49.290+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:49.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:49 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:49.290+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:49.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:49 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:49.290+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.317427+0000 front 2023-07-06T20:39:20.317398+0000 (oldest deadline 2023-07-06T20:39:45.017338+0000) 2023-07-06T20:39:50.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:49 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:49.933+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:50.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:49 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:49.933+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:50.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:49 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:49.933+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:50.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:49 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:49.933+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:50.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:49 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:49.933+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:50.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:49 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:49.933+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:50.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:50 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:50.298+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:50.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:50 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:50.298+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:50.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:50 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:50.298+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:50.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:50 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:50.298+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:50.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:50 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:50.298+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:50.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:50 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:50.298+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:50.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:50 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:50.298+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.317427+0000 front 2023-07-06T20:39:20.317398+0000 (oldest deadline 2023-07-06T20:39:45.017338+0000) 2023-07-06T20:39:51.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:50 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:50.956+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:51.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:50 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:50.956+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:51.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:50 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:50.956+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:51.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:50 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:50.956+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:51.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:50 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:50.956+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:51.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:50 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:50.956+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:51.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:51 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:51.314+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:51.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:51 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:51.314+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:51.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:51 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:51.314+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:51.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:51 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:51.314+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:51.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:51 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:51.314+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:51.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:51 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:51.314+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:51.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:51 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:51.314+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.317427+0000 front 2023-07-06T20:39:20.317398+0000 (oldest deadline 2023-07-06T20:39:45.017338+0000) 2023-07-06T20:39:52.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:51 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:51.968+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:52.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:51 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:51.968+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:52.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:51 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:51.968+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:52.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:51 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:51.968+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:52.359 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:51 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:51.968+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:52.359 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:51 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:51.968+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:52.359 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:51 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:51.968+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-07-06T20:39:26.268352+0000 front 2023-07-06T20:39:26.268338+0000 (oldest deadline 2023-07-06T20:39:51.568387+0000) 2023-07-06T20:39:52.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:52 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:52.360+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.507155+0000 front 2023-07-06T20:38:14.507170+0000 (oldest deadline 2023-07-06T20:38:55.108844+0000) 2023-07-06T20:39:52.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:52 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:52.360+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.309998+0000 front 2023-07-06T20:38:40.310027+0000 (oldest deadline 2023-07-06T20:39:04.410233+0000) 2023-07-06T20:39:52.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:52 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:52.360+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:48.912359+0000 front 2023-07-06T20:38:48.912264+0000 (oldest deadline 2023-07-06T20:39:11.712788+0000) 2023-07-06T20:39:52.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:52 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:52.360+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2023-07-06T20:38:53.413778+0000 front 2023-07-06T20:38:53.413751+0000 (oldest deadline 2023-07-06T20:39:19.313668+0000) 2023-07-06T20:39:52.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:52 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:52.360+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:04.614654+0000 front 2023-07-06T20:39:04.614675+0000 (oldest deadline 2023-07-06T20:39:30.514782+0000) 2023-07-06T20:39:52.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:52 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:52.360+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:12.215949+0000 front 2023-07-06T20:39:12.215895+0000 (oldest deadline 2023-07-06T20:39:34.515860+0000) 2023-07-06T20:39:52.627 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:52 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10[150613]: 2023-07-06T20:39:52.360+0000 7fd7d1838700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.317427+0000 front 2023-07-06T20:39:20.317398+0000 (oldest deadline 2023-07-06T20:39:45.017338+0000) 2023-07-06T20:39:53.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:52 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:52.937+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:53.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:52 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:52.937+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:53.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:52 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:52.937+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:53.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:52 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:52.937+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:53.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:52 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:52.937+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:53.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:52 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:52.937+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:53.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:52 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:52.937+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-07-06T20:39:26.268352+0000 front 2023-07-06T20:39:26.268338+0000 (oldest deadline 2023-07-06T20:39:51.568387+0000) 2023-07-06T20:39:54.125 INFO:journalctl@ceph.osd.10.smithi098.stdout:Jul 06 20:39:53 smithi098 bash[161154]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-10 2023-07-06T20:39:54.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:53 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:53.894+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:54.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:53 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:53.894+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:54.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:53 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:53.894+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:54.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:53 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:53.894+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:54.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:53 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:53.894+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:54.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:53 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:53.894+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:54.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:53 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:53.894+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-07-06T20:39:26.268352+0000 front 2023-07-06T20:39:26.268338+0000 (oldest deadline 2023-07-06T20:39:51.568387+0000) 2023-07-06T20:39:55.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.10.service' 2023-07-06T20:39:55.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:39:55.241 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-06T20:39:55.241 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-06T20:39:55.241 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.11 2023-07-06T20:39:55.271 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:54 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:54.932+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:55.272 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:54 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:54.932+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:55.272 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:54 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:54.932+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:55.272 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:54 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:54.932+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:55.272 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:54 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:54.932+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:55.272 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:54 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:54.932+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:55.273 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:54 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:54.932+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-07-06T20:39:26.268352+0000 front 2023-07-06T20:39:26.268338+0000 (oldest deadline 2023-07-06T20:39:51.568387+0000) 2023-07-06T20:39:55.600 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:55 smithi156 systemd[1]: Stopping Ceph osd.11 for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:39:55.857 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:55 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:55.597+0000 7fab162a6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:39:55.857 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:55 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:55.597+0000 7fab162a6700 -1 osd.11 90 *** Got signal Terminated *** 2023-07-06T20:39:55.857 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:55 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:55.597+0000 7fab162a6700 -1 osd.11 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-06T20:39:56.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:55 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:55.961+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:56.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:55 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:55.961+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:56.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:55 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:55.961+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:56.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:55 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:55.961+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:56.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:55 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:55.961+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:56.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:55 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:55.961+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:56.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:55 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:55.961+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-07-06T20:39:26.268352+0000 front 2023-07-06T20:39:26.268338+0000 (oldest deadline 2023-07-06T20:39:51.568387+0000) 2023-07-06T20:39:57.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:56.995+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:57.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:56.995+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:57.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:56.995+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:57.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:56.995+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:57.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:56.995+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:57.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:56.995+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:57.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:56 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:56.995+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-07-06T20:39:26.268352+0000 front 2023-07-06T20:39:26.268338+0000 (oldest deadline 2023-07-06T20:39:51.568387+0000) 2023-07-06T20:39:58.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:58 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:58.044+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:58.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:58 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:58.044+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:58.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:58 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:58.044+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:58.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:58 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:58.044+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:58.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:58 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:58.044+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:58.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:58 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:58.044+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:58.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:58 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:58.044+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-07-06T20:39:26.268352+0000 front 2023-07-06T20:39:26.268338+0000 (oldest deadline 2023-07-06T20:39:51.568387+0000) 2023-07-06T20:39:59.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:59 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:59.066+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:39:59.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:59 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:59.066+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:39:59.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:59 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:59.066+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:39:59.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:59 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:59.066+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:39:59.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:59 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:59.066+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:39:59.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:59 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:59.066+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:39:59.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:39:59 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:39:59.066+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-07-06T20:39:26.268352+0000 front 2023-07-06T20:39:26.268338+0000 (oldest deadline 2023-07-06T20:39:51.568387+0000) 2023-07-06T20:40:00.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:40:00 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:40:00.046+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2023-07-06T20:38:14.655430+0000 front 2023-07-06T20:38:14.655568+0000 (oldest deadline 2023-07-06T20:38:57.458380+0000) 2023-07-06T20:40:00.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:40:00 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:40:00.046+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2023-07-06T20:38:40.359300+0000 front 2023-07-06T20:38:40.359409+0000 (oldest deadline 2023-07-06T20:39:02.659454+0000) 2023-07-06T20:40:00.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:40:00 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:40:00.046+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-06T20:38:44.861221+0000 front 2023-07-06T20:38:44.861005+0000 (oldest deadline 2023-07-06T20:39:10.761146+0000) 2023-07-06T20:40:00.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:40:00 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:40:00.046+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2023-07-06T20:39:02.864900+0000 front 2023-07-06T20:39:02.864837+0000 (oldest deadline 2023-07-06T20:39:30.465722+0000) 2023-07-06T20:40:00.357 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:40:00 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:40:00.046+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-06T20:39:10.466103+0000 front 2023-07-06T20:39:10.466158+0000 (oldest deadline 2023-07-06T20:39:35.166265+0000) 2023-07-06T20:40:00.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:40:00 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:40:00.046+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2023-07-06T20:39:20.367826+0000 front 2023-07-06T20:39:20.367806+0000 (oldest deadline 2023-07-06T20:39:46.267807+0000) 2023-07-06T20:40:00.358 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:40:00 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11[151496]: 2023-07-06T20:40:00.046+0000 7fab0f095700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2023-07-06T20:39:26.268352+0000 front 2023-07-06T20:39:26.268338+0000 (oldest deadline 2023-07-06T20:39:51.568387+0000) 2023-07-06T20:40:01.481 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 06 20:40:01 smithi156 bash[163597]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-osd-11 2023-07-06T20:40:02.741 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@osd.11.service' 2023-07-06T20:40:02.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:02.823 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-06T20:40:02.824 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-06T20:40:02.824 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.a 2023-07-06T20:40:03.259 INFO:journalctl@ceph.mds.a.smithi069.stdout:Jul 06 20:40:02 smithi069 systemd[1]: Stopping Ceph mds.a for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:40:03.759 INFO:journalctl@ceph.mds.a.smithi069.stdout:Jul 06 20:40:03 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-a[161438]: 2023-07-06T20:40:03.313+0000 7f708c94e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:40:03.759 INFO:journalctl@ceph.mds.a.smithi069.stdout:Jul 06 20:40:03 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-a[161438]: 2023-07-06T20:40:03.313+0000 7f708c94e700 -1 mds.a *** got signal Terminated *** 2023-07-06T20:40:04.689 INFO:journalctl@ceph.mds.a.smithi069.stdout:Jul 06 20:40:04 smithi069 bash[184960]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-a 2023-07-06T20:40:04.982 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.a.service' 2023-07-06T20:40:05.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:05.065 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-06T20:40:05.065 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-06T20:40:05.065 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.d 2023-07-06T20:40:05.259 INFO:journalctl@ceph.mds.d.smithi069.stdout:Jul 06 20:40:05 smithi069 systemd[1]: Stopping Ceph mds.d for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:40:06.009 INFO:journalctl@ceph.mds.d.smithi069.stdout:Jul 06 20:40:05 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-d[160431]: 2023-07-06T20:40:05.546+0000 7f11bbaf4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:40:06.009 INFO:journalctl@ceph.mds.d.smithi069.stdout:Jul 06 20:40:05 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-d[160431]: 2023-07-06T20:40:05.546+0000 7f11bbaf4700 -1 mds.d *** got signal Terminated *** 2023-07-06T20:40:07.324 INFO:journalctl@ceph.mds.d.smithi069.stdout:Jul 06 20:40:07 smithi069 bash[185100]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-d 2023-07-06T20:40:07.620 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.d.service' 2023-07-06T20:40:07.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:07.702 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-06T20:40:07.702 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-06T20:40:07.703 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.g 2023-07-06T20:40:08.009 INFO:journalctl@ceph.mds.g.smithi069.stdout:Jul 06 20:40:07 smithi069 systemd[1]: Stopping Ceph mds.g for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:40:08.509 INFO:journalctl@ceph.mds.g.smithi069.stdout:Jul 06 20:40:08 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-g[161050]: 2023-07-06T20:40:08.187+0000 7ff566702700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:40:08.509 INFO:journalctl@ceph.mds.g.smithi069.stdout:Jul 06 20:40:08 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-g[161050]: 2023-07-06T20:40:08.187+0000 7ff566702700 -1 mds.g *** got signal Terminated *** 2023-07-06T20:40:10.009 INFO:journalctl@ceph.mds.g.smithi069.stdout:Jul 06 20:40:09 smithi069 bash[185239]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-g 2023-07-06T20:40:10.134 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.g.service' 2023-07-06T20:40:10.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:10.216 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-06T20:40:10.216 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-06T20:40:10.216 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.j 2023-07-06T20:40:10.509 INFO:journalctl@ceph.mds.j.smithi069.stdout:Jul 06 20:40:10 smithi069 systemd[1]: Stopping Ceph mds.j for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:40:11.009 INFO:journalctl@ceph.mds.j.smithi069.stdout:Jul 06 20:40:10 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-j[160041]: 2023-07-06T20:40:10.704+0000 7f8f8ea3c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:40:11.009 INFO:journalctl@ceph.mds.j.smithi069.stdout:Jul 06 20:40:10 smithi069 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-j[160041]: 2023-07-06T20:40:10.704+0000 7f8f8ea3c700 -1 mds.j *** got signal Terminated *** 2023-07-06T20:40:12.509 INFO:journalctl@ceph.mds.j.smithi069.stdout:Jul 06 20:40:12 smithi069 bash[185381]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-j 2023-07-06T20:40:12.628 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.j.service' 2023-07-06T20:40:12.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:12.709 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-06T20:40:12.709 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-06T20:40:12.709 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.b 2023-07-06T20:40:13.072 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:40:12 smithi098 systemd[1]: Stopping Ceph mds.b for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:40:13.375 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:40:13 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-b[154398]: 2023-07-06T20:40:13.069+0000 7fe1611e0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:40:13.376 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:40:13 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-b[154398]: 2023-07-06T20:40:13.069+0000 7fe1611e0700 -1 mds.b *** got signal Terminated *** 2023-07-06T20:40:16.435 INFO:journalctl@ceph.mds.b.smithi098.stdout:Jul 06 20:40:16 smithi098 bash[161426]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-b 2023-07-06T20:40:16.623 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.b.service' 2023-07-06T20:40:16.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:16.662 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-06T20:40:16.662 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-06T20:40:16.663 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.e 2023-07-06T20:40:16.878 INFO:journalctl@ceph.mds.e.smithi098.stdout:Jul 06 20:40:16 smithi098 systemd[1]: Stopping Ceph mds.e for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:40:17.375 INFO:journalctl@ceph.mds.e.smithi098.stdout:Jul 06 20:40:17 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-e[153716]: 2023-07-06T20:40:17.051+0000 7f2fa832f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:40:17.376 INFO:journalctl@ceph.mds.e.smithi098.stdout:Jul 06 20:40:17 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-e[153716]: 2023-07-06T20:40:17.051+0000 7f2fa832f700 -1 mds.e *** got signal Terminated *** 2023-07-06T20:40:20.409 INFO:journalctl@ceph.mds.e.smithi098.stdout:Jul 06 20:40:20 smithi098 bash[161562]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-e 2023-07-06T20:40:20.596 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.e.service' 2023-07-06T20:40:20.631 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:20.631 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-06T20:40:20.632 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-06T20:40:20.633 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.h 2023-07-06T20:40:20.876 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:40:20 smithi098 systemd[1]: Stopping Ceph mds.h for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:40:21.376 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:40:21 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-h[154057]: 2023-07-06T20:40:21.010+0000 7f237b2e8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:40:21.376 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:40:21 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-h[154057]: 2023-07-06T20:40:21.010+0000 7f237b2e8700 -1 mds.h *** got signal Terminated *** 2023-07-06T20:40:24.625 INFO:journalctl@ceph.mds.h.smithi098.stdout:Jul 06 20:40:24 smithi098 bash[161702]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-h 2023-07-06T20:40:24.634 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.h.service' 2023-07-06T20:40:24.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:24.669 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-06T20:40:24.669 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-06T20:40:24.670 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.k 2023-07-06T20:40:25.044 INFO:journalctl@ceph.mds.k.smithi098.stdout:Jul 06 20:40:24 smithi098 systemd[1]: Stopping Ceph mds.k for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:40:25.375 INFO:journalctl@ceph.mds.k.smithi098.stdout:Jul 06 20:40:25 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-k[153376]: 2023-07-06T20:40:25.042+0000 7f4e323b5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:40:25.376 INFO:journalctl@ceph.mds.k.smithi098.stdout:Jul 06 20:40:25 smithi098 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-k[153376]: 2023-07-06T20:40:25.042+0000 7f4e323b5700 -1 mds.k *** got signal Terminated *** 2023-07-06T20:40:29.318 INFO:journalctl@ceph.mds.k.smithi098.stdout:Jul 06 20:40:29 smithi098 bash[161849]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-k 2023-07-06T20:40:29.519 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.k.service' 2023-07-06T20:40:29.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:29.554 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-06T20:40:29.554 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-06T20:40:29.555 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.c 2023-07-06T20:40:29.856 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:40:29 smithi156 systemd[1]: Stopping Ceph mds.c for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:40:30.357 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:40:29 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-c[155006]: 2023-07-06T20:40:29.917+0000 7fe4dbf56700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:40:30.357 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:40:29 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-c[155006]: 2023-07-06T20:40:29.917+0000 7fe4dbf56700 -1 mds.c *** got signal Terminated *** 2023-07-06T20:40:32.374 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 06 20:40:32 smithi156 bash[163865]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-c 2023-07-06T20:40:32.564 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.c.service' 2023-07-06T20:40:32.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:32.601 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-06T20:40:32.602 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-06T20:40:32.602 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.f 2023-07-06T20:40:32.857 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:40:32 smithi156 systemd[1]: Stopping Ceph mds.f for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:40:33.357 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:40:32 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-f[154665]: 2023-07-06T20:40:32.975+0000 7fedf76dd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:40:33.357 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:40:32 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-f[154665]: 2023-07-06T20:40:32.975+0000 7fedf76dd700 -1 mds.f *** got signal Terminated *** 2023-07-06T20:40:36.606 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 06 20:40:36 smithi156 bash[164009]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-f 2023-07-06T20:40:36.684 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.f.service' 2023-07-06T20:40:36.717 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:36.718 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-06T20:40:36.718 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-06T20:40:36.718 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.i 2023-07-06T20:40:37.093 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:40:36 smithi156 systemd[1]: Stopping Ceph mds.i for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:40:37.357 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:40:37 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-i[155344]: 2023-07-06T20:40:37.090+0000 7f31f8420700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:40:37.357 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:40:37 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-i[155344]: 2023-07-06T20:40:37.090+0000 7f31f8420700 -1 mds.i *** got signal Terminated *** 2023-07-06T20:40:40.366 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 06 20:40:40 smithi156 bash[164154]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-i 2023-07-06T20:40:40.552 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.i.service' 2023-07-06T20:40:40.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:40.587 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-06T20:40:40.588 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-06T20:40:40.588 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.l 2023-07-06T20:40:40.857 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 06 20:40:40 smithi156 systemd[1]: Stopping Ceph mds.l for e3e8ec0c-1c38-11ee-9b2f-001a4aab830c... 2023-07-06T20:40:41.357 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 06 20:40:40 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-l[154325]: 2023-07-06T20:40:40.956+0000 7f7a9dbe8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-06T20:40:41.357 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 06 20:40:40 smithi156 ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-l[154325]: 2023-07-06T20:40:40.956+0000 7f7a9dbe8700 -1 mds.l *** got signal Terminated *** 2023-07-06T20:40:43.288 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 06 20:40:42 smithi156 bash[164295]: ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c-mds-l 2023-07-06T20:40:43.294 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e3e8ec0c-1c38-11ee-9b2f-001a4aab830c@mds.l.service' 2023-07-06T20:40:43.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-06T20:40:43.327 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-06T20:40:43.328 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c --force --keep-logs 2023-07-06T20:40:46.913 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c --force --keep-logs 2023-07-06T20:40:48.809 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c --force --keep-logs 2023-07-06T20:40:50.694 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-06T20:40:50.772 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-06T20:40:50.808 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-06T20:40:50.837 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-06T20:40:50.839 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148/remote/smithi069/crash 2023-07-06T20:40:50.840 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/crash -- . 2023-07-06T20:40:50.983 INFO:teuthology.orchestra.run.smithi069.stderr:tar: /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-06T20:40:50.984 INFO:teuthology.orchestra.run.smithi069.stderr:tar: Error is not recoverable: exiting now 2023-07-06T20:40:50.987 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148/remote/smithi098/crash 2023-07-06T20:40:50.988 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/crash -- . 2023-07-06T20:40:51.135 INFO:teuthology.orchestra.run.smithi098.stderr:tar: /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-06T20:40:51.135 INFO:teuthology.orchestra.run.smithi098.stderr:tar: Error is not recoverable: exiting now 2023-07-06T20:40:51.137 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148/remote/smithi156/crash 2023-07-06T20:40:51.138 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/crash -- . 2023-07-06T20:40:51.221 INFO:teuthology.orchestra.run.smithi156.stderr:tar: /var/lib/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-06T20:40:51.222 INFO:teuthology.orchestra.run.smithi156.stderr:tar: Error is not recoverable: exiting now 2023-07-06T20:40:51.223 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-06T20:40:51.223 DEBUG:teuthology.orchestra.run.smithi069:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-06T20:40:51.302 INFO:teuthology.orchestra.run.smithi069.stderr:grep: /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph.log: No such file or directory 2023-07-06T20:40:51.308 INFO:tasks.cephadm:Compressing logs... 2023-07-06T20:40:51.308 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-06T20:40:51.312 DEBUG:teuthology.orchestra.run.smithi098:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-06T20:40:51.315 DEBUG:teuthology.orchestra.run.smithi156:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-06T20:40:51.342 INFO:teuthology.orchestra.run.smithi156.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-06T20:40:51.342 INFO:teuthology.orchestra.run.smithi156.stderr:: No such file or directory 2023-07-06T20:40:51.342 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.5.log 2023-07-06T20:40:51.343 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.2.log 2023-07-06T20:40:51.343 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.11.log 2023-07-06T20:40:51.344 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.5.log: /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mgr.z.log 2023-07-06T20:40:51.344 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.8.log 2023-07-06T20:40:51.345 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.f.log 2023-07-06T20:40:51.345 INFO:teuthology.orchestra.run.smithi156.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-06T20:40:51.345 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.c.log 2023-07-06T20:40:51.351 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.f.log: /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.c.log: 93.2% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mgr.z.log.gz 2023-07-06T20:40:51.352 INFO:teuthology.orchestra.run.smithi098.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-06T20:40:51.352 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-06T20:40:51.354 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.i.log 2023-07-06T20:40:51.355 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mon.c.log 2023-07-06T20:40:51.355 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.10.log 2023-07-06T20:40:51.356 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.h.log 2023-07-06T20:40:51.356 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.e.log 2023-07-06T20:40:51.357 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.10.log: /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.4.log 2023-07-06T20:40:51.358 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.1.log 2023-07-06T20:40:51.358 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.k.log 2023-07-06T20:40:51.358 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-volume.log 2023-07-06T20:40:51.359 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.7.log 2023-07-06T20:40:51.359 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mon.b.log 2023-07-06T20:40:51.359 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.b.log 2023-07-06T20:40:51.359 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mgr.y.log 2023-07-06T20:40:51.362 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.b.log: /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mgr.y.log: 94.3% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.k.log.gz 2023-07-06T20:40:51.364 INFO:teuthology.orchestra.run.smithi098.stderr: 93.1% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mgr.y.log.gz 2023-07-06T20:40:51.365 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.i.log: /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-volume.log 2023-07-06T20:40:51.375 INFO:teuthology.orchestra.run.smithi069.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-06T20:40:51.375 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-06T20:40:51.376 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.0.log 2023-07-06T20:40:51.376 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.g.log 2023-07-06T20:40:51.378 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.a.log 2023-07-06T20:40:51.379 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.6.log 2023-07-06T20:40:51.381 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.3.log 2023-07-06T20:40:51.382 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.d.log 2023-07-06T20:40:51.383 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.3.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-06T20:40:51.384 INFO:teuthology.orchestra.run.smithi069.stderr: 94.4% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.g.log.gz 2023-07-06T20:40:51.384 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mgr.x.log 2023-07-06T20:40:51.385 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.9.log 2023-07-06T20:40:51.386 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.l.log 2023-07-06T20:40:51.386 INFO:teuthology.orchestra.run.smithi069.stderr: 94.4% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.a.log.gz 2023-07-06T20:40:51.387 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-volume.log 2023-07-06T20:40:51.388 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.j.log 2023-07-06T20:40:51.398 INFO:teuthology.orchestra.run.smithi156.stderr: 91.4% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-volume.log.gz 2023-07-06T20:40:51.399 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-volume.log: /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.j.log: 94.4% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.d.log.gz 2023-07-06T20:40:51.399 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mon.a.log 2023-07-06T20:40:51.399 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mon.a.log: 94.4% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.j.log.gz 2023-07-06T20:40:51.399 INFO:teuthology.orchestra.run.smithi069.stderr: 91.5% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-volume.log.gz 2023-07-06T20:40:51.406 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.l.log: 94.4% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.l.log.gz 2023-07-06T20:40:51.553 INFO:teuthology.orchestra.run.smithi156.stderr: 93.4% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.f.log.gz 2023-07-06T20:40:51.575 INFO:teuthology.orchestra.run.smithi098.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-06T20:40:51.601 INFO:teuthology.orchestra.run.smithi069.stderr: 90.1% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mgr.x.log.gz 2023-07-06T20:40:51.929 INFO:teuthology.orchestra.run.smithi156.stderr: 91.9% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mon.c.log.gz 2023-07-06T20:40:51.953 INFO:teuthology.orchestra.run.smithi156.stderr: 94.1% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.i.log.gz 2023-07-06T20:40:51.981 INFO:teuthology.orchestra.run.smithi156.stderr: 95.6% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.c.log.gz 2023-07-06T20:40:52.155 INFO:teuthology.orchestra.run.smithi098.stderr: 91.4% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-volume.log.gz 2023-07-06T20:40:52.965 INFO:teuthology.orchestra.run.smithi069.stderr: 90.4% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mon.a.log.gz 2023-07-06T20:40:54.806 INFO:teuthology.orchestra.run.smithi098.stderr: 92.0% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mon.b.log.gz 2023-07-06T20:41:06.593 INFO:teuthology.orchestra.run.smithi069.stderr: 93.5% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.9.log.gz 2023-07-06T20:41:12.733 INFO:teuthology.orchestra.run.smithi156.stderr: 93.6% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.8.log.gz 2023-07-06T20:41:15.635 INFO:teuthology.orchestra.run.smithi156.stderr: 93.6% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.2.log.gz 2023-07-06T20:41:18.452 INFO:teuthology.orchestra.run.smithi069.stderr: 93.6% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.6.log.gz 2023-07-06T20:41:18.857 INFO:teuthology.orchestra.run.smithi156.stderr: 93.6% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.5.log.gz 2023-07-06T20:41:21.057 INFO:teuthology.orchestra.run.smithi156.stderr: 93.7% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.11.log.gz 2023-07-06T20:41:21.059 INFO:teuthology.orchestra.run.smithi156.stderr: 2023-07-06T20:41:21.059 INFO:teuthology.orchestra.run.smithi156.stderr:real 0m29.728s 2023-07-06T20:41:21.059 INFO:teuthology.orchestra.run.smithi156.stderr:user 1m41.009s 2023-07-06T20:41:21.059 INFO:teuthology.orchestra.run.smithi156.stderr:sys 0m3.760s 2023-07-06T20:41:23.833 INFO:teuthology.orchestra.run.smithi069.stderr: 93.6% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.3.log.gz 2023-07-06T20:41:25.140 INFO:teuthology.orchestra.run.smithi069.stderr: 93.6% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.0.log.gz 2023-07-06T20:41:25.143 INFO:teuthology.orchestra.run.smithi069.stderr: 2023-07-06T20:41:25.143 INFO:teuthology.orchestra.run.smithi069.stderr:real 0m33.795s 2023-07-06T20:41:25.144 INFO:teuthology.orchestra.run.smithi069.stderr:user 1m41.133s 2023-07-06T20:41:25.144 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m8.852s 2023-07-06T20:41:39.145 INFO:teuthology.orchestra.run.smithi098.stderr: 94.3% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.h.log.gz 2023-07-06T20:42:03.096 INFO:teuthology.orchestra.run.smithi098.stderr: 93.5% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.4.log.gz 2023-07-06T20:42:14.758 INFO:teuthology.orchestra.run.smithi098.stderr: 93.6% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.10.log.gz 2023-07-06T20:42:27.427 INFO:teuthology.orchestra.run.smithi098.stderr: 93.6% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.1.log.gz 2023-07-06T20:42:30.458 INFO:teuthology.orchestra.run.smithi098.stderr: 93.7% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-osd.7.log.gz 2023-07-06T20:45:11.777 INFO:teuthology.orchestra.run.smithi098.stderr: 97.3% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.e.log.gz 2023-07-06T20:46:50.769 INFO:teuthology.orchestra.run.smithi098.stderr: 97.8% -- replaced with /var/log/ceph/e3e8ec0c-1c38-11ee-9b2f-001a4aab830c/ceph-mds.b.log.gz 2023-07-06T20:46:50.779 INFO:teuthology.orchestra.run.smithi098.stderr: 2023-07-06T20:46:50.779 INFO:teuthology.orchestra.run.smithi098.stderr:real 5m59.447s 2023-07-06T20:46:50.779 INFO:teuthology.orchestra.run.smithi098.stderr:user 6m16.007s 2023-07-06T20:46:50.780 INFO:teuthology.orchestra.run.smithi098.stderr:sys 0m19.907s 2023-07-06T20:46:50.780 INFO:tasks.cephadm:Archiving logs... 2023-07-06T20:46:50.781 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148/remote/smithi069/log 2023-07-06T20:46:50.782 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-06T20:47:02.888 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148/remote/smithi098/log 2023-07-06T20:47:02.890 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-06T20:47:34.354 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148/remote/smithi156/log 2023-07-06T20:47:34.355 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-06T20:47:46.934 INFO:tasks.cephadm:Removing cluster... 2023-07-06T20:47:46.935 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c --force 2023-07-06T20:47:48.366 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c --force 2023-07-06T20:47:49.804 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e3e8ec0c-1c38-11ee-9b2f-001a4aab830c --force 2023-07-06T20:47:50.715 INFO:tasks.cephadm:Removing cephadm ... 2023-07-06T20:47:50.715 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-06T20:47:50.763 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-06T20:47:50.805 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-06T20:47:50.824 INFO:tasks.cephadm:Teardown complete 2023-07-06T20:47:50.824 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-06T20:47:50.837 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-06T20:47:50.838 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-06T20:47:50.843 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-06T20:47:50.848 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-06T20:47:51.000 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-06T20:47:51.012 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-06T20:47:51.012 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T20:47:51.018 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T20:47:51.021 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T20:47:51.037 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-07-06T20:47:51.037 INFO:teuthology.orchestra.run.smithi098.stderr:bash: ntpq: command not found 2023-07-06T20:47:51.041 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T20:47:51.041 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-07-06T20:47:51.041 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 +1710us[+1723us] +/- 73ms 2023-07-06T20:47:51.041 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 -1129us[-1115us] +/- 51ms 2023-07-06T20:47:51.041 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -98us[ -85us] +/- 79ms 2023-07-06T20:47:51.042 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:47:51.056 INFO:teuthology.orchestra.run.smithi069.stderr:bash: ntpq: command not found 2023-07-06T20:47:51.057 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T20:47:51.057 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2023-07-06T20:47:51.057 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 +1698us[+1686us] +/- 73ms 2023-07-06T20:47:51.057 INFO:teuthology.orchestra.run.smithi098.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -1129us[-1141us] +/- 51ms 2023-07-06T20:47:51.058 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -120us[ -132us] +/- 79ms 2023-07-06T20:47:51.058 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:47:51.064 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T20:47:51.064 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2023-07-06T20:47:51.064 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 +1736us[+1756us] +/- 73ms 2023-07-06T20:47:51.065 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -1102us[-1082us] +/- 51ms 2023-07-06T20:47:51.065 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 -79us[ -79us] +/- 79ms 2023-07-06T20:47:51.065 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T20:47:51.066 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-06T20:47:51.097 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-06T20:47:51.098 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-06T20:47:51.134 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-06T20:47:51.341 DEBUG:teuthology.orchestra.run.smithi098:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-06T20:47:51.525 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-06T20:47:51.620 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T20:47:51.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:47:51.696 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T20:47:51.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:47:51.732 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T20:47:51.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:47:51.765 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-06T20:47:51.777 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-06T20:47:51.802 INFO:teuthology.task.internal:Duration was 3685.405684 seconds 2023-07-06T20:47:51.803 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-06T20:47:51.835 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-06T20:47:51.836 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-06T20:47:51.841 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-06T20:47:51.843 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-06T20:47:51.890 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T20:47:51.912 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T20:47:51.952 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T20:47:52.371 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-06T20:47:52.372 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2023-07-06T20:47:52.373 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-06T20:47:52.432 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2023-07-06T20:47:52.433 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-06T20:47:52.457 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2023-07-06T20:47:52.457 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-06T20:47:52.482 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-06T20:47:52.483 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-06T20:47:52.487 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-06T20:47:52.502 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-06T20:47:52.581 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-06T20:47:52.581 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-06T20:47:52.588 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-06T20:47:52.591 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-06T20:47:52.744 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-06T20:47:52.757 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-06T20:47:52.757 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-06T20:47:52.824 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-06T20:47:52.883 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-06T20:47:52.911 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-06T20:47:52.925 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-06T20:47:52.930 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-06T20:47:52.932 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-06T20:47:52.966 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2023-07-06T20:47:52.978 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2023-07-06T20:47:52.992 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2023-07-06T20:47:53.028 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-06T20:47:53.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:47:53.071 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-06T20:47:53.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:47:53.088 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-06T20:47:53.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T20:47:53.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-06T20:47:53.124 INFO:teuthology.task.internal:Transferring archived files... 2023-07-06T20:47:53.127 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148/remote/smithi069 2023-07-06T20:47:53.127 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-06T20:47:53.207 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148/remote/smithi098 2023-07-06T20:47:53.208 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-06T20:47:53.247 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148/remote/smithi156 2023-07-06T20:47:53.247 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-06T20:47:53.288 INFO:teuthology.task.internal:Removing archive directory... 2023-07-06T20:47:53.288 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-06T20:47:53.293 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-06T20:47:53.296 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-06T20:47:53.336 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-06T20:47:53.353 INFO:teuthology.task.internal:Not uploading archives. 2023-07-06T20:47:53.354 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-06T20:47:53.387 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-06T20:47:53.387 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-06T20:47:53.392 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-06T20:47:53.395 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-06T20:47:53.409 INFO:teuthology.orchestra.run.smithi098.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 6 20:47 /home/ubuntu/cephtest 2023-07-06T20:47:53.411 INFO:teuthology.orchestra.run.smithi156.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 6 20:47 /home/ubuntu/cephtest 2023-07-06T20:47:53.431 INFO:teuthology.orchestra.run.smithi069.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 6 20:47 /home/ubuntu/cephtest 2023-07-06T20:47:53.434 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-06T20:47:53.451 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-06T20:47:53.505 INFO:teuthology.nuke:Checking targets against current locks 2023-07-06T20:47:53.539 DEBUG:teuthology.nuke:shortname: smithi069 2023-07-06T20:47:53.539 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T20:47:53.566 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 19:34:37.258375', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-06T20:47:53.570 DEBUG:teuthology.nuke:shortname: smithi098 2023-07-06T20:47:53.571 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T20:47:53.598 DEBUG:teuthology.nuke:shortname: smithi156 2023-07-06T20:47:53.598 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T20:47:53.601 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 19:34:37.259548', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-06T20:47:53.625 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/7328148', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 19:34:37.257099', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-06T20:47:53.639 INFO:teuthology.orchestra.console:Power off smithi069 2023-07-06T20:47:53.640 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-06T20:47:53.664 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:47:53.683 INFO:teuthology.orchestra.console:Power off smithi098 2023-07-06T20:47:53.684 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-06T20:47:53.705 INFO:teuthology.orchestra.console:Power off smithi156 2023-07-06T20:47:53.706 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-06T20:47:53.718 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:47:53.736 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:47:57.719 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:47:57.751 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:47:57.765 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:48:01.832 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:48:01.865 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:48:01.878 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:48:02.077 INFO:teuthology.orchestra.console:Power off for smithi098 completed 2023-07-06T20:48:02.092 INFO:teuthology.orchestra.console:Power off for smithi156 completed 2023-07-06T20:48:05.948 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T20:48:06.159 INFO:teuthology.orchestra.console:Power off for smithi069 completed 2023-07-06T20:48:06.265 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} duration: 3685.4056844711304 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-07-06T20:48:06.266 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-06T20:48:06.363 INFO:teuthology.run:FAIL